Configuration

IMPERANDI configuration is done through manifests and hooks.

A manifest is a JSON document that declares dataset-specific settings such as identifier extraction, segmentation tasks, post-processing, and radiomics options.
A hook is a Python callable referenced by the manifest when declarative JSON is not enough, for example to normalize patient identifiers or derive cohort columns from existing values.

Pass a built-in manifest name (generic or operandi) or a path to a JSON file:

imperandi ingest --root_path ./dicom --manifest generic
imperandi segment ./nifti_index.csv --manifest ./site-a.json

Built-ins live under src/imperandi/datasets_config/manifests/. Treat them as examples; keep institution-specific configuration in a reviewed, versioned file rather than editing package defaults in place. This Markdown page is the reference for how configuration is expressed through manifest JSON plus hook functions.

How configuration works

In practice, configuration usually follows this flow:

start from a built-in manifest or a project-owned copy;
edit manifest keys for parsing, segmentation, and radiomics behavior;
point manifest hook entries at Python functions when you need custom logic;
run IMPERANDI with --manifest your_config.json.

Use manifests for declarative settings that can live safely in versioned JSON. Use hooks for executable logic such as parsing institution-specific identifiers, mapping cohort metadata, or other transformations that need Python code.

Manifest structure

{
  "dataset_name": "site-a",
  "id_extraction": {
    "source": "auto",
    "force_dicom_read": false,
    "patient_key": {"from_tag": "PatientID", "fallback": "path"},
    "study_id": {"from_tag": "StudyInstanceUID", "fallback": "path"},
    "series_id": {"from_tag": "SeriesInstanceUID", "fallback": "path"}
  },
  "id_standardization": {
    "hook_module": "datasets_config.hooks.generic",
    "function": "standardize_patient_key"
  },
  "derived_columns": [
    {
      "hook_module": "datasets_config.hooks.operandi",
      "function": "extract_from_patient_key",
      "from_column": "patient_key",
      "join_mode": "missing_only"
    }
  ],
  "segmentation": {
    "backend": "totalsegmentator",
    "tasks": [
      {"task": "total", "extra": {"roi_subset": ["liver"]}},
      {
        "task": "liver_lesions",
        "output": "liver_tumor",
        "fetch_output": "liver_lesions"
      }
    ],
    "postprocess": {
      "merge_keys": ["liver", "liver_tumor"],
      "output": "liver",
      "radius_mm": 5.0,
      "largest_cc": true,
      "fill_holes": true,
      "close": true
    }
  },
  "radiomics": {
    "pyradiomics": {
      "setting": {"binWidth": 25},
      "imageType": {"Original": {}}
    },
    "filters": {"totalseg_phase": ["portal_venous", "arterial_late"]}
  }
}

Customize a manifest file

The usual workflow is:

copy src/imperandi/datasets_config/manifests/generic.json or src/imperandi/datasets_config/manifests/operandi.json;
save it as a project-owned file such as site-a.json;
edit the JSON keys that match your dataset;
pass the file path with --manifest ./site-a.json.

The easiest way to read the manifest is by configuration area:

Identity and cohort metadata

dataset_name Use this as a short, stable label for the configuration. It is mainly for readability and review, so names such as site-a, operandi_v2, or external_validation are usually better than vague labels.

id_extraction This controls how raw patient, study, and series identifiers are obtained. The usual fields are source (auto, tags, or path), force_dicom_read, and the per-ID blocks patient_key, study_id, and series_id with from_tag and fallback. Typical from_tag values are DICOM keywords such as PatientID, PatientName, StudyInstanceUID, and SeriesInstanceUID. Typical fallback usage is "path" when a tag is missing or unreliable. Keep this section aligned with the parse or ingest CLI options you actually run: --id_source, --force_dicom_read, --patient_key_from, --study_id_from, and --series_id_from.

id_standardization Use this when raw identifiers need cleanup after extraction. It points to a hook that rewrites patient_key into your canonical cohort ID, for example by removing prefixes, normalizing zero-padding, or extracting the parts of the string that matter for downstream grouping.

derived_columns Use this when you want to compute extra cohort columns from an existing value. Each entry can define hook_module, function, from_column, and join_mode (missing_only or overwrite). This is a good place to derive fields such as center, source, tumor_type, or other site-specific metadata from a patient key or similar identifier.

Segmentation

segmentation.backend This selects the segmentation engine. In the current implementation, the supported value is totalsegmentator.

segmentation.tasks This section defines which TotalSegmentator runs happen and how IMPERANDI names their outputs. Each task entry needs at least task. Common optional keys are extra for backend kwargs, output or outputs for the logical mask names you want in IMPERANDI, and fetch_output or fetch_outputs when the backend-produced filenames differ from your preferred output names. Inside extra, common runtime options are roi_subset, roi_subset_robust, fast, and fastest, but other supported TotalSegmentator kwargs can also be passed through. If you use roi_subset or roi_subset_robust, IMPERANDI can infer mask names from those class names, but explicit output names are usually clearer and more stable for downstream CSV columns. Official references: TotalSegmentator subtasks guide and class details. To inspect the exact capabilities of your installed version, run totalseg_info --list-tasks and totalseg_info --classes -ta total.

Segmentation Mask Postprocessing

segmentation.postprocess Use this section when several masks should be merged or cleaned after segmentation. merge_keys is required when post-processing is enabled and should name the logical outputs to combine, such as liver and liver_tumor or their mask_* column equivalents. Common options are output for the merged mask name, radius_mm for morphological closing, close, fill_holes, and largest_cc for cleanup, and on_failure with warn_only or fail.

Radiomics

radiomics.pyradiomics Use this when you want the manifest itself to carry your feature extraction settings instead of referencing a separate YAML file. Common top-level sections are setting, imageType, and featureClass, plus filter-specific keys such as LoG sigma. This is the manifest equivalent of a PyRadiomics parameter file and is useful when you want one versioned configuration file to describe both cohort logic and radiomics behavior. Official guide: PyRadiomics customization and parameter file docs.

radiomics.filters Use this when radiomics should run only on a subset of rows. The format is column_name -> [allowed_value, ...]. Typical columns include totalseg_phase, phase, center, source, or any other cohort column already present in the CSV. This is the manifest equivalent of repeated CLI filters such as --filter totalseg_phase=portal_venous,arterial_late.

Example:

{
  "dataset_name": "site-a",
  "id_extraction": {
    "source": "auto",
    "patient_key": {"from_tag": "PatientID", "fallback": "path"},
    "study_id": {"from_tag": "StudyInstanceUID", "fallback": "path"},
    "series_id": {"from_tag": "SeriesInstanceUID", "fallback": "path"}
  },
  "id_standardization": {
    "hook_module": "datasets_config.hooks.site_a",
    "function": "standardize_patient_key"
  },
  "radiomics": {
    "filters": {"totalseg_phase": ["portal_venous"]}
  }
}

Keep custom manifests outside package defaults so they can be reviewed, versioned, and reused across runs.

Hooks

id_standardization resolves a callable below the imperandi package and applies it to the raw patient key. A derived_columns list can similarly call functions that return mappings of extra fields. Each derived entry names a from_column and may set join_mode to missing_only (default) or overwrite.

At runtime, IMPERANDI imports hooks as:

imperandi.<hook_module>.<function>

That means this manifest block:

{
  "hook_module": "datasets_config.hooks.generic",
  "function": "standardize_patient_key"
}

loads imperandi.datasets_config.hooks.generic.standardize_patient_key.

`id_standardization` hooks

An id_standardization hook receives one raw patient_key value and should return the normalized value to write back into patient_key.

IMPERANDI preserves the original value in _patient_key_raw.
If a non-empty raw key becomes empty or null, the row is flagged with patient_key_std_failed.
The built-in generic hook extracts numeric groups and joins them with -.
The built-in operandi hook applies project-specific parsing rules for that dataset.

`derived_columns` hooks

A derived_columns hook receives the value from from_column for each row and returns a mapping or pandas.Series of new fields to join back into the table.

join_mode: "missing_only" adds only columns that do not already exist.
join_mode: "overwrite" replaces existing columns with the derived values.
The built-in operandi hook extract_from_patient_key derives center, source, and tumor_type from the standardized patient key.

Example:

{
  "derived_columns": [
    {
      "hook_module": "datasets_config.hooks.operandi",
      "function": "extract_from_patient_key",
      "from_column": "patient_key",
      "join_mode": "missing_only"
    }
  ]
}

Writing a custom hook

Create an importable module under src/imperandi/, for example src/imperandi/datasets_config/hooks/site_a.py:

def standardize_patient_key(value):
    if value is None:
        return None
    return str(value).strip().upper()

Then reference it from your manifest:

{
  "id_standardization": {
    "hook_module": "datasets_config.hooks.site_a",
    "function": "standardize_patient_key"
  }
}

Custom hooks are executable Python, not passive configuration. Review them as code, test them against malformed and missing identifiers, and never load an untrusted manifest that points to untrusted modules.

Precedence

For radiomics, manifest radiomics.filters are merged with CLI --filter values; when both specify the same column, the manifest values win. --skip_filter disables both CLI and manifest filters. Manifest radiomics.pyradiomics settings take precedence when both a manifest settings object and --pyradiomics_settings are supplied.

For parse and ingest identity controls, the runtime CLI flags remain the authoritative switches to review: --id_source, --force_dicom_read, --patient_key_from, --study_id_from, and --series_id_from.

Validation advice

Before a full run:

run each command with --dry-run;
parse a small representative sample;
compare raw and standardized identifiers;
verify expected mask column names;
confirm radiomics filters retain the intended phases.