Cross-cohort Instance Segmentation

The pipeline run_segmentation_tasks evaluates how well a model segments individual cell instances in datasets it was not trained on.

Important

To assess cross-cohort generalization, it is the responsibility of the user to ensure that the segmentation model has not seen the target datasets during supervised training.

Running the Benchmark

python -m spora_bench.benchmarks.run_segmentation_tasks \
  model_config=<path-to-model-config> \
  datasets_config=<path-to-datasets-config>

No benchmarks_config is required. Batched submission of datasets is possible via lists or wildcards. For example, Cellpose on cords2024cancer:

python -m spora_bench.benchmarks.run_segmentation_tasks \
  model_config=configs/models/cellpose.yaml \
  datasets_config=configs/datasets/virtues/cords2024cancer.yaml

Evaluation

For every test tissue, predicted instances are matched to the reference cell masks of spora [data] at IoU thresholds from 0.0 to 1.0 in steps of 0.1; images larger than 1024 pixels are matched in 1024-pixel tiles, and tissues smaller than 256 × 256 pixels are skipped. Precision, recall, and F1 score per threshold are saved to output_dir/<model_name>/results/<dataset>_instance_segmentation_results.parquet.

The datasets and configs used in the paper are listed in the Task Zoo.