Standardization

Marker intensities in multiplexed images differ widely between imaging technologies, cohorts, and individual images. To make them comparable for model training, MultiplexImagingDataset can automatically apply a standardization procedure to every tissue and tile as it is loaded.

Available Methods

The standardization method is controlled via the standardization parameter of the dataset class. Its value specifies both the name of the method and the relative path to the precomputed statistics the method uses. This path is resolved below <modality>/<resolution>/standardization/ of the dataset.

For example, the value quantile_clipping/uq_0.99_image selects the quantile clipping method and loads its statistics from <modality>/<resolution>/standardization/quantile_clipping/uq_0.99_image/.

The following standardization methods are available in spora-io:

standardization	Transform
identity	Converts the image to a tensor without changing its values. No statistics are required.
quantile_clipping/uq_0.99_image	Clips each channel at its 99th percentile, computed per image, and rescales the clipped values linearly to the range [0, 1].
quantile_clipping_log1p/uq_0.99_image	Clips each channel in the same way and then applies a log1p transform instead of the linear rescaling.

The lower clipping bound is 0 unless lower quantiles were computed. As a final step, both methods standardize each channel to zero mean and unit variance, using the channel means and standard deviations from the precomputed statistics. This step is enabled by default and can be disabled with use_mean_std=False.

The second part of the value, such as uq_0.99_image or uq_0.99_global, records the upper clipping quantile and whether it was computed per image or across the whole dataset. Because the means and standard deviations are computed on the transformed values, statistics computed with different clipping settings are not interchangeable and are therefore stored in separate directories.

Internally, the factory function build_standardizer() resolves the standardization value to its standardizer class. To add a custom method, see Adding a Dataset.

Computing the Statistics

All methods except identity require statistics that are computed in advance with scripts/compute_standardization_stats.py. The statistics must be computed once for each combination of dataset, modality, resolution, and method. Compute Multiplex Standardization Statistics describes the script, its arguments, the output layout, and the settings used by the spora models.

Loading a Standardized Dataset

Once the statistics are available, pass the method to the dataset class:

from spora_io import MultiplexImagingDataset

dataset = MultiplexImagingDataset(
    name="schurch2020coordinated",
    modality="codex",
    standardization="quantile_clipping/uq_0.99_image",
    resolution=1.0,
    tile_size=224,
)

H&E Normalization

H&E images do not require precomputed statistics. HEImagingDataset divides the pixel intensities by 255 and then normalizes them with the ImageNet or HIBOU mean and standard deviation. The mean_std_type parameter selects this normalization, which can also be disabled.