biapy.data.norm

Normalization utilities for image and mask data in deep learning workflows.

This module provides the Normalization class, which supports various normalization strategies for images and masks, including percentile clipping, scaling, and zero-mean unit-variance normalization. It is designed to work with both NumPy arrays and PyTorch tensors, and integrates with BiaPy’s DatasetFile for reproducible normalization statistics.

biapy.data.norm.normalize_image(img: ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, norm_module: Dict, apply_norm: bool = True) → Tuple[ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, Dict][source]

Compute and set normalization statistics from a single image.

Parameters:

  • img (NDArray | torch.Tensor) (Input image.) – E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

  • norm_module (dict) –

    Normalization module dict with the normalization parameters. Expected keys are:
    • type, str: type of normalization to apply to the image. Expected values are:

      • div: normalize the image by dividing it by a value. The value can be either 255 or 65535 depending on the maximum value of the image or the maximum and minimum values of the image if div_using_max_and_scale is True.

      • scale_range: normalize the image by using the following operation: results = ((x - x_min)/(x_max - x_min)) * (out_max - out_min). In this case, the values used to do the normalization are computed from the data itself.

      • zero_mean_unit_variance: apply zero-mean, unit-variance normalization to the image.

    • percentile_clip, bool: whether to apply percentile clipping to the image before applying the normalization. If True, the values used for clipping

      will be computed from the data itself by using the percentiles specified in per_lower_bound and per_upper_bound or the values specified in lower_bound_val and upper_bound_val.

    • out_dtype, str: output dtype to convert the image to after applying the normalization. Expected values are: β€˜uint8’, β€˜uint16’ or β€˜float32’.

    • For zero_mean_unit_variance type, expected keys are:

      • mean, list of float or single float: mean to use in the normalization. If a single float is provided, it will be used for all channels. If None, the mean of the data will be used.

      • std, list of float or single float: standard deviation to use in the normalization. If a single float is provided, it will be used for all channels. If None, the std of the data will be used.

biapy.data.norm.normalize_mask(mask: ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, norm_module: dict, ignore_index: int | None = None, n_classes: int = 1, instance_problem: bool = False, apply_norm: bool = True, is_training: bool = True) → Tuple[ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, dict][source]

Apply normalization to a mask.

Parameters:

  • mask (NDArray | torch.Tensor) – Mask to normalize. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

  • norm_module (dict) –

    Normalization module dict with the normalization parameters. Expected keys are:
    • mask_norm, str: type of normalization to apply to the mask. Expected values are:

      • as_mask: apply normalization as if the mask were a mask. This means that the function will check if the channels of the mask are binary or not and if they need to be divided by 255 (e.g. if they are in 255 instead of 1). The function will also check if there are non-binary channels (e.g. distance transform channel) and set them as non-binary in the normalization information. This is essential to know how to handle the data in other parts of the pipeline such as during data augmentation.

      • as_image: apply normalization as if the mask were an image. This means that the same normalization specified in norm_module for images will be applied to the mask.

  • ignore_index (Optional[int]) – Value of the pixels to ignore when normalizing. If None, it will not be considered that there are pixels to ignore.

  • n_classes (int) – Number of classes in the problem. It is used to check if the mask channels are binary or not. If there are more than 2 classes and it is an instance segmentation problem, it is expected that there is a channel per class plus one additional channel for the instance ids, and the function will check that the channel with the instance ids is not binary.

  • instance_problem (bool) – Whether it is an instance segmentation problem or not. It is used to check if the mask channels are binary or not. If there are more than 2 classes and it is an instance segmentation problem, it is expected that there is a channel per class plus one additional channel for the instance ids, and the function will check that the channel with the instance ids is not binary.

  • apply_norm (bool) – Whether to apply the normalization or just compute the normalization information. If False, the function will return the original mask and the computed normalization information without applying the normalization.

  • is_training (bool) – Whether the normalization is being applied in training or not. If False, the normalization will be applied as if the mask were an image, as we do not want to apply the normalization as if it were a mask in test/validation as it could be that the model is expecting the mask to be normalized as an image in test/validation if the normalization information was computed from an image.

Returns:

  • mask (3D/4D Numpy array or torch.Tensor) – Y element normalized. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

  • norm_info (dict) – Normalization information computed from the mask. It contains the original dtype of the mask and information about which channels are binary or not and if they need to be divided by 255 or not.

biapy.data.norm.update_mask_norm_info(old_mask_norm_info: Dict, new_mask_norm_info: Dict) → Dict[source]

Update the mask normalization information by replacing the values of the old mask normalization information with the ones of the new mask normalization information when they are more restrictive. For example:

  • orig_dtype: float will be set instead of int, as it is more restrictive.

  • For each channel:

    • type: β€œno_bin” will be set instead of β€œbin” or β€œclasses”, and β€œclasses” will be set instead of β€œbin”, as they are more restrictive.

    • div: True will be set instead of False, as it is more restrictive.

Parameters:

  • old_mask_norm_info (dict) – Old mask normalization information to update.

  • new_mask_norm_info (dict) – New mask normalization information to update the old one with.

Returns:

new_norm – Updated mask normalization information.

Return type:

dict

biapy.data.norm.percentile_clip(data: ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, per_lower_bound: float | None = None, per_upper_bound: float | None = None, lower_bound_val: float | None = None, upper_bound_val: float | None = None, apply_norm: bool = True) → Tuple[ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, float, float][source]

Percentile clipping.

Parameters:

  • data (NDArray | torch.Tensor) (Input data.) – Data to normalize. E.g. (y, x) in 2D and (z, y, x) in 3D.

  • per_lower_bound (Optional[float]) – Lower bound percentile to use for clipping. Should be between 0 and 100. If None, lower_bound_val should be provided.

  • per_upper_bound (Optional[float]) – Upper bound percentile to use for clipping. Should be between 0 and 100. If None, upper_bound_val should be provided.

  • lower_bound_val (Optional[float]) – Lower bound value to use for clipping. If None, per_lower_bound should be provided.

  • upper_bound_val (Optional[float]) – Upper bound value to use for clipping. If None, per_upper_bound should be provided.

  • apply_norm (bool) – Whether to apply the percentile clipping or just compute the lower and upper bound values. If False, the function will return the original data and the computed lower and upper bound values without applying the clipping.

Returns:

  • data (3D/4D Numpy array or torch.Tensor) – Clipped data if apply_norm is True. E.g. (y, x) in 2D and (z, y, x) in 3D.

  • x_lwrs (float) – Lower bound used for clipping.

  • x_uprs (float) – Upper bound used for clipping.

biapy.data.norm.torch_percentile(data: Tensor, q: float) → int | float[source]

Return the q-th percentile of the flattened input tensor’s data.

Copied from: https://gist.github.com/sailfish009/28b54c8aa6398148a6358b8f03c0b611

Parameters:

  • data (torch.Tensor) (Input tensor.) – Data to normalize. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

  • q (float, optional) – Percentile to compute, which must be between 0 and 100 inclusive.

Returns:

int | float

Return type:

Percentile value.

biapy.data.norm.norm_range01(data: ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, div_using_max_and_scale: bool, max_val_to_div: int | float | None, min_val_to_div: int | float | None, apply_norm: bool = True, eps: float = 1e-06) → Tuple[ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, float, float][source]

Normalize given data by dividing it by a value.

Parameters:

  • data (3D/4D Numpy array or torch.Tensor) – Data to normalize. E.g. (y, x) in 2D and (z, y, x) in 3D.

  • div_using_max_and_scale (bool) – Whether to normalize the data by doing a division (when it is False) or by using the following operation (when it is True): results = ((x - x_min)/(x_max - x_min)) * (out_max - out_min).

  • max_val_to_div (int or float or None) – Maximum value to use to divide the data. If not provided it will be computed from the data itself. It is 255 or 65535 if div_using_max_and_scale is False and the maximum value of the data if div_using_max_and_scale is True.

  • min_val_to_div (int or float or None) – Minimum value to use to divide the data. If not provided it will be computed from the data itself. It is 0 if div_using_max_and_scale is False and the minimum value of the data if div_using_max_and_scale is True.

  • apply_norm (bool) – Whether to apply the normalization or just compute the values to do it. If False, the function will return the original data and the computed values without applying the normalization.

  • eps (float) – Small value to add to the denominator to prevent division by zero when normalizing by using the maximum and minimum values of the data.

Returns:

  • data (3D/4D Numpy array or torch.Tensor) – Normalized data if apply_norm is True. E.g. (y, x) in 2D and (z, y, x) in 3D.

  • max_val_to_div (float) – Maximum value used to divide the data. It is 255 or 65535 if div_using_max_and_scale is False and the maximum value of the data if div_using_max_and_scale is True.

  • min_val_to_div (float) – Minimum value used to divide the data. It is 0 if div_using_max_and_scale is False and the minimum value of the data if div_using_max_and_scale is True.

biapy.data.norm.zero_mean_unit_variance_normalization(data: ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, mean: float | None = None, std: float | None = None, apply_norm: bool = True, eps: float = 1e-06) → Tuple[ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, float, float][source]

Apply zero-mean, unit-variance normalization.

Parameters:

  • data ((NDArray | torch.Tensor)) – Data to normalize. E.g. (y, x) in 2D and (z, y, x) in 3D.

  • mean (Optional[float]) – Mean to use in the normalization. If None, the mean of the data will be used.

  • std (Optional[float]) – Standard deviation to use in the normalization. If None, the std of the data will be used.

  • apply_norm (bool) – Whether to apply the normalization or just compute the mean and std values. If False, the function will return the original data and the computed mean and std values without applying the normalization.

  • eps (float) – Small value to add to the denominator to prevent division by zero when normalizing.

Returns:

  • data (3D/4D Numpy array or torch.Tensor) – Normalized data if apply_norm is True. E.g. (y, x) in 2D and (z, y, x) in 3D.

  • mean (float) – Mean used in the normalization.

  • std (float) – Standard deviation used in the normalization.

biapy.data.norm.undo_image_norm(data: ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, norm_info: Dict) → ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor[source]

Unnormalize given input data following the normalization steps done before for normalizing it.

Parameters:

  • data (3D/4D Numpy array) – Data to unnormalize. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

  • norm_info (dict) – Normalization information to undo the normalization.

Returns:

data – Unnormalized data. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

Return type:

3D/4D Numpy array

biapy.data.norm.undo_norm_range01(data: ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, norm_info: Dict) → ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor[source]

Undo normalization by multiplaying a factor and optionally summing a minimum value. Opposite function of __norm_range01.

Parameters:

  • data (3D/4D Numpy array) – Data to unnormalize. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

  • norm_info (dict) –

    Information about the normalization. Expected keys are:

    • "max_val_to_div", int/float: maximum value used to divide the data in the normalization.

    • "min_val_to_div", int/float: minimum value used to divide the data in the normalization.

Returns:

data – Unnormalized data. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

Return type:

3D/4D Numpy array

biapy.data.norm.undo_zero_mean_unit_variance_normalization(data: ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor, norm_info: Dict) → ndarray[tuple[int, ...], dtype[_ScalarType_co]] | Tensor[source]

Unnormalization of input data by multiplying by the std and adding the mean.

Opposite function of zero_mean_unit_variance_normalization.

Parameters:

  • data (3D/4D Numpy array) – Image to unnormalize. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

  • norm_info (dict) –

    Information about the normalization. Expected keys are:

    • "mean", int/float: mean used in normalization.

    • "std", int/float: std used in normalization.

Returns:

data – Unnormalized data. E.g. (y, x, channels) in 2D and (z, y, x, channels) in 3D.

Return type:

3D/4D Numpy array