Skip to content

Data

Source

Deprecated v0.1.0 data Pydantic configuration models.

DataConfig

Bases: BaseModel

Data configuration.

If std is specified, mean must be specified as well. Note that setting the std first and then the mean (if they were both None before) will raise a validation error. Prefer instead set_mean_and_std to set both at once. Means and stds are expected to be lists of floats, one for each channel. For supervised tasks, the mean and std of the target could be different from the input data.

All supported transforms are defined in the SupportedTransform enum.

Examples:

Minimum example:

>>> data = DataConfig(
...     data_type="array", # defined in SupportedData
...     patch_size=[128, 128],
...     batch_size=4,
...     axes="YX"
... )

To change the image_means and image_stds of the data:

>>> data.set_means_and_stds(image_means=[214.3], image_stds=[84.5])

One can pass also a list of transformations, by keyword, using the SupportedTransform value:

>>> from careamics.config.support import SupportedTransform
>>> data = DataConfig(
...     data_type="tiff",
...     patch_size=[128, 128],
...     batch_size=4,
...     axes="YX",
...     transforms=[
...         {
...             "name": "XYFlip",
...         }
...     ]
... )

axes instance-attribute

Axes of the data, as defined in SupportedAxes.

batch_size = Field(default=1, ge=1, validate_default=True) class-attribute instance-attribute

Batch size for training.

data_type instance-attribute

Type of input data, numpy.ndarray (array) or paths (tiff, czi, and custom), as defined in SupportedData.

image_means = Field(default=None, min_length=0, max_length=32) class-attribute instance-attribute

Means of the data across channels, used for normalization.

image_stds = Field(default=None, min_length=0, max_length=32) class-attribute instance-attribute

Standard deviations of the data across channels, used for normalization.

patch_size = Field(..., min_length=2, max_length=3) class-attribute instance-attribute

Patch size, as used during training.

target_means = Field(default=None, min_length=0, max_length=32) class-attribute instance-attribute

Means of the target data across channels, used for normalization.

target_stds = Field(default=None, min_length=0, max_length=32) class-attribute instance-attribute

Standard deviations of the target data across channels, used for normalization.

train_dataloader_params = Field(default={'shuffle': True}, validate_default=True) class-attribute instance-attribute

Dictionary of PyTorch training dataloader parameters. The dataloader parameters, should include the shuffle key, which is set to True by default. We strongly recommend to keep it as True to ensure the best training results.

transforms = Field(default=[XYFlipConfig(), XYRandomRotate90Config()], validate_default=True) class-attribute instance-attribute

List of transformations to apply to the data, available transforms are defined in SupportedTransform.

val_dataloader_params = Field(default={}, validate_default=True) class-attribute instance-attribute

Dictionary of PyTorch validation dataloader parameters.

__str__()

Pretty string reprensenting the configuration.

Returns:

  • str –

    Pretty string.

all_elements_power_of_2_minimum_8(patch_list) classmethod

Validate patch size.

Patch size must be powers of 2 and minimum 8.

Parameters:

  • patch_list (list of int) –

    Patch size.

Returns:

  • list of int –

    Validated patch size.

Raises:

  • ValueError –

    If the patch size is smaller than 8.

  • ValueError –

    If the patch size is not a power of 2.

axes_valid(axes) classmethod

Validate axes.

Axes must: - be a combination of 'STCZYX' - not contain duplicates - contain at least 2 contiguous axes: X and Y - contain at most 4 axes - not contain both S and T axes

Parameters:

  • axes (str) –

    Axes to validate.

Returns:

  • str –

    Validated axes.

Raises:

set_3D(axes, patch_size)

Set 3D parameters.

Parameters:

  • axes (str) –

    Axes.

  • patch_size (list of int) –

    Patch size.

set_default_pin_memory(dataloader_params) classmethod

Set default pin_memory for dataloader parameters if not provided.

  • If 'pin_memory' is not set, it defaults to True if CUDA is available.

Parameters:

  • dataloader_params (dict of {str: Any}) –

    The dataloader parameters.

Returns:

  • dict of {str: Any} –

    The dataloader parameters with pin_memory default applied.

set_default_train_workers(dataloader_params) classmethod

Set default num_workers for training dataloader if not provided.

  • If 'num_workers' is not set, it defaults to the number of available CPU cores.

Parameters:

  • dataloader_params (dict of {str: Any}) –

    The training dataloader parameters.

Returns:

  • dict of {str: Any} –

    The dataloader parameters with num_workers default applied.

set_means_and_stds(image_means, image_stds, target_means=None, target_stds=None)

Set mean and standard deviation of the data across channels.

This method should be used instead setting the fields directly, as it would otherwise trigger a validation error.

Parameters:

  • image_means ((ndarray, tuple or list)) –

    Mean values for normalization.

  • image_stds ((ndarray, tuple or list)) –

    Standard deviation values for normalization.

  • target_means ((ndarray, tuple or list), default: None ) –

    Target mean values for normalization, by default ().

  • target_stds ((ndarray, tuple or list), default: None ) –

    Target standard deviation values for normalization, by default ().

set_val_workers_to_match_train()

Set validation dataloader num_workers to match training dataloader.

If num_workers is not specified in val_dataloader_params, it will be set to the same value as train_dataloader_params["num_workers"].

Returns:

  • Self –

    Validated data model with synchronized num_workers.

shuffle_train_dataloader(train_dataloader_params) classmethod

Validate that "shuffle" is included in the training dataloader params.

A warning will be raised if shuffle=False.

Parameters:

  • train_dataloader_params (dict of {str: Any}) –

    The training dataloader parameters.

Returns:

  • dict of {str: Any} –

    The validated training dataloader parameters.

Raises:

  • ValueError –

    If "shuffle" is not included in the training dataloader params.

std_only_with_mean()

Check that mean and std are either both None, or both specified.

Returns:

  • Self –

    Validated data model.

Raises:

  • ValueError –

    If std is not None and mean is None.

validate_dimensions()

Validate 2D/3D dimensions between axes, patch size and transforms.

Returns:

  • Self –

    Validated data model.

Raises:

InferenceConfig

Bases: BaseModel

Configuration class for the prediction model.

axes instance-attribute

Data axes (TSCZYX) in the order of the input data.

batch_size = Field(default=1, ge=1) class-attribute instance-attribute

Batch size for prediction.

data_type instance-attribute

Type of input data: numpy.ndarray (array) or path (tiff, czi, or custom).

image_means = Field(..., min_length=0, max_length=32) class-attribute instance-attribute

Mean values for each input channel.

image_stds = Field(..., min_length=0, max_length=32) class-attribute instance-attribute

Standard deviation values for each input channel.

tile_overlap = Field(default=None, min_length=2, max_length=3) class-attribute instance-attribute

Overlap between tiles, only effective if tile_size is specified.

tile_size = Field(default=None, min_length=2, max_length=3) class-attribute instance-attribute

Tile size of prediction, only effective if tile_overlap is specified.

tta_transforms = Field(default=True) class-attribute instance-attribute

Whether to apply test-time augmentation (all 90 degrees rotations and flips).

all_elements_non_zero_even(tile_overlap) classmethod

Validate tile overlap.

Overlaps must be non-zero, positive and even.

Parameters:

  • tile_overlap (list[int] or None) –

    Patch size.

Returns:

  • list[int] or None –

    Validated tile overlap.

Raises:

axes_valid(axes) classmethod

Validate axes.

Axes must: - be a combination of 'STCZYX' - not contain duplicates - contain at least 2 contiguous axes: X and Y - contain at most 4 axes - not contain both S and T axes

Parameters:

  • axes (str) –

    Axes to validate.

Returns:

  • str –

    Validated axes.

Raises:

set_3D(axes, tile_size, tile_overlap)

Set 3D parameters.

Parameters:

  • axes (str) –

    Axes.

  • tile_size (list of int) –

    Tile size.

  • tile_overlap (list of int) –

    Tile overlap.

std_only_with_mean()

Check that mean and std are either both None, or both specified.

Returns:

  • Self –

    Validated prediction model.

Raises:

  • ValueError –

    If std is not None and mean is None.

tile_min_8_power_of_2(tile_list) classmethod

Validate that each entry is greater or equal than 8 and a power of 2.

Parameters:

  • tile_list (list of int) –

    Patch size.

Returns:

  • list of int –

    Validated patch size.

Raises:

  • ValueError –

    If the patch size if smaller than 8.

  • ValueError –

    If the patch size is not a power of 2.

validate_dimensions()

Validate 2D/3D dimensions between axes and tile size.

Returns:

  • Self –

    Validated prediction model.

TileInformation

Bases: BaseModel

Pydantic model containing tile information.

This model is used to represent the information required to stitch back a tile into a larger image. It is used throughout the prediction pipeline of CAREamics.

Array shape should be C(Z)YX, where Z is an optional dimensions.

array_shape instance-attribute

Shape of the original (untiled) array.

last_tile = False class-attribute instance-attribute

Whether this tile is the last one of the array.

overlap_crop_coords instance-attribute

Inner coordinates of the tile where to crop the prediction in order to stitch it back into the original image.

sample_id instance-attribute

Sample ID of the tile.

stitch_coords instance-attribute

Coordinates in the original image where to stitch the cropped tile back.

__eq__(other_tile)

Check if two tile information objects are equal.

Parameters:

  • other_tile (object) –

    Tile information object to compare with.

Returns:

  • bool –

    Whether the two tile information objects are equal.