Skip to content

Config

Source

Deprecated configuration from CAREamics v0.1.0.

Configuration

Bases: BaseModel

CAREamics configuration.

The configuration defines all parameters used to build and train a CAREamics model. These parameters are validated to ensure that they are compatible with each other.

It contains three sub-configurations:

  • AlgorithmModel: configuration for the algorithm training, which includes the architecture, loss function, optimizer, and other hyperparameters.
  • DataModel: configuration for the dataloader, which includes the type of data, transformations, mean/std and other parameters.
  • TrainingModel: configuration for the training, which includes the number of epochs or the callbacks.

Attributes:

  • experiment_name (str) –

    Name of the experiment, used when saving logs and checkpoints.

  • algorithm (AlgorithmModel) –

    Algorithm configuration.

  • data (DataModel) –

    Data configuration.

  • training (TrainingModel) –

    Training configuration.

Methods:

  • set_3D –

    Switch configuration between 2D and 3D.

  • model_dump –

    exclude_defaults: bool = False, exclude_none: bool = True, **kwargs: Dict ) -> Dict Export configuration to a dictionary.

Raises:

  • ValueError –

    Configuration parameter type validation errors.

  • ValueError –

    If the experiment name contains invalid characters or is empty.

  • ValueError –

    If the algorithm is 3D but there is not "Z" in the data axes, or 2D algorithm with "Z" in data axes.

  • ValueError –

    Algorithm, data or training validation errors.

Notes

We provide convenience methods to create standards configurations, for instance:

from careamics.compat.config import create_n2v_configuration config = create_n2v_configuration( ... experiment_name="n2v_experiment", ... data_type="array", ... axes="YX", ... patch_size=[64, 64], ... batch_size=32, ... )

The configuration can be exported to a dictionary using the model_dump method:

config_dict = config.model_dump()

Configurations can also be exported or imported from yaml files:

from careamics.compat.config.utils.configuration_io import save_configuration from careamics.compat.config.utils.configuration_io import load_configuration path_to_config = save_configuration(config, my_path / "config.yml") other_config = load_configuration(path_to_config)

Examples:

Minimum example:

>>> from careamics.compat.config import Configuration
>>> config_dict = {
...         "experiment_name": "N2V_experiment",
...         "algorithm_config": {
...             "algorithm": "n2v",
...             "loss": "n2v",
...             "model": {
...                 "architecture": "UNet",
...             },
...         },
...         "training_config": {},
...         "data_config": {
...             "data_type": "tiff",
...             "patch_size": [64, 64],
...             "axes": "SYX",
...         },
...     }
>>> config = Configuration(**config_dict)

algorithm_config = Field(discriminator='algorithm') class-attribute instance-attribute

Algorithm configuration, holding all parameters required to configure the model.

data_config instance-attribute

Data configuration, holding all parameters required to configure the training data loader.

experiment_name instance-attribute

Name of the experiment, used to name logs and checkpoints.

training_config instance-attribute

Training configuration, holding all parameters required to configure the training process.

version = '0.1.0' class-attribute instance-attribute

CAREamics configuration version.

__str__()

Pretty string reprensenting the configuration.

Returns:

  • str –

    Pretty string.

get_algorithm_citations()

Return a list of citation entries of the current algorithm.

This is used to generate the model description for the BioImage Model Zoo.

Returns:

  • List[CiteEntry] –

    List of citation entries.

get_algorithm_description()

Return a description of the algorithm.

This method is used to generate the README of the BioImage Model Zoo export.

Returns:

  • str –

    Description of the algorithm.

get_algorithm_friendly_name()

Get the algorithm name.

Returns:

  • str –

    Algorithm name.

get_algorithm_keywords()

Get algorithm keywords.

Returns:

get_algorithm_references()

Get the algorithm references.

This is used to generate the README of the BioImage Model Zoo export.

Returns:

  • str –

    Algorithm references.

get_safe_experiment_name()

Return the experiment name safe for use in paths and filenames.

Spaces are replaced with underscores to avoid issues with folder creation and checkpoint naming.

Returns:

  • str –

    Experiment name with spaces replaced with underscores.

model_dump(**kwargs)

Override model_dump method in order to set default values.

As opposed to the parent model_dump method, this method sets exclude none by default.

Parameters:

  • **kwargs (Any, default: {} ) –

    Additional arguments to pass to the parent model_dump method.

Returns:

  • dict –

    Dictionary containing the model parameters.

no_symbol(name) classmethod

Validate experiment name.

A valid experiment name is a non-empty string with only contains letters, numbers, underscores, dashes and spaces.

Parameters:

  • name (str) –

    Name to validate.

Returns:

  • str –

    Validated name.

Raises:

  • ValueError –

    If the name is empty or contains invalid characters.

set_3D(is_3D, axes, patch_size)

Set 3D flag and axes.

Parameters:

  • is_3D (bool) –

    Whether the algorithm is 3D or not.

  • axes (str) –

    Axes of the data.

  • patch_size (list[int]) –

    Patch size.

validate_3D()

Change algorithm dimensions to match data.axes.

Returns:

  • Self –

    Validated configuration.

validate_n2v_mask_pixel_perc()

Validate that there will always be at least one blind-spot pixel in every patch.

The probability of creating a blind-spot pixel is a function of the chosen masked pixel percentage and patch size.

Returns:

  • Self –

    Validated configuration.

Raises:

  • ValueError –

    If the probability of masking a pixel within a patch is less than 1 for the chosen masked pixel percentage and patch size.

algorithm_factory(algorithm)

Create an algorithm model for training CAREamics.

Parameters:

  • algorithm (dict) –

    Algorithm dictionary.

Returns:

create_care_configuration(experiment_name, data_type, axes, patch_size, batch_size, num_epochs=100, num_steps=None, augmentations=None, independent_channels=False, loss='mae', n_channels_in=None, n_channels_out=None, logger='none', trainer_params=None, model_params=None, optimizer='Adam', optimizer_params=None, lr_scheduler='ReduceLROnPlateau', lr_scheduler_params=None, train_dataloader_params=None, val_dataloader_params=None, checkpoint_params=None)

Create a configuration for training CARE.

If "Z" is present in axes, then patch_size must be a list of length 3, otherwise 2.

If "C" is present in axes, then you need to set n_channels_in to the number of channels. Likewise, if you set the number of channels, then "C" must be present in axes.

To set the number of output channels, use the n_channels_out parameter. If it is not specified, it will be assumed to be equal to n_channels_in.

By default, all channels are trained together. To train all channels independently, set independent_channels to True.

By setting augmentations to None, the default augmentations (flip in X and Y, rotations by 90 degrees in the XY plane) are applied. Rather than the default augmentations, a list of augmentations can be passed to the augmentations parameter. To disable the augmentations, simply pass an empty list.

Parameters:

  • experiment_name (str) –

    Name of the experiment.

  • data_type (Literal['array', 'tiff', 'czi', 'custom']) –

    Type of the data.

  • axes (str) –

    Axes of the data (e.g. SYX).

  • patch_size (List[int]) –

    Size of the patches along the spatial dimensions (e.g. [64, 64]).

  • batch_size (int) –

    Batch size.

  • num_epochs (int, default: 100 ) –

    Number of epochs to train for. If provided, this will be added to trainer_params.

  • num_steps (int, default: None ) –

    Number of batches in 1 epoch. If provided, this will be added to trainer_params. Translates to limit_train_batches in PyTorch Lightning Trainer. See relevant documentation for more details.

  • augmentations (list of augmentations, default: None ) –

    List of augmentations to apply, either both or one of XYFlipConfig and XYRandomRotate90Config. By default, it applies both XYFlip (on X and Y) and XYRandomRotate90 (in XY) to the images.

  • independent_channels (bool, default: False ) –

    Whether to train all channels independently, by default False.

  • loss (Literal['mae', 'mse'], default: "mae" ) –

    Loss function to use.

  • n_channels_in (int or None, default: None ) –

    Number of channels in.

  • n_channels_out (int or None, default: None ) –

    Number of channels out.

  • logger (Literal['wandb', 'tensorboard', 'none'], default: "none" ) –

    Logger to use.

  • trainer_params (dict, default: None ) –

    Parameters for the trainer class, see PyTorch Lightning documentation.

  • model_params (dict, default: None ) –

    UNetModel parameters.

  • optimizer (Literal['Adam', 'Adamax', 'SGD'], default: "Adam" ) –

    Optimizer to use.

  • optimizer_params (dict, default: None ) –

    Parameters for the optimizer, see PyTorch documentation for more details.

  • lr_scheduler (Literal['ReduceLROnPlateau', 'StepLR'], default: "ReduceLROnPlateau" ) –

    Learning rate scheduler to use.

  • lr_scheduler_params (dict, default: None ) –

    Parameters for the learning rate scheduler, see PyTorch documentation for more details.

  • train_dataloader_params (dict, default: None ) –

    Parameters for the training dataloader, see the PyTorch docs for DataLoader. If left as None, the dict {"shuffle": True} will be used, this is set in the GeneralDataConfig.

  • val_dataloader_params (dict, default: None ) –

    Parameters for the validation dataloader, see PyTorch the docs for DataLoader. If left as None, the empty dict {} will be used, this is set in the GeneralDataConfig.

  • checkpoint_params (dict, default: None ) –

    Parameters for the checkpoint callback, see PyTorch Lightning documentation (ModelCheckpoint) for the list of available parameters.

Returns:

Examples:

Minimum example:

>>> config = create_care_configuration(
...     experiment_name="care_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100
... )

You can also limit the number of batches per epoch:

>>> config = create_care_configuration(
...     experiment_name="care_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_steps=100  # limit to 100 batches per epoch
... )

To disable augmentations, simply set augmentations to an empty list:

>>> config = create_care_configuration(
...     experiment_name="care_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     augmentations=[]
... )

A list of augmentations can be passed to the augmentations parameter: to replace the default augmentations:

>>> from careamics.config.augmentations import XYFlipConfig
>>> config = create_care_configuration(
...     experiment_name="care_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     augmentations=[
...         # No rotation and only Y flipping
...         XYFlipConfig(flip_x = False, flip_y = True)
...     ]
... )

If you are training multiple channels they will be trained together by default, you simply need to specify the number of channels input (and the output if different):

>>> config = create_care_configuration(
...     experiment_name="care_experiment",
...     data_type="array",
...     axes="YXC", # channels must be in the axes
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     n_channels_in=3,
...     n_channels_out=2 # if applicable
... )

If instead you want to train channels independently, you need to turn on the independent_channels parameter (input and output channels must be the same in this case):

>>> config = create_care_configuration(
...     experiment_name="care_experiment",
...     data_type="array",
...     axes="YXC", # channels must be in the axes
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     independent_channels=True,
...     n_channels_in=3,
... )

If you would like to train on CZI files, use "czi" as data_type and "SCYX" as axes for 2-D or "SCZYX" for 3-D denoising. Note that "SCYX" can also be used for 3-D data but spatial context along the Z dimension will then not be taken into account.

>>> config_2d = create_care_configuration(
...     experiment_name="care_experiment",
...     data_type="czi",
...     axes="SCYX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     n_channels_in=1,
... )
>>> config_3d = create_care_configuration(
...     experiment_name="care_experiment",
...     data_type="czi",
...     axes="SCZYX",
...     patch_size=[16, 64, 64],
...     batch_size=16,
...     num_epochs=100,
...     n_channels_in=1,
... )

create_hdn_configuration(experiment_name, data_type, axes, patch_size, batch_size, num_epochs=100, num_steps=None, encoder_conv_strides=(2, 2), decoder_conv_strides=(2, 2), multiscale_count=1, z_dims=(128, 128), output_channels=1, encoder_n_filters=32, decoder_n_filters=32, encoder_dropout=0.0, decoder_dropout=0.0, nonlinearity='ReLU', analytical_kl=False, predict_logvar=None, logvar_lowerbound=None, logger='none', trainer_params=None, augmentations=None, train_dataloader_params=None, val_dataloader_params=None)

Create a configuration for training HDN.

If "Z" is present in axes, then patch_size must be a list of length 3, otherwise 2.

If "C" is present in axes, then you need to set n_channels_in to the number of channels. Likewise, if you set the number of channels, then "C" must be present in axes.

To set the number of output channels, use the n_channels_out parameter. If it is not specified, it will be assumed to be equal to n_channels_in.

By default, all channels are trained independently. To train all channels together, set independent_channels to False.

By setting augmentations to None, the default augmentations (flip in X and Y, rotations by 90 degrees in the XY plane) are applied. Rather than the default augmentations, a list of augmentations can be passed to the augmentations parameter. To disable the augmentations, simply pass an empty list.

TODO revisit the necessity of model_params

Parameters:

  • experiment_name (str) –

    Name of the experiment.

  • data_type (Literal['array', 'tiff', 'custom']) –

    Type of the data.

  • axes (str) –

    Axes of the data (e.g. SYX).

  • patch_size (List[int]) –

    Size of the patches along the spatial dimensions (e.g. [64, 64]).

  • batch_size (int) –

    Batch size.

  • num_epochs (int, default: 100 ) –

    Number of epochs to train for. If provided, this will be added to trainer_params.

  • num_steps (int, default: None ) –

    Number of batches in 1 epoch. If provided, this will be added to trainer_params. Translates to limit_train_batches in PyTorch Lightning Trainer. See relevant documentation for more details.

  • encoder_conv_strides (tuple[int, ...], default: (2, 2) ) –

    Strides for the encoder convolutional layers, by default (2, 2).

  • decoder_conv_strides (tuple[int, ...], default: (2, 2) ) –

    Strides for the decoder convolutional layers, by default (2, 2).

  • multiscale_count (int, default: 1 ) –

    Number of scales in the multiscale architecture, by default 1.

  • z_dims (tuple[int, ...], default: (128, 128) ) –

    Dimensions of the latent space, by default (128, 128).

  • output_channels (int, default: 1 ) –

    Number of output channels, by default 1.

  • encoder_n_filters (int, default: 32 ) –

    Number of filters in the encoder, by default 32.

  • decoder_n_filters (int, default: 32 ) –

    Number of filters in the decoder, by default 32.

  • encoder_dropout (float, default: 0.0 ) –

    Dropout rate for the encoder, by default 0.0.

  • decoder_dropout (float, default: 0.0 ) –

    Dropout rate for the decoder, by default 0.0.

  • nonlinearity (Literal, default: 'ReLU' ) –

    Nonlinearity function to use, by default "ReLU".

  • analytical_kl (bool, default: False ) –

    Whether to use analytical KL divergence, by default False.

  • predict_logvar (Literal[None, 'pixelwise'], default: None ) –

    Type of log variance prediction, by default None.

  • logvar_lowerbound (Union[float, None], default: None ) –

    Lower bound for the log variance, by default None.

  • logger (Literal['wandb', 'tensorboard', 'none'], default: 'none' ) –

    Logger to use for training, by default "none".

  • trainer_params (dict, default: None ) –

    Parameters for the trainer class, see PyTorch Lightning documentation.

  • augmentations (list[XYFlipConfig | XYRandomRotate90Config] | None, default: None ) –

    List of augmentations to apply, by default None.

  • train_dataloader_params (Optional[dict[str, Any]], default: None ) –

    Parameters for the training dataloader, by default None.

  • val_dataloader_params (Optional[dict[str, Any]], default: None ) –

    Parameters for the validation dataloader, by default None.

Returns:

Examples:

Minimum example:

>>> config = create_hdn_configuration(
...     experiment_name="hdn_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100
... )

You can also limit the number of batches per epoch:

>>> config = create_hdn_configuration(
...     experiment_name="hdn_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_steps=100  # limit to 100 batches per epoch
... )

create_microsplit_configuration(experiment_name, data_type, axes, patch_size, batch_size, lr=0.001, num_epochs=100, num_steps=None, encoder_conv_strides=(2, 2), decoder_conv_strides=(2, 2), encoder_n_filters=32, decoder_n_filters=32, multiscale_count=3, grid_size=32, z_dims=(128, 128), output_channels=1, encoder_dropout=0.1, decoder_dropout=0.1, nonlinearity='ELU', analytical_kl=False, predict_logvar='pixelwise', logvar_lowerbound=-5.0, loss_type='denoisplit_musplit', kl_type='kl_restricted', reconstruction_weight=1.0, kl_weight=1.0, musplit_weight=0.1, denoisplit_weight=0.9, mmse_count=10, optimizer='Adamax', lr_scheduler_patience=30, logger='none', trainer_params=None, augmentations=None, nm_paths=None, data_stats=None, train_dataloader_params=None, val_dataloader_params=None)

Create a configuration for training MicroSplit.

Parameters:

  • experiment_name (str) –

    Name of the experiment.

  • data_type (Literal['array', 'tiff', 'custom']) –

    Type of the data.

  • axes (str) –

    Axes of the data (e.g. SYX).

  • patch_size (Sequence[int]) –

    Size of the patches along the spatial dimensions (e.g. [64, 64]).

  • batch_size (int) –

    Batch size.

  • lr (float, default: 0.001 ) –

    Learning rate, by default 1e-3.

  • num_epochs (int, default: 100 ) –

    Number of epochs to train for. If provided, this will be added to trainer_params.

  • num_steps (int, default: None ) –

    Number of batches in 1 epoch. If provided, this will be added to trainer_params. Translates to limit_train_batches in PyTorch Lightning Trainer. See relevant documentation for more details.

  • encoder_conv_strides (tuple[int, ...], default: (2, 2) ) –

    Strides for the encoder convolutional layers, by default (2, 2).

  • decoder_conv_strides (tuple[int, ...], default: (2, 2) ) –

    Strides for the decoder convolutional layers, by default (2, 2).

  • encoder_n_filters (int, default: 32 ) –

    Number of filters in the encoder, by default 32.

  • decoder_n_filters (int, default: 32 ) –

    Number of filters in the decoder, by default 32.

  • multiscale_count (int, default: 3 ) –

    Number of multiscale levels, by default 3.

  • grid_size (int, default: 32 ) –

    Size of the grid for multiscale training, by default 32.

  • z_dims (tuple[int, ...], default: (128, 128) ) –

    List of latent dims for each hierarchy level in the LVAE, default (128, 128).

  • output_channels (int, default: 1 ) –

    Number of output channels for the model, by default 1.

  • encoder_dropout (float, default: 0.1 ) –

    Dropout rate for the encoder, by default 0.0.

  • decoder_dropout (float, default: 0.1 ) –

    Dropout rate for the decoder, by default 0.0.

  • nonlinearity (Literal, default: 'ELU' ) –

    Nonlinearity to use in the model, by default "ELU".

  • analytical_kl (bool, default: False ) –

    Whether to use analytical KL divergence, by default False.

  • predict_logvar (Literal['pixelwise'], default: 'pixelwise' ) –

    Type of log-variance prediction, by default "pixelwise".

  • logvar_lowerbound (float | None, default: -5.0 ) –

    Lower bound for the log variance, by default -5.0.

  • loss_type (Literal['musplit', 'denoisplit', 'denoisplit_musplit'], default: 'denoisplit_musplit' ) –

    Type of loss function, by default "denoisplit_musplit".

  • kl_type (Literal['kl', 'kl_restricted'], default: 'kl_restricted' ) –

    Type of KL divergence, by default "kl_restricted".

  • reconstruction_weight (float, default: 1.0 ) –

    Weight for reconstruction loss, by default 1.0.

  • kl_weight (float, default: 1.0 ) –

    Weight for KL loss, by default 1.0.

  • musplit_weight (float, default: 0.1 ) –

    Weight for muSplit loss, by default 0.1.

  • denoisplit_weight (float, default: 0.9 ) –

    Weight for denoiSplit loss, by default 0.9.

  • mmse_count (int, default: 10 ) –

    Number of MMSE samples to use, by default 10.

  • optimizer (Literal['Adam', 'SGD', 'Adamax'], default: 'Adamax' ) –

    Optimizer to use, by default "Adamax".

  • lr_scheduler_patience (int, default: 30 ) –

    Patience for learning rate scheduler, by default 30.

  • logger (Literal['wandb', 'tensorboard', 'none'], default: 'none' ) –

    Logger to use for training, by default "none".

  • trainer_params (dict, default: None ) –

    Parameters for the trainer class, see PyTorch Lightning documentation.

  • augmentations (list[Union[XYFlipConfig, XYRandomRotate90Config]] | None, default: None ) –

    List of augmentations to apply, by default None.

  • nm_paths (list[str] | None, default: None ) –

    Paths to the noise model files, by default None.

  • data_stats (tuple[float, float] | None, default: None ) –

    Data statistics (mean, std), by default None.

  • train_dataloader_params (dict[str, Any] | None, default: None ) –

    Parameters for the training dataloader, by default None.

  • val_dataloader_params (dict[str, Any] | None, default: None ) –

    Parameters for the validation dataloader, by default None.

Returns:

  • Configuration –

    A configuration object for the microsplit algorithm.

Examples:

Minimum example:

>>> config = create_microsplit_configuration(

... experiment_name="microsplit_experiment",

... data_type="array",

... axes="YX",

... patch_size=[64, 64],

... batch_size=32,

... num_epochs=100

... )

You can also limit the number of batches per epoch:

>>> config = create_microsplit_configuration(

... experiment_name="microsplit_experiment",

... data_type="array",

... axes="YX",

... patch_size=[64, 64],

... batch_size=32,

... num_steps=100 # limit to 100 batches per epoch

... )

create_n2n_configuration(experiment_name, data_type, axes, patch_size, batch_size, num_epochs=100, num_steps=None, augmentations=None, independent_channels=True, loss='mae', n_channels_in=None, n_channels_out=None, logger='none', trainer_params=None, model_params=None, optimizer='Adam', optimizer_params=None, lr_scheduler='ReduceLROnPlateau', lr_scheduler_params=None, train_dataloader_params=None, val_dataloader_params=None, checkpoint_params=None)

Create a configuration for training Noise2Noise.

If "Z" is present in axes, then patch_size must be a list of length 3, otherwise 2.

If "C" is present in axes, then you need to set n_channels_in to the number of channels. Likewise, if you set the number of channels, then "C" must be present in axes.

To set the number of output channels, use the n_channels_out parameter. If it is not specified, it will be assumed to be equal to n_channels_in.

By default, all channels are trained independently. To train all channels together, set independent_channels to True. If the number of input and output channels are the same, channels cannot be independent.

By setting augmentations to None, the default augmentations (flip in X and Y, rotations by 90 degrees in the XY plane) are applied. Rather than the default augmentations, a list of augmentations can be passed to the augmentations parameter. To disable the augmentations, simply pass an empty list.

Parameters:

  • experiment_name (str) –

    Name of the experiment.

  • data_type (Literal['array', 'tiff', 'czi', 'custom']) –

    Type of the data.

  • axes (str) –

    Axes of the data (e.g. SYX).

  • patch_size (List[int]) –

    Size of the patches along the spatial dimensions (e.g. [64, 64]).

  • batch_size (int) –

    Batch size.

  • num_epochs (int, default: 100 ) –

    Number of epochs to train for. If provided, this will be added to trainer_params.

  • num_steps (int, default: None ) –

    Number of batches in 1 epoch. If provided, this will be added to trainer_params. Translates to limit_train_batches in PyTorch Lightning Trainer. See relevant documentation for more details.

  • augmentations (list of augmentations, default: None ) –

    List of augmentations to apply, either both or one of XYFlipConfig and XYRandomRotate90Config. By default, it applies both XYFlip (on X and Y) and XYRandomRotate90 (in XY) to the images.

  • independent_channels (bool, default: True ) –

    Whether to train all channels independently, by default False.

  • loss (Literal['mae', 'mse'], default: 'mae' ) –

    Loss function to use, by default "mae".

  • n_channels_in (int or None, default: None ) –

    Number of channels in.

  • n_channels_out (int or None, default: None ) –

    Number of channels out.

  • logger (Literal['wandb', 'tensorboard', 'none'], default: 'none' ) –

    Logger to use, by default "none".

  • trainer_params (dict, default: None ) –

    Parameters for the trainer class, see PyTorch Lightning documentation.

  • model_params (dict, default: None ) –

    UNetModel parameters.

  • optimizer (Literal['Adam', 'Adamax', 'SGD'], default: "Adam" ) –

    Optimizer to use.

  • optimizer_params (dict, default: None ) –

    Parameters for the optimizer, see PyTorch documentation for more details.

  • lr_scheduler (Literal['ReduceLROnPlateau', 'StepLR'], default: "ReduceLROnPlateau" ) –

    Learning rate scheduler to use.

  • lr_scheduler_params (dict, default: None ) –

    Parameters for the learning rate scheduler, see PyTorch documentation for more details.

  • train_dataloader_params (dict, default: None ) –

    Parameters for the training dataloader, see the PyTorch docs for DataLoader. If left as None, the dict {"shuffle": True} will be used, this is set in the GeneralDataConfig.

  • val_dataloader_params (dict, default: None ) –

    Parameters for the validation dataloader, see PyTorch the docs for DataLoader. If left as None, the empty dict {} will be used, this is set in the GeneralDataConfig.

  • checkpoint_params (dict, default: None ) –

    Parameters for the checkpoint callback, see PyTorch Lightning documentation (ModelCheckpoint) for the list of available parameters.

Returns:

Examples:

Minimum example:

>>> config = create_n2n_configuration(
...     experiment_name="n2n_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100
... )

You can also limit the number of batches per epoch:

>>> config = create_n2n_configuration(
...     experiment_name="n2n_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_steps=100  # limit to 100 batches per epoch
... )

To disable augmentations, simply set augmentations to an empty list:

>>> config = create_n2n_configuration(
...     experiment_name="n2n_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     augmentations=[]
... )

A list of augmentations can be passed to the augmentations parameter:

>>> from careamics.config.augmentations import XYFlipConfig
>>> config = create_n2n_configuration(
...     experiment_name="n2n_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     augmentations=[
...         # No rotation and only Y flipping
...         XYFlipConfig(flip_x = False, flip_y = True)
...     ]
... )

If you are training multiple channels they will be trained independently by default, you simply need to specify the number of channels input (the number of output channels must be the same as input in this case):

>>> config = create_n2n_configuration(
...     experiment_name="n2n_experiment",
...     data_type="array",
...     axes="YXC", # channels must be in the axes
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     n_channels_in=3, # number of input channels
... )

If instead you want to train multiple channels with different number of input and output channels, you need to turn off the independent_channels parameter:

>>> config = create_n2n_configuration(
...     experiment_name="n2n_experiment",
...     data_type="array",
...     axes="YXC", # channels must be in the axes
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     independent_channels=False,
...     n_channels_in=3,
...     n_channels_out=1 # if applicable
... )

If you would like to train on CZI files, use "czi" as data_type and "SCYX" as axes for 2-D or "SCZYX" for 3-D denoising. Note that "SCYX" can also be used for 3-D data but spatial context along the Z dimension will then not be taken into account.

>>> config_2d = create_n2n_configuration(
...     experiment_name="n2n_experiment",
...     data_type="czi",
...     axes="SCYX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     n_channels_in=1,
... )
>>> config_3d = create_n2n_configuration(
...     experiment_name="n2n_experiment",
...     data_type="czi",
...     axes="SCZYX",
...     patch_size=[16, 64, 64],
...     batch_size=16,
...     num_epochs=100,
...     n_channels_in=1,
... )

create_n2v_configuration(experiment_name, data_type, axes, patch_size, batch_size, num_epochs=100, num_steps=None, augmentations=None, independent_channels=True, use_n2v2=False, n_channels=None, roi_size=11, masked_pixel_percentage=0.2, struct_n2v_axis='none', struct_n2v_span=5, trainer_params=None, logger='none', model_params=None, optimizer='Adam', optimizer_params=None, lr_scheduler='ReduceLROnPlateau', lr_scheduler_params=None, train_dataloader_params=None, val_dataloader_params=None, checkpoint_params=None, seed=None)

Create a configuration for training Noise2Void.

N2V uses a UNet model to denoise images in a self-supervised manner. To use its variants structN2V and N2V2, set the struct_n2v_axis and struct_n2v_span (structN2V) parameters, or set use_n2v2 to True (N2V2).

N2V2 modifies the UNet architecture by adding blur pool layers and removes the skip connections, thus removing checkboard artefacts. StructN2V is used when vertical or horizontal correlations are present in the noise; it applies an additional mask to the manipulated pixel neighbors.

If "Z" is present in axes, then patch_size must be a list of length 3, otherwise 2.

If "C" is present in axes, then you need to set n_channels to the number of channels.

By default, all channels are trained independently. To train all channels together, set independent_channels to False.

By default, the augmentations applied are a random flip along X or Y, and a random 90 degrees rotation in the XY plane. Normalization is always applied, as well as the N2V manipulation.

By setting augmentations to None, the default augmentations (flip in X and Y, rotations by 90 degrees in the XY plane) are applied. Rather than the default augmentations, a list of augmentations can be passed to the augmentations parameter. To disable the augmentations, simply pass an empty list.

The roi_size parameter specifies the size of the area around each pixel that will be manipulated by N2V. The masked_pixel_percentage parameter specifies how many pixels per patch will be manipulated.

The parameters of the UNet can be specified in the model_params (passed as a parameter-value dictionary). Note that use_n2v2 and 'n_channels' override the corresponding parameters passed in model_params.

If you pass "horizontal" or "vertical" to struct_n2v_axis, then structN2V mask will be applied to each manipulated pixel.

Parameters:

  • experiment_name (str) –

    Name of the experiment.

  • data_type (Literal['array', 'tiff', 'czi', 'custom']) –

    Type of the data.

  • axes (str) –

    Axes of the data (e.g. SYX).

  • patch_size (List[int]) –

    Size of the patches along the spatial dimensions (e.g. [64, 64]).

  • batch_size (int) –

    Batch size.

  • num_epochs (int, default: 100 ) –

    Number of epochs to train for. If provided, this will be added to trainer_params.

  • num_steps (int, default: None ) –

    Number of batches in 1 epoch. If provided, this will be added to trainer_params. Translates to limit_train_batches in PyTorch Lightning Trainer. See relevant documentation for more details.

  • augmentations (list of augmentations, default: None ) –

    List of augmentations to apply, either both or one of XYFlipConfig and XYRandomRotate90Config. By default, it applies both XYFlip (on X and Y) and XYRandomRotate90 (in XY) to the images.

  • independent_channels (bool, default: True ) –

    Whether to train all channels together, by default True.

  • use_n2v2 (bool, default: False ) –

    Whether to use N2V2, by default False.

  • n_channels (int or None, default: None ) –

    Number of channels (in and out).

  • roi_size (int, default: 11 ) –

    N2V pixel manipulation area, by default 11.

  • masked_pixel_percentage (float, default: 0.2 ) –

    Percentage of pixels masked in each patch, by default 0.2.

  • struct_n2v_axis (Literal['horizontal', 'vertical', 'none'], default: 'none' ) –

    Axis along which to apply structN2V mask, by default "none".

  • struct_n2v_span (int, default: 5 ) –

    Span of the structN2V mask, by default 5.

  • trainer_params (dict, default: None ) –

    Parameters for the trainer, see the relevant documentation.

  • logger (Literal['wandb', 'tensorboard', 'none'], default: 'none' ) –

    Logger to use, by default "none".

  • model_params (dict, default: None ) –

    UNetModel parameters.

  • optimizer (Literal['Adam', 'Adamax', 'SGD'], default: "Adam" ) –

    Optimizer to use.

  • optimizer_params (dict, default: None ) –

    Parameters for the optimizer, see PyTorch documentation for more details.

  • lr_scheduler (Literal['ReduceLROnPlateau', 'StepLR'], default: "ReduceLROnPlateau" ) –

    Learning rate scheduler to use.

  • lr_scheduler_params (dict, default: None ) –

    Parameters for the learning rate scheduler, see PyTorch documentation for more details.

  • train_dataloader_params (dict, default: None ) –

    Parameters for the training dataloader, see the PyTorch docs for DataLoader. If left as None, the dict {"shuffle": True} will be used, this is set in the GeneralDataConfig.

  • val_dataloader_params (dict, default: None ) –

    Parameters for the validation dataloader, see PyTorch the docs for DataLoader. If left as None, the empty dict {} will be used, this is set in the GeneralDataConfig.

  • checkpoint_params (dict, default: None ) –

    Parameters for the checkpoint callback, see PyTorch Lightning documentation (ModelCheckpoint) for the list of available parameters.

  • seed (int or None, default: None ) –

    Random seed for reproducibility of N2V pixel manipulation, by default None.

Returns:

Examples:

Minimum example:

>>> config = create_n2v_configuration(
...     experiment_name="n2v_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100
... )

You can also limit the number of batches per epoch:

>>> config = create_n2v_configuration(
...     experiment_name="n2v_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_steps=100  # limit to 100 batches per epoch
... )

To disable augmentations, simply set augmentations to an empty list:

>>> config = create_n2v_configuration(
...     experiment_name="n2v_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     augmentations=[]
... )

A list of augmentations can be passed to the augmentations parameter:

>>> from careamics.config.augmentations import XYFlipConfig
>>> config = create_n2v_configuration(
...     experiment_name="n2v_experiment",
...     data_type="array",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     augmentations=[
...         # No rotation and only Y flipping
...         XYFlipConfig(flip_x = False, flip_y = True)
...     ]
... )

To use N2V2, simply pass the use_n2v2 parameter:

>>> config = create_n2v_configuration(
...     experiment_name="n2v2_experiment",
...     data_type="tiff",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     use_n2v2=True
... )

For structN2V, there are two parameters to set, struct_n2v_axis and struct_n2v_span:

>>> config = create_n2v_configuration(
...     experiment_name="structn2v_experiment",
...     data_type="tiff",
...     axes="YX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     struct_n2v_axis="horizontal",
...     struct_n2v_span=7
... )

If you are training multiple channels they will be trained independently by default, you simply need to specify the number of channels:

>>> config = create_n2v_configuration(
...     experiment_name="n2v_experiment",
...     data_type="array",
...     axes="YXC",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     n_channels=3
... )

If instead you want to train multiple channels together, you need to turn off the independent_channels parameter:

>>> config = create_n2v_configuration(
...     experiment_name="n2v_experiment",
...     data_type="array",
...     axes="YXC",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     independent_channels=False,
...     n_channels=3
... )

If you would like to train on CZI files, use "czi" as data_type and "SCYX" as axes for 2-D or "SCZYX" for 3-D denoising. Note that "SCYX" can also be used for 3-D data but spatial context along the Z dimension will then not be taken into account.

>>> config_2d = create_n2v_configuration(
...     experiment_name="n2v_experiment",
...     data_type="czi",
...     axes="SCYX",
...     patch_size=[64, 64],
...     batch_size=32,
...     num_epochs=100,
...     n_channels=1,
... )
>>> config_3d = create_n2v_configuration(
...     experiment_name="n2v_experiment",
...     data_type="czi",
...     axes="SCZYX",
...     patch_size=[16, 64, 64],
...     batch_size=16,
...     num_epochs=100,
...     n_channels=1,
... )

create_pn2v_configuration(experiment_name, data_type, axes, patch_size, batch_size, nm_path, num_epochs=100, num_steps=None, augmentations=None, independent_channels=True, use_n2v2=False, num_in_channels=1, num_out_channels=100, roi_size=11, masked_pixel_percentage=0.2, struct_n2v_axis='none', struct_n2v_span=5, trainer_params=None, logger='none', model_params=None, optimizer='Adam', optimizer_params=None, lr_scheduler='ReduceLROnPlateau', lr_scheduler_params=None, train_dataloader_params=None, val_dataloader_params=None, checkpoint_params=None, seed=None)

Create a configuration for training Probabilistic Noise2Void (PN2V).

PN2V extends N2V by incorporating a probabilistic noise model to estimate the posterior distibution of each pixel more precisely.

If "Z" is present in axes, then path_size must be a list of length 3, otherwise 2.

If "C" is present in axes, then you need to set num_in_channels to the number of channels.

By default, all channels are trained independently. To train all channels together, set independent_channels to False. When training independently, each input channel will have num_out_channels outputs (default 400). When training together, all input channels will share num_out_channels outputs.

By default, the augmentations applied are a random flip along X or Y, and a random 90 degrees rotation in the XY plane. Normalization is always applied, as well as the N2V manipulation.

By setting augmentations to None, the default augmentations (flip in X and Y, rotations by 90 degrees in the XY plane) are applied. Rather than the default augmentations, a list of augmentations can be passed to the augmentations parameter. To disable the augmentations, simply pass an empty list.

The roi_size parameter specifies the size of the area around each pixel that will be manipulated by N2V. The masked_pixel_percentage parameter specifies how many pixels per patch will be manipulated.

The parameters of the UNet can be specified in the model_params (passed as a parameter-value dictionary). Note that use_n2v2, num_in_channels, and num_out_channels override the corresponding parameters passed in model_params.

If you pass "horizontal" or "vertical" to struct_n2v_axis, then structN2V mask will be applied to each manipulated pixel.

Parameters:

  • experiment_name (str) –

    Name of the experiment.

  • data_type (Literal['array', 'tiff', 'czi', 'custom']) –

    Type of the data.

  • axes (str) –

    Axes of the data (e.g. SYX).

  • patch_size (List[int]) –

    Size of the patches along the spatial dimensions (e.g. [64, 64]).

  • batch_size (int) –

    Batch size.

  • nm_path (str) –

    Path to the noise model file.

  • num_epochs (int, default: 100 ) –

    Number of epochs to train for. If provided, this will be added to trainer_params.

  • num_steps (int, default: None ) –

    Number of batches in 1 epoch. If provided, this will be added to trainer_params. Translates to limit_train_batches in PyTorch Lightning Trainer. See relevant documentation for more details.

  • augmentations (list of augmentations, default: None ) –

    List of augmentations to apply, either both or one of XYFlipModel and XYRandomRotate90Model. By default, it applies both XYFlip (on X and Y) and XYRandomRotate90 (in XY) to the images.

  • independent_channels (bool, default: True ) –

    Whether to train all channels independently, by default True. If True, each input channel will correspond to num_out_channels output channels (e.g., 3 input channels with num_out_channels=400 results in 1200 total output channels).

  • use_n2v2 (bool, default: False ) –

    Whether to use N2V2, by default False.

  • num_in_channels (int, default: 1 ) –

    Number of input channels.

  • num_out_channels (int, default: 400 ) –

    Number of output channels per input channel when independent_channels is True, or total number of output channels when independent_channels is False.

  • roi_size (int, default: 11 ) –

    N2V pixel manipulation area, by default 11.

  • masked_pixel_percentage (float, default: 0.2 ) –

    Percentage of pixels masked in each patch, by default 0.2.

  • struct_n2v_axis (Literal['horizontal', 'vertical', 'none'], default: 'none' ) –

    Axis along which to apply structN2V mask, by default "none".

  • struct_n2v_span (int, default: 5 ) –

    Span of the structN2V mask, by default 5.

  • trainer_params (dict, default: None ) –

    Parameters for the trainer, see the relevant documentation.

  • logger (Literal['wandb', 'tensorboard', 'none'], default: 'none' ) –

    Logger to use, by default "none".

  • model_params (dict, default: None ) –

    UNetModel parameters.

  • optimizer (Literal['Adam', 'Adamax', 'SGD'], default: "Adam" ) –

    Optimizer to use.

  • optimizer_params (dict, default: None ) –

    Parameters for the optimizer, see PyTorch documentation for more details.

  • lr_scheduler (Literal['ReduceLROnPlateau', 'StepLR'], default: "ReduceLROnPlateau" ) –

    Learning rate scheduler to use.

  • lr_scheduler_params (dict, default: None ) –

    Parameters for the learning rate scheduler, see PyTorch documentation for more details.

  • train_dataloader_params (dict, default: None ) –

    Parameters for the training dataloader, see the PyTorch docs for DataLoader. If left as None, the dict {"shuffle": True} will be used, this is set in the GeneralDataConfig.

  • val_dataloader_params (dict, default: None ) –

    Parameters for the validation dataloader, see PyTorch the docs for DataLoader. If left as None, the empty dict {} will be used, this is set in the GeneralDataConfig.

  • checkpoint_params (dict, default: None ) –

    Parameters for the checkpoint callback, see PyTorch Lightning documentation (ModelCheckpoint) for the list of available parameters.

  • seed (int or None, default: None ) –

    Random seed for reproducibility of N2V pixel manipulation, by default None.

Returns:

Examples:

Minimum example:

>>> config = create_pn2v_configuration(

... experiment_name="pn2v_experiment",

... data_type="array",

... axes="YX",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_epochs=100

... )

You can also limit the number of batches per epoch:

>>> config = create_pn2v_configuration(

... experiment_name="pn2v_experiment",

... data_type="array",

... axes="YX",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_steps=100 # limit to 100 batches per epoch

... )

To disable augmentations, simply set augmentations to an empty list:

>>> config = create_pn2v_configuration(

... experiment_name="pn2v_experiment",

... data_type="array",

... axes="YX",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_epochs=100,

... augmentations=[]

... )

A list of augmentations can be passed to the augmentations parameter:

>>> from careamics.config.augmentations import XYFlipModel

>>> config = create_pn2v_configuration(

... experiment_name="pn2v_experiment",

... data_type="array",

... axes="YX",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_epochs=100,

... augmentations=[

... # No rotation and only Y flipping

... XYFlipModel(flip_x = False, flip_y = True)

... ]

... )

To use N2V2, simply pass the use_n2v2 parameter:

>>> config = create_pn2v_configuration(

... experiment_name="pn2v2_experiment",

... data_type="tiff",

... axes="YX",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_epochs=100,

... use_n2v2=True

... )

For structN2V, there are two parameters to set, struct_n2v_axis and

struct_n2v_span:

>>> config = create_pn2v_configuration(

... experiment_name="structpn2v_experiment",

... data_type="tiff",

... axes="YX",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_epochs=100,

... struct_n2v_axis="horizontal",

... struct_n2v_span=7

... )

If you are training multiple channels they will be trained independently by

default, you simply need to specify the number of input channels. Each input

channel will correspond to num_out_channels outputs (1200 total for 3

channels with default num_out_channels=400):

>>> config = create_pn2v_configuration(

... experiment_name="pn2v_experiment",

... data_type="array",

... axes="YXC",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_epochs=100,

... num_in_channels=3

... )

If instead you want to train multiple channels together, you need to turn

off the independent_channels parameter (resulting in 400 total output

channels regardless of the number of input channels):

>>> config = create_pn2v_configuration(

... experiment_name="pn2v_experiment",

... data_type="array",

... axes="YXC",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_epochs=100,

... independent_channels=False,

... num_in_channels=3

... )

>>> config_2d = create_pn2v_configuration(

... experiment_name="pn2v_experiment",

... data_type="czi",

... axes="SCYX",

... patch_size=[64, 64],

... batch_size=32,

... nm_path="path/to/noise_model.npz",

... num_epochs=100,

... num_in_channels=1,

... )

>>> config_3d = create_pn2v_configuration(

... experiment_name="pn2v_experiment",

... data_type="czi",

... axes="SCZYX",

... patch_size=[16, 64, 64],

... batch_size=16,

... nm_path="path/to/noise_model.npz",

... num_epochs=100,

... num_in_channels=1,

... )