Prediction#

Prediction is done by calling careamist.predict on either path or arrays. By default, the prediction function will expect the same type of data (e.g. array or path) as it was trained on, but it is possible to predict on a different type of data.

Prediction is performed using the current weights.

Predict on arrays or paths#

On numpy arrays On Paths

import numpy as np

array = np.random.rand(256, 256)

careamist.predict(
    source=array,
)

careamist.predict(
    source=path_to_data,
)

Tiling#

Often, an image will be too large to fit on the GPU memory, or will have dimensions that are incompatible with the model (e.g. odd dimensions). To solve this, the image can be tiled into smaller overlapping patches, predicted upon and ultimately recombined.

In careamist.predict, this is done by passing two parameters:

careamist.predict(
    source=array,
    tile_size=[64, 64],  # (1)!
    tile_overlap=[32, 32],  # (2)!
)

The tile sizes correspond to each spatial dimensions. A good start is the patch size used during training.
The overlap is the number of pixels that each patch will overlap with its neighbors.

After prediction, each tile is cropped to half of the overlap in each of the directions it overlaps with a neighboring tile. The reason is to minimize edge artifacts.

Tile and overlap model constraints

Some models, e.g. UNet model, impose constraints on the tile and overlap sizes. This is a direct consequence of the model architecture.

For instance, when using a UNet model, the pooling and upsampling operations are not compatible with any tile size:

tile sizes must be equal to \(k2^n\), where \(n\) is the number of pooling layers (equal to the model depth) and \(k\) is an integer.
overlaps must be even and larger than twice the receptive field.

Test time augmentation#

Test-time augmentation applies augmentations to the prediction input and averages the (de-augmented) predictions. This can improve the quality of the prediction. The TTA generates all possible flipped and rotated versions of the image.

By default, test-time augmentation is applied by CAREamics. In order to deactivate TTA, you can set tta to False.

careamist.predict(
    source=array,
    tta=False,  # (1)!
)

By default, TTA is activated!

Transforms and TTA

If you have turned off transforms, or used the non default ones, then you should turn off TTA, as it would otherwise create images that do not correspond to your training data.

Using batches#

To potentially predict faster, you can predict on batches of images.

careamist.predict(
    source=array,
    batch_size=2,  # (1)!
)

Each prediction step will be performed on 2 images or tiles.

Batch and TTA

Having batch_size>1 is compatible with the TTA.

Changing the data type#

You can use a different type (in the sense path vs array) of data at prediction time by changing the data_type parameter.

careamist.predict(
    source=path_to_data,
    data_type="tiff",  # (1)!
)

As in the rest of CAREamics, the supported value are tiff, array and custom.

Changing the axes#

Similarly, if you want to predict on data that has different axes than the training data, as long as those have the same number of channels and spatial dimensions, then you can change the axes parameter.

careamist.predict(
    source=other_array,
    axes="SYX",  # (1)!
)

Obviously, this need to match source.

(Advanced) Predict on custom data type#

As for the training, one can predict on custom data types by providing a function that reads the data from a path and a function to filter the requested extension.

def read_npy(
    path: Path,
    *args: Any,
    **kwargs: Any,
) -> np.ndarray:
    return np.load(path)


# example data
predict_array = np.random.rand(128, 128)
np.save("train_array.npy", train_array)

# Train
careamist.predict(
    source=predict_array,
    read_source_func=read_npy,
    extension_filter="*.npy",
)