SEM
Find me on Github The SEM dataset is composed of a training and a validation images acquired on a scanning electron microscopy (SEM). They were originally used in Buchholtz et al (2019) to showcase CARE denoising. Here, we demonstrate the performances of Noise2Void on this particular dataset!
In [1]:
Copied!
# Imports necessary to execute the code
from pathlib import Path
import matplotlib.pyplot as plt
import tifffile
import numpy as np
from PIL import Image
from careamics import CAREamist
from careamics.config import create_n2v_configuration
from careamics_portfolio import PortfolioManager
# Imports necessary to execute the code from pathlib import Path import matplotlib.pyplot as plt import tifffile import numpy as np from PIL import Image from careamics import CAREamist from careamics.config import create_n2v_configuration from careamics_portfolio import PortfolioManager
Import the dataset¶
The dataset can be directly downloaded using the careamics-portfolio package, which uses pooch to download the data.
In [ ]:
Copied!
# instantiate data portfolio manage
portfolio = PortfolioManager()
# and download the data
root_path = Path("./data")
files = portfolio.denoising.N2V_SEM.download(root_path)
# instantiate data portfolio manage portfolio = PortfolioManager() # and download the data root_path = Path("./data") files = portfolio.denoising.N2V_SEM.download(root_path)
Data description¶
In [3]:
Copied!
portfolio.denoising.N2V_SEM.description
portfolio.denoising.N2V_SEM.description
Out[3]:
'Cropped images from a SEM dataset from T.-O. Buchholz et al (Methods Cell Biol, 2020).'
Visualize data¶
In [4]:
Copied!
# load training and validation image and show them side by side
train_image = tifffile.imread(files[0])
val_image = tifffile.imread(files[1])
fig, ax = plt.subplots(1, 2, figsize=(10, 5))
ax[0].imshow(train_image, cmap="gray")
ax[0].set_title("Training Image")
ax[1].imshow(val_image, cmap="gray")
ax[1].set_title("Validation Image")
# load training and validation image and show them side by side train_image = tifffile.imread(files[0]) val_image = tifffile.imread(files[1]) fig, ax = plt.subplots(1, 2, figsize=(10, 5)) ax[0].imshow(train_image, cmap="gray") ax[0].set_title("Training Image") ax[1].imshow(val_image, cmap="gray") ax[1].set_title("Validation Image")
Out[4]:
Text(0.5, 1.0, 'Validation Image')
Train with CAREamics¶
The easiest way to use CAREamics is to create a configuration and a CAREamist.
Create configuration¶
The configuration can be built from scratch, giving the user full control over the various parameters available in CAREamics. However, a straightforward way to create a configuration for a particular algorithm is to use one of the convenience functions.
In [5]:
Copied!
config = create_n2v_configuration(
experiment_name="sem_n2v",
data_type="array",
axes="YX",
patch_size=(64, 64),
batch_size=32,
num_epochs=30,
)
print(config)
config = create_n2v_configuration( experiment_name="sem_n2v", data_type="array", axes="YX", patch_size=(64, 64), batch_size=32, num_epochs=30, ) print(config)
{'algorithm_config': {'algorithm': 'n2v',
'loss': 'n2v',
'lr_scheduler': {'name': 'ReduceLROnPlateau',
'parameters': {}},
'model': {'architecture': 'UNet',
'conv_dims': 2,
'depth': 2,
'final_activation': 'None',
'in_channels': 1,
'independent_channels': True,
'n2v2': False,
'num_channels_init': 32,
'num_classes': 1},
'optimizer': {'name': 'Adam',
'parameters': {'lr': 0.0001}}},
'data_config': {'axes': 'YX',
'batch_size': 32,
'data_type': 'array',
'patch_size': [64, 64],
'transforms': [{'flip_x': True,
'flip_y': True,
'name': 'XYFlip',
'p': 0.5},
{'name': 'XYRandomRotate90', 'p': 0.5},
{'masked_pixel_percentage': 0.2,
'name': 'N2VManipulate',
'roi_size': 11,
'strategy': 'uniform',
'struct_mask_axis': 'none',
'struct_mask_span': 5}]},
'experiment_name': 'sem_n2v',
'training_config': {'accumulate_grad_batches': 1,
'check_val_every_n_epoch': 1,
'checkpoint_callback': {'auto_insert_metric_name': False,
'mode': 'min',
'monitor': 'val_loss',
'save_last': True,
'save_top_k': 3,
'save_weights_only': False,
'verbose': False},
'enable_progress_bar': True,
'gradient_clip_algorithm': 'norm',
'max_steps': -1,
'num_epochs': 30,
'precision': '32'},
'version': '0.1.0'}
Train¶
A CAREamist can be created using a configuration alone, and then be trained by using the data already loaded in memory.
In [ ]:
Copied!
# instantiate a CAREamist
careamist = CAREamist(source=config)
# train
careamist.train(
train_source=train_image,
val_source=val_image,
)
# instantiate a CAREamist careamist = CAREamist(source=config) # train careamist.train( train_source=train_image, val_source=val_image, )
Predict with CAREamics¶
Prediction is done with the same CAREamist used for training. Because the image is large we predict using tiling.
In [ ]:
Copied!
prediction = careamist.predict(source=train_image, tile_size=(256, 256))
prediction = careamist.predict(source=train_image, tile_size=(256, 256))
Visualize the prediction¶
In [8]:
Copied!
# Show the full image and crops
x_start, x_end = 600, 850
y_start, y_end = 200, 450
fig, ax = plt.subplots(2, 2, figsize=(10, 10))
ax[0, 0].imshow(train_image, cmap="gray")
ax[0, 1].imshow(prediction[0].squeeze(), cmap="gray")
ax[1, 0].imshow(train_image[y_start:y_end, x_start:x_end], cmap="gray")
ax[1, 1].imshow(prediction[0].squeeze()[y_start:y_end, x_start:x_end], cmap="gray")
# Show the full image and crops x_start, x_end = 600, 850 y_start, y_end = 200, 450 fig, ax = plt.subplots(2, 2, figsize=(10, 10)) ax[0, 0].imshow(train_image, cmap="gray") ax[0, 1].imshow(prediction[0].squeeze(), cmap="gray") ax[1, 0].imshow(train_image[y_start:y_end, x_start:x_end], cmap="gray") ax[1, 1].imshow(prediction[0].squeeze()[y_start:y_end, x_start:x_end], cmap="gray")
Out[8]:
<matplotlib.image.AxesImage at 0x7fb508551270>
Export the model¶
The model is automatically saved during training (the so-called checkpoints) and can be loaded back easily, but you can also export the model to the BioImage Model Zoo format.
In [9]:
Copied!
# create a cover image
x_start, width = 500, 512
y_start, height = 1400, 512
# create image
cover = np.zeros((height, width))
# normalize train and prediction
norm_train = (train_image - train_image.min()) / (train_image.max() - train_image.min())
pred = prediction[0].squeeze()
norm_pred = (pred - pred.min()) / (pred.max() - pred.min())
# fill in halves
cover[:, :width // 2] = norm_train[y_start:y_start + height, x_start:x_start + width // 2]
cover[:, width // 2:] = norm_pred[y_start:y_start + height, x_start + width // 2:x_start + width]
# plot the single image
plt.imshow(cover, cmap="gray")
# save the image
im = Image.fromarray(cover * 255)
im = im.convert('L')
im.save("SEM_Noise2Void.jpeg")
# create a cover image x_start, width = 500, 512 y_start, height = 1400, 512 # create image cover = np.zeros((height, width)) # normalize train and prediction norm_train = (train_image - train_image.min()) / (train_image.max() - train_image.min()) pred = prediction[0].squeeze() norm_pred = (pred - pred.min()) / (pred.max() - pred.min()) # fill in halves cover[:, :width // 2] = norm_train[y_start:y_start + height, x_start:x_start + width // 2] cover[:, width // 2:] = norm_pred[y_start:y_start + height, x_start + width // 2:x_start + width] # plot the single image plt.imshow(cover, cmap="gray") # save the image im = Image.fromarray(cover * 255) im = im.convert('L') im.save("SEM_Noise2Void.jpeg")
In [10]:
Copied!
general_description = (
"This model is a UNet trained using the Noise2Void algorithm to denoise "
"images. The training data consists of crops from an SEM dataset "
"(T.-O. Buchholz et al., Methods Cell Biol, 2020). The notebook used to "
"train this model is available on the CAREamics documentation website; "
"find it at the following link: "
"https://careamics.github.io/0.1/applications/Noise2Void/SEM/."
)
print(general_description)
general_description = ( "This model is a UNet trained using the Noise2Void algorithm to denoise " "images. The training data consists of crops from an SEM dataset " "(T.-O. Buchholz et al., Methods Cell Biol, 2020). The notebook used to " "train this model is available on the CAREamics documentation website; " "find it at the following link: " "https://careamics.github.io/0.1/applications/Noise2Void/SEM/." ) print(general_description)
This model is a UNet trained using the Noise2Void algorithm to denoise images. The training data consists of crops from an SEM dataset (T.-O. Buchholz et al., Methods Cell Biol, 2020). The notebook used to train this model is available on the CAREamics documentation website; find it at the following link: https://careamics.github.io/0.1/applications/Noise2Void/SEM/.
In [ ]:
Copied!
# Export the model
careamist.export_to_bmz(
path_to_archive="sem_n2v_model.zip",
friendly_model_name="SEM_N2V",
input_array=train_image[1400:1656, 500:756],
authors=[{"name": "CAREamics authors", "affiliation": "Human Technopole"}],
general_description=general_description,
data_description=portfolio.denoising.N2V_SEM.description
)
# Export the model careamist.export_to_bmz( path_to_archive="sem_n2v_model.zip", friendly_model_name="SEM_N2V", input_array=train_image[1400:1656, 500:756], authors=[{"name": "CAREamics authors", "affiliation": "Human Technopole"}], general_description=general_description, data_description=portfolio.denoising.N2V_SEM.description )