data_module_utils
Utility functions for file and paths solver.
InputType = ItemType | Sequence[ItemType] | None module-attribute #
Type of input data passed to the dataset.
ItemType = Path | str | NDArray[Any] module-attribute #
Type of input items passed to the dataset.
convert_paths_to_pathlib(input_data, target_data=None) #
Create a list of file paths from the input and target data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_data | Sequence[str | Path] | Input data, can be a path to a folder, or a list of paths. | required |
target_data | Sequence[str | Path] | None | Target data, can be None, a path to a folder, or a list of paths. | None |
Returns:
| Type | Description |
|---|---|
list[Path] | A list of file paths for input data. |
list[Path] | None | A list of file paths for target data, or None if target_data is None. |
Source code in src/careamics/lightning/dataset_ng/data_module_utils.py
initialize_data_pair(data_type, input_data, target_data=None, extension_filter='', custom_loader=False) #
Initialize a pair of input and target data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_type | Literal['array', 'tiff', 'zarr', 'czi', 'custom'] | The type of data to initialize. | required |
input_data | InputType | Input data, can be None, a path to a folder, a list of paths, or a numpy array. | required |
target_data | InputType | None | Target data, can be None, a path to a folder, a list of paths, or a numpy array. | None |
extension_filter | str | File extension filter to apply when listing files. | "" |
custom_loader | bool | Whether a custom image stack loader is used. | False |
Returns:
| Type | Description |
|---|---|
list[ndarray] | list[Path] | Initialized input data. For file paths, returns a list of Path objects. For numpy arrays, returns the arrays directly. |
list[ndarray] | list[Path] | None | Initialized target data. For file paths, returns a list of Path objects. For numpy arrays, returns the arrays directly. Returns None if target_data is None. |
Source code in src/careamics/lightning/dataset_ng/data_module_utils.py
list_files_in_directory(data_type, input_data, target_data=None, extension_filter='') #
List files from input and target directories.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_type | Literal['tiff', 'zarr', 'czi', 'custom'] | The type of data to validate. | required |
input_data | InputType | Input data, can be a path to a folder, a list of paths, or a numpy array. | required |
target_data | Optional[InputType] | Target data, can be None, a path to a folder, a list of paths, or a numpy array. | None |
extension_filter | str | File extension filter to apply when listing files. | "" |
Returns:
| Type | Description |
|---|---|
list[Path] | A list of file paths for input data. |
list[Path] | None | A list of file paths for target data, or None if target_data is None. |
Source code in src/careamics/lightning/dataset_ng/data_module_utils.py
validate_array_input(input_data, target_data) #
Validate if the input data is a numpy array.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_data | InputType | Input data, can be a path to a folder, a list of paths, or a numpy array. | required |
target_data | Optional[InputType] | Target data, can be None, a path to a folder, a list of paths, or a numpy array. | required |
Returns:
| Type | Description |
|---|---|
list[ndarray] | Validated input data. |
list[ndarray] | None | Validated target data, None if the target data is None. |
Raises:
| Type | Description |
|---|---|
ValueError | If the input data is not a numpy array or a list of numpy arrays. |
Source code in src/careamics/lightning/dataset_ng/data_module_utils.py
validate_input_target_type_consistency(input_data, target_data) #
Validate if the input and target data types are consistent.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_data | InputType | Input data, can be a path to a folder, a list of paths, or a numpy array. | required |
target_data | Optional[InputType] | Target data, can be None, a path to a folder, a list of paths, or a numpy array. | required |
Raises:
| Type | Description |
|---|---|
ValueError | If the input and target data types are not consistent. |
Source code in src/careamics/lightning/dataset_ng/data_module_utils.py
validate_path_input(data_type, input_data, target_data, extension_filter='') #
Validate if the input data is a path or a list of paths.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_type | Literal['tiff', 'zarr', 'czi', 'custom'] | The type of data to validate. | required |
input_data | str | Path | list[str | Path] | Input data, can be a path to a folder, a list of paths, or a numpy array. | required |
target_data | str | Path | list[str | Path] | None | Target data, can be None, a path to a folder, a list of paths, or a numpy array. | required |
extension_filter | str | File extension filter to apply when listing files. | "" |
Returns:
| Type | Description |
|---|---|
list[Path] | A list of file paths for input data. |
list[Path] | None | A list of file paths for target data, or None if target_data is None. |
Raises:
| Type | Description |
|---|---|
ValueError | If the input data is not a path or a list of paths. |
Source code in src/careamics/lightning/dataset_ng/data_module_utils.py
validate_zarr_input(input_data, target_data) #
Validate if the input data corresponds a zarr input.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_data | str | Path | list[str | Path] | Input data, can be a path to a folder, to zarr file, a URI pointing to a zarr dataset, or a list. | required |
target_data | str | Path | list[str | Path] | None | Target data, can be None. | required |
Returns:
| Type | Description |
|---|---|
list[str] or list[Path] | A list of zarr URIs or path for input data. |
list[str] or list[Path] | None | A list of zarr URIs or paths for target data, or None if target_data is None. |
Raises:
| Type | Description |
|---|---|
ValueError | If the input and target data types are not consistent. |
ValueError | If the input data is not a zarr URI or path, or a list of zarr URIs or paths. |
Source code in src/careamics/lightning/dataset_ng/data_module_utils.py
246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 | |