Iterable Dataset
Iterable dataset used to load data file by file.
PathIterableDataset
Bases: IterableDataset
Dataset allowing extracting patches w/o loading whole data into memory.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_config
|
DataConfig
|
Data configuration. |
required |
src_files
|
list of pathlib.Path
|
List of data files. |
required |
target_files
|
list of pathlib.Path
|
Optional list of target files, by default None. |
None
|
read_source_func
|
Callable
|
Read source function for custom types, by default read_tiff. |
read_tiff
|
Attributes:
| Name | Type | Description |
|---|---|---|
data_path |
list of pathlib.Path
|
Path to the data, must be a directory. |
axes |
str
|
Description of axes in format STCZYX. |
get_data_statistics()
Return training data statistics.
Returns:
| Type | Description |
|---|---|
tuple of list of floats
|
Means and standard deviations across channels of the training data. |
get_number_of_files()
Return the number of files in the dataset.
Returns:
| Type | Description |
|---|---|
int
|
Number of files in the dataset. |
split_dataset(percentage=0.1, minimum_number=5)
Split up dataset in two.
Splits the datest sing a percentage of the data (files) to extract, or the minimum number of the percentage is less than the minimum number.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
percentage
|
float
|
Percentage of files to split up, by default 0.1. |
0.1
|
minimum_number
|
int
|
Minimum number of files to split up, by default 5. |
5
|
Returns:
| Type | Description |
|---|---|
IterableDataset
|
Dataset containing the split data. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the percentage is smaller than 0 or larger than 1. |
ValueError
|
If the minimum number is smaller than 1 or larger than the number of files. |