Image classification

Description of the task

The goal of this workflow is to assign a category (or class) to every input image.

In the figure below a few examples of this workflow’s input are depicted:

../_images/MedMNIST_DermaMNIST_test1008_0.png
../_images/MedMNIST_DermaMNIST_test10_1.png
../_images/MedMNIST_DermaMNIST_test1002_2.png
../_images/MedMNIST_DermaMNIST_test1030_3.png
../_images/MedMNIST_DermaMNIST_test1003_4.png
../_images/MedMNIST_DermaMNIST_test0_5.png
../_images/MedMNIST_DermaMNIST_test1021_6.png

Each of these examples are of a different class and were obtained from MedMNIST v2 ([12]), concretely from DermaMNIST dataset which is a large collection of multi-source dermatoscopic images of common pigmented skin lesions.

Inputs and outputs

The image classification workflows in BiaPy expect a series of folders as input:

  • Training Raw Images: A folder that contains the unprocessed (single-channel or multi-channel) images that will be used to train the model. As explained later, all images of the same category are expected to be in the same sub-folder.

    Expand to see how to configure

    In the current BiaPy GUI, this option is defined through the Wizard questions. Alternatively, you can edit the DATA.TRAIN.PATH in your YAML file before clicking Run Workflow and loading that YAML file.

  • [Optional] Test Raw Images: A folder that contains the images to evaluate the model's performance. Optionaly, if the category of each test image is known, all images of the same category are expected to be in the same sub-folder.
    Expand to see how to configure

    In the current BiaPy GUI, this option is defined through the Wizard questions. Alternatively, you can edit the DATA.TEST.PATH in your YAML file before clicking Run Workflow and loading that YAML file.

Upon successful execution, a directory will be generated with the results of the classification. Therefore, you will need to define:

  • Output Folder: A designated path to save the classification outcomes.

    Expand to see how to configure

    Under Run Workflow, click on the Browse button of Output folder to save the results:

    ../_images/GUI-run-workflow.png
Graphical description of minimal inputs and outputs in BiaPy for image classification.

BiaPy input and output folders for image classification. Notice the test folder
and its sub-folders are optional.

Data structure

To ensure the proper operation of the workflow, the directory tree should be something like this:

dataset/
β”œβ”€β”€ train
β”‚   β”œβ”€β”€ class_0
β”‚   β”‚   β”œβ”€β”€ train0_0.png
β”‚   β”‚   β”œβ”€β”€ train1013_0.png
β”‚   β”‚   β”œβ”€β”€ . . .
β”‚   β”‚   └── train932_0.png
β”‚   β”œβ”€β”€ class_1
β”‚   β”‚   β”œβ”€β”€ train104_1.png
β”‚   β”‚   β”œβ”€β”€ train1049_1.png
β”‚   β”‚   β”œβ”€β”€ . . .
β”‚   β”‚   └── train964_1.png
| . . .
β”‚   └── class_6
β”‚       β”œβ”€β”€ train1105_6.png
β”‚       β”œβ”€β”€ train1148_6.png
β”‚       β”œβ”€β”€ . . .
β”‚       └── train98_6.png
└── test
    β”œβ”€β”€ class_0
    β”‚   β”œβ”€β”€ test1008_0.png
    β”‚   β”œβ”€β”€ test1084_0.png
    β”‚   β”œβ”€β”€ . . .
    β”‚   └── test914_0.png
    β”œβ”€β”€ class_1
    β”‚   β”œβ”€β”€ test10_1.png
    β”‚   β”œβ”€β”€ test1034_1.png
    β”‚   β”œβ”€β”€ . . .
    β”‚   └── test984_1.png
  . . .
    └── class_6
        β”œβ”€β”€ test1021_6.png
        β”œβ”€β”€ test1069_6.png
        β”œβ”€β”€ . . .
        └── test806_6.png

Each image category is obtained from the sub-folder name in which that image resides. That is why is so important to follow the directory tree as described above. If you have a .csv file with each image category, as is provided by MedMNIST v2, you can use our script from_class_csv_to_folders.py to create such directory tree.

The sub-folder names can be any number or string. They will be considered as the class names. Regarding the test, if you have no classes it doesn’t matter if the images are separated in several folders or are all in one folder.

Example datasets

Below is a list of publicly available datasets that are ready to be used in BiaPy for image classification:

Example dataset

Image dimensions

Link to data

DermaMNIST

2D

DermaMNIST.zip

OrganMNIST3D

3D

organMNIST3D.zip

Butterfly Image Classification

2D

butterfly_data.zip

Minimal configuration

Apart from the input and output folders, there are a few basic parameters that always need to be specified in order to run an image classification workflow in BiaPy. Depending on the parameter, they can be defined through the GUI Wizard, in the code-free notebooks, or by editing the YAML configuration file.

Experiment name

Also known as β€œmodel name” or β€œjob name”, this will be the name of the current experiment you want to run, so it can be differenciated from other past and future experiments.

Expand to see how to configure

Under Run Workflow, type the name you want for the job in the Job name field:

../_images/GUI-run-workflow.png

Note

Use only my_model -style, not my-model (Use β€œ_” not β€œ-β€œ). Do not use spaces in the name. Avoid using the name of an existing experiment/model/job (saved in the same result folder) as it will be overwritten.

Data management

Validation Set

With the goal to monitor the training process, it is common to use a third dataset called the β€œValidation Set”. This is a subset of the whole dataset that is used to evaluate the model’s performance and optimize training parameters. This subset will not be directly used for training the model, and thus, when applying the model to these images, we can see if the model is learning the training set’s patterns too specifically or if it is generalizing properly.

Graphical description of data partitions in BiaPy

Graphical description of data partitions in BiaPy.

To define such set, there are two options:

  • Validation proportion/percentage: Select a proportion (or percentage) of your training dataset to be used to validate the network during the training. Usual values are 0.1 (10%) or 0.2 (20%), and the samples of that set will be selected at random.

    Expand to see how to configure

    In the current BiaPy GUI, this option is configured by editing the DATA.VAL.SPLIT_TRAIN in your YAML file before clicking Run Workflow and loading that YAML file.

  • Validation path: Similar to the training set, you can select a folder that contains the unprocessed (single-channel or multi-channel) raw images that will be used to validate the current model during training. As it happened with the training images, all images of the same category are expected to be in the same sub-folder.

    Expand to see how to configure

    In the current BiaPy GUI, this option is configured by editing the DATA.VAL.PATH in your YAML file before clicking Run Workflow and loading that YAML file.

Test ground-truth

Do you have labels (classes) for the test set? This is a key question so BiaPy knows if your test set will be used for evaluation in new data (unseen during training) or simply produce predictions on that new data. All supervised workflows contain a parameter to specify this aspect.

Expand to see how to configure

In the current BiaPy GUI, this option is defined through the Wizard questions. Alternatively, you can edit the DATA.TEST.LOAD_GT in your YAML file before clicking Run Workflow and loading that YAML file.

Basic training parameters

At the core of each BiaPy workflow there is a deep learning model. Although we try to simplify the number of parameters to tune, these are the basic parameters that need to be defined for training an image classification workflow:

  • Number of classes: The number of classes present in the problem. It must be equal to the number of subfolders in the training folder.

    Expand to see how to configure

    In the current BiaPy GUI, this option is configured by editing the MODEL.N_CLASSES in your YAML file before clicking Run Workflow and loading that YAML file.

  • Number of input channels: The number of channels of your raw images (grayscale = 1, RGB = 3). Notice the dimensionality of your images (2D/3D) is set by default depending on the workflow template you select.

    Expand to see how to configure

    In the current BiaPy GUI, this option is configured by editing the DATA.PATCH_SIZE in your YAML file before clicking Run Workflow and loading that YAML file.

  • Number of epochs: This number indicates how many rounds the network will be trained. On each round, the network usually sees the full training set. The value of this parameter depends on the size and complexity of each dataset. You can start with something like 100 epochs and tune it depending on how fast the loss (error) is reduced.

    Expand to see how to configure

    In the current BiaPy GUI, this option is configured by editing the TRAIN.EPOCHS in your YAML file before clicking Run Workflow and loading that YAML file.

  • Patience: This is a number that indicates how many epochs you want to wait without the model improving its results in the validation set to stop training. Again, this value depends on the data you’re working on, but you can start with something like 20.

    Expand to see how to configure

    In the current BiaPy GUI, this option is configured by editing the TRAIN.PATIENCE in your YAML file before clicking Run Workflow and loading that YAML file.

For improving performance, other advanced parameters can be optimized, for example, the model’s architecture. The architecture assigned as default is the ViT, as it is effective in image classification tasks. This architecture allows a strong baseline, but further exploration could potentially lead to better results.

Note

Once the parameters are correctly assigned, the training phase can be executed. Note that to train large models effectively the use of a GPU (Graphics Processing Unit) is essential. This hardware accelerator performs parallel computations and has larger RAM memory compared to the CPUs, which enables faster training times.

How to run

BiaPy offers different options to run workflows depending on your degree of computer expertise. Select whichever is more approppriate for you:

In the BiaPy GUI, click on the Wizard, then follow the next instructions to select the image classification workflow:

Note

BiaPy’s GUI requires that all data and configuration files reside on the same machine where the GUI is being executed.

Tip

If you need additional help, watch BiaPy’s GUI walkthrough video.

Templates

In the templates/classification folder of BiaPy, you can find a few YAML configuration templates for this workflow.

[Advanced] Special workflow configuration

Note

This section is recommended for experienced users only to improve the performance of their workflows. When in doubt, do not hesitate to check our FAQ & Troubleshooting or open a question in the image.sc discussion forum.

Advanced Parameters

Many of the parameters of our workflows are set by default to values that work commonly well. However, it may be needed to tune them to improve the results of the workflow. For instance, you may modify the following parameters

  • Model architecture: Select the architecture of the deep neural network used as backbone of the pipeline. ViT, EfficientNetB0, EfficientNetB1, EfficientNetB2, EfficientNetB3, EfficientNetB4, EfficientNetB5, EfficientNetB6, EfficientNetB7 and simple CNN. Default value: ViT.

  • Batch size: This parameter defines the number of patches seen in each training step. Reducing or increasing the batch size may slow or speed up your training, respectively, and can influence network performance. Common values are 4, 8, 16, etc.

  • Patch size: Input the size of the patches use to train your model (length in pixels in X and Y). The value should be smaller or equal to the dimensions of the image. The default value is 256 in 2D, i.e. 256x256 pixels.

  • Optimizer: Select the optimizer used to train your model. Options: ADAM, ADAMW, Stochastic Gradient Descent (SGD). ADAM usually converges faster, while ADAMW provides a balance between fast convergence and better handling of weight decay regularization. SGD is known for better generalization. Default value: ADAMW.

  • Initial learning rate: Input the initial value to be used as learning rate. If you select ADAM as optimizer, this value should be around 10e-4.

  • Learning rate scheduler: Select to adjust the learning rate between epochs. The current options are β€œReduce on plateau”, β€œOne cycle”, β€œWarm-up cosine decay” or no scheduler.

  • Test time augmentation (TTA): Select to apply augmentation (flips and rotations) at test time. It usually provides more robust results but uses more time to produce each result. By default, no TTA is peformed.

Metrics

During the inference phase the performance of the test data is measured using different metrics if test masks were provided (i.e. ground truth) and, consequently, DATA.TEST.LOAD_GT is True. In the case of classification the accuracy, precision, recall, and F1 are calculated. Apart from that, the confusion matrix is also printed.

Results

The main output of this workflow will be a file named predictions.csv that will contain the predicted image class:

../_images/classification_csv_output.svg

Classification workflow output

All files are placed in results folder under --result_dir directory with the --name given. Following the example, you should see that the directory /home/user/exp_results/classification has been created. If the same experiment is run 5 times, varying --run_id argument only, you should find the following directory tree:

Expand directory tree
my_2d_classification/
β”œβ”€β”€ config_files
β”‚   └── 2d_classification.yaml
β”œβ”€β”€ checkpoints
β”‚   └── model_weights_classification_1.h5
└── results
    β”œβ”€β”€ my_2d_classification_1
    β”œβ”€β”€ . . .
    └── my_2d_classification_5
        β”œβ”€β”€ predictions.csv
        β”œβ”€β”€ aug
        β”‚   └── .tif files
        β”œβ”€β”€ charts
        β”‚   β”œβ”€β”€ my_2d_classification_1_*.png
        β”‚   └── my_2d_classification_1_loss.png
        β”œβ”€β”€ train_logs
        └── tensorboard

  • config_files: directory where the .yaml filed used in the experiment is stored.

    • 2d_classification.yaml: YAML configuration file used (it will be overwrited every time the code is run).

  • checkpoints, optional: directory where model’s weights are stored. Only created when TRAIN.ENABLE is True and the model is trained for at least one epoch.

    • model_weights_my_2d_classification_1.h5, optional: checkpoint file (best in validation) where the model’s weights are stored among other information. Only created when the model is trained for at least one epoch.

    • normalization_mean_value.npy, optional: normalization mean value. Is saved to not calculate it everytime and to use it in inference. Only created if DATA.NORMALIZATION.TYPE is custom.

    • normalization_std_value.npy, optional: normalization std value. Is saved to not calculate it everytime and to use it in inference. Only created if DATA.NORMALIZATION.TYPE is custom.

  • results: directory where all the generated checks and results will be stored. There, one folder per each run are going to be placed.

    • my_2d_classification_1: run 1 experiment folder. Can contain:

      • predictions.csv: list of assigned class per test image.

      • aug, optional: image augmentation samples. Only created if AUGMENTOR.AUG_SAMPLES is True.

      • charts, optional. Only created when TRAIN.ENABLE is True and epochs trained are more or equal LOG.CHART_CREATION_FREQ:

        • my_2d_classification_1_*.png: plot of each metric used during training.

        • my_2d_classification_1_loss.png: loss over epochs plot.

      • train_logs: each row represents a summary of each epoch stats. Only avaialable if training was done.

      • tensorboard: tensorboard logs.

      • test_results_metrics.csv: a CSV file containing all the evaluation metrics obtained on each file of the test set if ground truth was provided.

Note

Here, for visualization purposes, only my_2d_classification_1 has been described but my_2d_classification_2, my_2d_classification_3, my_2d_classification_4 and my_2d_classification_5 directories will follow the same structure.