The goal of this workflow is to localize objects in the input image, not requiring a pixel-level class. Common strategies produce either bounding boxes containing the objects or individual points at their center of mass [ZWKrahenbuhl19], which is the one adopted by BiaPy.
.csvfile containing the list of points to detect.
Image with the detected points as white dots.
.csvfile with the list of detected points in napari format.
_prob.csvfile with the same list of points as above but now with their detection probability (also in napary format).
In the figure below an example of this workflow’s input is depicted:
Description of the
Each row represents the middle point of the object to be detected. Each column is a coordinate in the image dimension space.
The first column name does not matter but it needs to be there. No matter also the enumeration and order for that column.
If the images are
3D, three columns need to be present and their names must be
[axis-0, axis-1, axis-2], which represent
(z,y,x)axes. If the images are
2D, only two columns are required
[axis-0, axis-1], which represent
For multi-class detection problem, i.e.
MODEL.N_CLASSES > 1, add an additional
classcolumn to the file. The classes need to start from
1and consecutive, i.e.
1,2,3,4...and not like
Coordinates can be float or int but they will be converted into ints so they can be translated to pixels.
To ensure the proper operation of the library the data directory tree should be something like this:
dataset/ ├── train │ ├── x │ │ ├── training-0001.tif │ │ ├── training-0002.tif │ │ ├── . . . │ │ ├── training-9999.tif │ └── y │ ├── training-0001.csv │ ├── training-0002.csv │ ├── . . . │ ├── training-9999.csv └── test ├── x │ ├── testing-0001.tif │ ├── testing-0002.tif │ ├── . . . │ ├── testing-9999.tif └── y ├── testing-0001.csv ├── testing-0002.csv ├── . . . ├── testing-9999.csv
In this workflow the name of each
.tif file and its corresponding
.csv file must be the same.
Firstly, a pre-processing step is done where the list of points of the
.csv file is transformed into point mask images. During this process some checks are made to ensure there is not repeated point in the
.csv. This option is enabled by default with
PROBLEM.DETECTION.CHECK_POINTS_CREATED so if any problem is found the point mask of that
.csv will not be created until the problem is solve.
After the train phase, the model output will be an image where each pixel of each channel will have the probability (in
[0-1] range) of being of the class that represents that channel. The image would be something similar to the left picture below:
So those probability images, as the left picture above, can be converted into the final points, as the rigth picture above, we use peak_local_max function to find peaks in those probability clouds. For that, you need to define a threshold,
TEST.DET_MIN_TH_TO_BE_PEAK variable in our case, for the minimum probability to be considered as a point. You can set different thresholds for each class in
After the points have been created a post-processing step begins. In this workflow this methods are available:
Remove close points: In the process of image segmentation, it may be necessary to remove points that are close to each other in order to improve the accuracy of the segmentation. The variable
TEST.POST_PROCESSING.REMOVE_CLOSE_POINTScan be used to remove points that are within a certain radius of each other. The radius value can be specified using the variable
TEST.POST_PROCESSING.REMOVE_CLOSE_POINTS_RADIUS. You can set different radius for each class, e.g.
[0.7,0.9]. In this post-processing is important to set
DATA.TEST.RESOLUTION, specially for
3Ddata where the resolution in
zdimension is usually less than in other axes. That resolution will be taken into account when removing points.
Create instances from points: Once the points have been detected and any close points have been removed, it is possible to create instances from the remaining points. The variable
TEST.POST_PROCESSING.DET_WATERSHEDcan be set to perform this step. However, sometimes cells have low contrast in their centers, for example due to the presence of a nucleus. This can result in the seed growing to fill only the nucleus while the cell is much larger. In order to address the issue of limited growth of certain types of seeds, a process has been implemented to expand the seeds beyond the borders of their nuclei. This process allows for improved growth of these seeds. To ensure that this process is applied only to the appropriate cells, variables such as
TEST.POST_PROCESSING.DET_WATERSHED_DONUTS_NUCLEUS_DIAMETERhave been created. It is important to note that these variables are necessary to prevent the expansion of the seed beyond the boundaries of the cell, which could lead to expansion into the background.
Find in templates/detection folder of BiaPy a few YAML configuration templates for this workflow.
Special workflow configuration
Here some special configuration options that can be selected in this workflow are described:
Metrics: during the inference phase the performance of the test data is measured using different metrics if test masks were provided (i.e. ground truth) and, consequently,
DATA.TEST.LOAD_GTis enabled. In the case of detection the Intersection over Union (IoU), precision, recall and F1 are calculated:
IoU metric, also referred as the Jaccard index, is essentially a method to quantify the percent of overlap between the target mask and the prediction output. Depending on the configuration different values are calculated (as explained in Test phase).
Precision, is the fraction of relevant instances among the retrieved instances. More info here.
Recall, is the fraction of relevant instances that were retrieved. More info here.
F1, is the harmonic mean of the precision and recall. More info here.
The last three metrics, i.e. precision, recall and F1, use
TEST.DET_TOLERANCEto determine when a point is considered as a true positive. In this process the test resolution is also taken into account. You can set different tolerances for each class, e.g.
Post-processing: you an use
TEST.POST_PROCESSING.REMOVE_CLOSE_POINTSto remove redundant close points to each other as described previously in Problem resolution.
# Configuration file job_cfg_file=/home/user/2d_detection.yaml # Where the experiment output directory should be created result_dir=/home/user/exp_results # Just a name for the job job_name=my_2d_detection # Number that should be increased when one need to run the same job multiple times (reproducibility) job_counter=1 # Number of the GPU to run the job in (according to 'nvidia-smi' command) gpu_number=0 # Move where BiaPy installation resides cd BiaPy # Load the environment conda activate BiaPy_env source $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh python -u main.py \ --config $job_cfg_file \ --result_dir $result_dir \ --name $job_name \ --run_id $job_counter \ --gpu $gpu_number
For multi-GPU training you can call BiaPy as follows:
gpu_number="0, 1, 2" python -u -m torch.distributed.run \ --nproc_per_node=3 \ main.py \ --config $job_cfg_file \ --result_dir $result_dir \ --name $job_name \ --run_id $job_counter \ --gpu $gpu_number
nproc_per_node need to be equal to the number of GPUs you are using (e.g.
# Configuration file job_cfg_file=/home/user/2d_detection.yaml # Path to the data directory data_dir=/home/user/data # Where the experiment output directory should be created result_dir=/home/user/exp_results # Just a name for the job job_name=my_2d_detection # Number that should be increased when one need to run the same job multiple times (reproducibility) job_counter=1 # Number of the GPU to run the job in (according to 'nvidia-smi' command) gpu_number=0 docker run --rm \ --gpus "device=$gpu_number" \ --mount type=bind,source=$job_cfg_file,target=$job_cfg_file \ --mount type=bind,source=$result_dir,target=$result_dir \ --mount type=bind,source=$data_dir,target=$data_dir \ danifranco/biapy \ -cfg $job_cfg_file \ -rdir $result_dir \ -name $job_name \ -rid $job_counter \ -gpu $gpu_number
data_dir must contain all the paths
DATA.*.GT_PATH so the container can find them. For instance, if you want to only train in this example
DATA.TRAIN.GT_PATH could be
The results are placed in
results folder under
--result_dir directory with the
Following the example, you should see that the directory
/home/user/exp_results/my_2d_detection has been created. If the same experiment is run 5 times, varying
--run_id argument only, you should find the following directory tree:
my_2d_detection/ ├── config_files/ │ └── my_2d_detection.yaml ├── checkpoints │ └── my_2d_detection_1-checkpoint-best.pth └── results ├── my_2d_detection_1 ├── . . . └── my_2d_detection_5 ├── cell_counter.csv ├── aug │ └── .tif files ├── charts │ ├── my_2d_detection_1_*.png │ ├── my_2d_detection_1_loss.png │ └── model_plot_my_2d_detection_1.png ├── per_image │ └── .tif files ├── per_image_local_max_check │ └── .tif files ├── point_associations │ ├── .tif files │ └── .csv files ├── train_logs └── tensorboard
config_files: directory where the .yaml filed used in the experiment is stored.
my_2d_detection.yaml: YAML configuration file used (it will be overwrited every time the code is run).
checkpoints: directory where model’s weights are stored.
my_2d_detection_1-checkpoint-best.pth: checkpoint file (best in validation) where the model’s weights are stored among other information.
normalization_mean_value.npy: normalization mean value (only created if
custom). Is saved to not calculate it everytime and to use it in inference.
normalization_std_value.npy: normalization std value (only created if
custom). Is saved to not calculate it everytime and to use it in inference.
results: directory where all the generated checks and results will be stored. There, one folder per each run are going to be placed.
my_2d_detection_1: run 1 experiment folder.
cell_counter.csv: file with a counter of detected objects for each test sample.
aug: image augmentation samples.
my_2d_detection_1_*.png: Plot of each metric used during training.
my_2d_detection_1_loss.png: Loss over epochs plot (when training is done).
model_plot_my_2d_detection_1.png: plot of the model.
.tif files: reconstructed images from patches.
.tif files: Same as
per_imagebut with the final detected points.
point_associations: optional. Only if ground truth was provided.
.tif files: prediction (
_pred_ids) and ground truth (
.csv files: false positives (
_fp) and ground truth associations (
train_logs: each row represents a summary of each epoch stats. Only avaialable if training was done.
tensorboard: Tensorboard logs.
Here, for visualization purposes, only
my_2d_detection_1 has been described but
my_2d_detection_5 will follow the same structure.