Hardware: RTX3080Ti
Network: Mask2Former
Docker image: nvcr.io/nvidia/tao/tao-toolkit:5.5.0-pyt
Spec file for training and validation:
exp_mask2former.txt (2.1 KB)
Issue:
During validation of the Mask2Former, the accuracy and mIoU metrics are always 1.0. This is obviously incorrect and should be lower. The issue occurs when validating on coco panoptic as well as coco instance annotations.
Troubleshooting:
Taking a look at the source code of the TAO pytorch backend, it looks like the dataset classes (tao_pytorch_backend/nvidia_tao_pytorch/cv/mask2former/dataloader/datasets.py at main · NVIDIA/tao_pytorch_backend · GitHub) used for coco always convert the segmentations to a semantic segmentation map.
Also, the predicted segmentation map passed to calculate the evaluation metrics always seem to be 0 in the validation_step() method in the pytorch lightning model (tao_pytorch_backend/nvidia_tao_pytorch/cv/mask2former/model/pl_model.py at main · NVIDIA/tao_pytorch_backend · GitHub).
Is there a way to fix the evaluation for the Mask2Former model for instance segmentation?
1 post - 1 participant