Please provide the following information when requesting support.
• Hardware (T4/V100/Xavier/Nano/etc): A5000 24 GB
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc): RetinaNet
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here): 5.2.0
• Training spec file(If have, please share here)
retinanet_retrain_resnet18_custom_ds.txt (1.6 KB)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)
I want to run inference of the Tao RetinaNet model in Tritonserver (tritonserver:23.06-py3) using TensorRT, but, I am getting wrong predictions compared to running the model using “tao deploy retinanet inference …”. Predictions are not completely wrong, but they are less good than in TAO deploy.
I am using the samel TensorRT plan for both and I double checked TensorRT and Cuda version in TAO deploy retinanet docker and tritonserver docker and both have the same version number (TensorRT 8.6.1. and Cuda 12.1).
For tritonserver, I used model config like this: tao-toolkit-triton-apps/model_repository/retinanet_tao/config.pbtxt at main · NVIDIA-AI-IOT/tao-toolkit-triton-apps · GitHub
I assume the difference is due to wrong pre and postprocessing of the images/model output. For triton server, I am normalising images using these rgb mean values: [123.7, 116.779, 103.9] and do output postprocessing like this tao-toolkit-triton-apps/tao_triton/python/postprocessing/retinanet_postprocessor.py at main · NVIDIA-AI-IOT/tao-toolkit-triton-apps · GitHub.
How do i have to perform pre/postprocessing to get similar prediction in tritonserver as in tao deploy retinanet?
2 posts - 2 participants