DINO Model performance when converted to TensorRT and Inference through Triton

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc) Dual A6000
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc) DINO
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here) 5.02
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.) Convert .pth to TensorRT model and deploy to Triton IS

Hello,

I created a custom dataset and used DINO to train a model. I got very good results (mAP>80% mAP50 >94%). I then exported the model to TensortRT using the Deploy gen_trt_engine option using the following spec file:

gen_trt_engine:
gpu_id: 0
input_width: 800
input_height: 800
tensorrt:
data_type: fp32
workspace_size: 4096
min_batch_size: 1
opt_batch_size: 8
max_batch_size: 8
dataset:
num_classes: 2
batch_size: -1
model:
backbone: fan_tiny

I then ran Poloygraphy on the model.plan file and obtained this:

[W] ‘colored’ module is not installed, will not use colors when logging. To enable colors, please install the ‘colored’ module: python3 -m pip install colored
[I] Loading bytes from /trt_optimize/Dino/Experiment1/results/tensorRT/model.plan
[I] ==== TensorRT Engine ====
Name: Unnamed Network 0 | Explicit Batch Engine

---- 1 Engine Input(s) ----
{inputs [dtype=float32, shape=(-1, 3, 800, 800)]}

---- 2 Engine Output(s) ----
{pred_logits [dtype=float32, shape=(-1, 300, 2)],
 pred_boxes [dtype=float32, shape=(-1, 300, 4)]}

---- Memory ----
Device Memory: 4645939200 bytes

---- 1 Profile(s) (3 Tensor(s) Each) ----
- Profile: 0
    Tensor: inputs               (Input), Index: 0 | Shapes: min=(1, 3, 800, 800), opt=(8, 3, 800, 800), max=(8, 3, 800, 800)
    Tensor: pred_logits         (Output), Index: 1 | Shape: (-1, 300, 2)
    Tensor: pred_boxes          (Output), Index: 2 | Shape: (-1, 300, 4)

---- 375 Layer(s) ----

I then created a config.pbtxt as follows:

name: “WindowService”
backend: “tensorrt”
max_batch_size: 1

input: [
{
name: “inputs”,
data_type: TYPE_FP32,
dims: [ 3, 800, 800]
}
]

output: [
{
name: “pred_logits”,
data_type: TYPE_FP32,
dims: [ 300, 2 ]
},
{
name: “pred_boxes”,
data_type: TYPE_FP32,
dims: [ 300, 4 ]
}
]

I’m using a python client:

import tritonclient.http as httpclient

I’ve used this approach before successfully with Triton. I get back 2 output tensors as expected. The first (pred_logits) has a shape of [300 2] (300 from the number of observations as set in a training.yaml) and the 2 numbers in the second dimension are confidence of background and forground respectively. I note that the majority of those numbers are above 1, suggesting that they are logits (as the name implies) and not been through a softmax function.

The pred_boxes have a shape of [300 4], with the second dimension detailing the 4 coordinate points for the bounding box. It is further assumed that these points are x1,y1,x2,y2 respectively.

My problem is that the bounding boxes make no sense insomuch that they do not represent any foreground object (windows in this case).

When I run the exact same image through TAO inference (using the .pth model) then the resulting boxes placed on the image and the result text files are correct, so it is not an image issue.

What other processing needs to be done to the bounding box output from Triton?

Cheers

3 posts - 2 participants

Read full topic