Run inference on video files using Mask_Rcnn

Dear @Morganh

I want to run inference on video using mask rcnn model trained using tao. My requirement is that I want to run mask rcnn model using Deepstream on video file otherwise we have created script which perform inference on image and that can be update for video easily.

To run this model on deepstream container I tried below approach.

1. With tao command

root@smarg:~/data/Pritam/Script/SEGMENTATION/DS_SEGMENTATION/MULTI_STREAM_SEGMENTATION/deepstream_tao_apps-release-tao3.0_ds6.1ga/apps/tao_segmentation# ./ds-tao-segmentation -c ../../../SCRIPT/BOX-SEGMENT-CONFIG-TAO.txt -i /root/data/Pritam/TAO/Model-Training/BOX-SEGMENTATION_V1.0/mask_rcnn/VIDEO/CONTAINER_1.h264 -b 1 -d 1

BOX-SEGMENT-CONFIG-TAO.txt

[property]
net-scale-factor=0.017507
offsets=123.675;116.28;103.53

gpu-id=0
net-scale-factor=1.0
model-color-format=0
tlt-model-key=
uff-file=/root/data/Pritam/Script/SEGMENTATION/MODELS/model.epoch-24.uff
model-engine-file=/root/data/Pritam/Script/SEGMENTATION/MODELS/model.epoch-24.uff_b1_gpu0_fp16.engine
infer-dims=3;640;640
uff-input-order=0
uff-input-blob-name=Input
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=2
interval=0
gie-unique-id=1
network-type=3
#output-blob-names=generate_detections;mask_fcn_logits/BiasAdd
output-blob-names=generate_detections
output-tensor-meta=0
segmentation-threshold=0.0

#parse-bbox-func-name=NvDsInferParseCustomSSD
#custom-lib-path=nvdsinfer_custom_impl_ssd/libnvdsinfer_custom_impl_ssd.so
#scaling-filter=0
#scaling-compute-hw=0
parse-bbox-instance-mask-func-name=NvDsInferParseCustomMrcnnTLTV2

custom-lib-path=/opt/nvidia/deepstream/deepstream/lib/libnvds_infercustomparser.so


cluster-mode=4
output-instance-mask=1

[class-attrs-all]
pre-cluster-threshold=0.6

OUTPUT:

I am unable to see a single output on a video but the images of same video giving output using our mask_rcnn inference below is out of mask_rcnn inference

I also explored some other sample application but not getting any single detection using deepstream or deepstream-tao. Can u please suggest how to run infer on video using mask rcnn inside deepstream or deepstream-tao. Is there any sample application available then please suggest.

Thanks.

5 posts - 2 participants

Read full topic