• Hardware : A5000
• Network Type : Mask_rcnn
• TLT Version tao 3.22.05
I tested the fps for my pruned MaskRCNN model and it had a lower FPS compared with the unpruned model, the test was done on an A5000 machine, and I remember the GPU compute time increased from ~20ms to ~28ms, the engine file was generated with batch size 1. I have done previous such experiments with different models trained on the same version of tao toolkit and have’nt noticed this behaviour. The pruned model is roughly 60% the original model’s size and my assumption here is that lower the number of parameters lower should be the time taken for inference.
2 posts - 2 participants