I’m encountering an error while attempting to convert Tfrecords for the KITTI trainval dataset using the TAO Toolkit in a Jupyter Notebook environment. The error message I’m receiving is as follows:
Converting Tfrecords for KITTI trainval dataset
2024-04-04 13:02:23,884 [TAO Toolkit] [INFO] root 160: Registry: ['nvcr.io']
2024-04-04 13:02:24,010 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 361: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5
2024-04-04 13:02:24,033 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 301: Printing tty value True
Docker instantiation failed with error: 500 Server Error: Internal Server Error ("failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'csv'
invoking the NVIDIA Container Runtime Hook directly (e.g. specifying the docker --gpus flag) is not supported. Please use the NVIDIA Container Runtime (e.g. specify the --runtime=nvidia flag) instead.: unknown")
I’m seeking assistance in resolving this issue. From my research, it appears to be related to Docker instantiation and the use of the NVIDIA Container Runtime. However, I’m unsure how to resolve it.
Could someone please provide guidance on how to properly configure Docker to use the NVIDIA Container Runtime with the --runtime=nvidia flag? Any insights or troubleshooting steps would be greatly appreciated.
Thank you!
2 posts - 2 participants