• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here) : 5.3.0
I’m getting following error while using tao train
WARNING: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/lprnet/scripts/train.py:82: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/lprnet/scripts/train.py", line 366, in <module>
main()
File "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/lprnet/scripts/train.py", line 362, in main
raise e
File "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/lprnet/scripts/train.py", line 345, in main
run_experiment(config_path=args.experiment_spec_file,
File "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/lprnet/scripts/train.py", line 86, in run_experiment
os.makedirs(results_dir)
File "/usr/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/usr/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/usr/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 3 more times]
File "/usr/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/home/mainak'
Execution status: FAIL
2024-06-03 15:32:19,782 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container.
here’s the detailed tao tool kit info:
task_group:
model:
dockers:
nvidia/tao/tao-toolkit:
5.0.0-tf2.11.0:
docker_registry: nvcr.io
tasks:
1. classification_tf2
2. efficientdet_tf2
5.0.0-tf1.15.5:
docker_registry: nvcr.io
tasks:
1. bpnet
2. classification_tf1
3. converter
4. detectnet_v2
5. dssd
6. efficientdet_tf1
7. faster_rcnn
8. fpenet
9. lprnet
10. mask_rcnn
11. multitask_classification
12. retinanet
13. ssd
14. unet
15. yolo_v3
16. yolo_v4
17. yolo_v4_tiny
5.3.0-pyt:
docker_registry: nvcr.io
tasks:
1. action_recognition
2. centerpose
3. deformable_detr
4. dino
5. mal
6. ml_recog
7. ocdnet
8. ocrnet
9. optical_inspection
10. pointpillars
11. pose_classification
12. re_identification
13. visual_changenet
14. classification_pyt
15. segformer
dataset:
dockers:
nvidia/tao/tao-toolkit:
5.3.0-data-services:
docker_registry: nvcr.io
tasks:
1. augmentation
2. auto_label
3. annotations
4. analytics
deploy:
dockers:
nvidia/tao/tao-toolkit:
5.3.0-deploy:
docker_registry: nvcr.io
tasks:
1. visual_changenet
2. centerpose
3. classification_pyt
4. classification_tf1
5. classification_tf2
6. deformable_detr
7. detectnet_v2
8. dino
9. dssd
10. efficientdet_tf1
11. efficientdet_tf2
12. faster_rcnn
13. lprnet
14. mask_rcnn
15. ml_recog
16. multitask_classification
17. ocdnet
18. ocrnet
19. optical_inspection
20. retinanet
21. segformer
22. ssd
23. trtexec
24. unet
25. yolo_v3
26. yolo_v4
27. yolo_v4_tiny
format_version: 3.0
toolkit_version: 5.3.0
published_date: 03/14/2024
here’s the tao_mounts.json:
{
"Mounts": [
{
"source": "/home/mainak/ms/getting_started_v5.3.0/notebooks/tao_launcher_starter_kit/lprnet",
"destination": "/workspace/tao-experiments"
},
{
"source": "/home/mainak/ms/getting_started_v5.3.0/notebooks/tao_launcher_starter_kit/lprnet/specs",
"destination": "/workspace/tao-experiments/lprnet/specs"
}
],
"DockerOptions": {
"user": "1000:1000"
}
}
I run using:
tao model lprnet train --gpus=1 -e /home/mainak/ms/getting_started_v5.3.0/notebooks/tao_launcher_starter_kit/lprnet/specs/tutorial_spec.txt -k nvidia_tlt -r /home/mainak/ms/getting_started_v5.3.0/notebooks/tao_launcher_starter_kit/lprnet/experiment_dir_unpruned -m /home/mainak/ms/getting_started_v5.3.0/notebooks/tao_launcher_starter_kit/lprnet/lprnet_vtrainable_v1.0/us_lprnet_baseline18_trainable.tlt
Any help is highly appreciated
@Morganh
2 posts - 2 participants