Introduction To Importing Caffe, TensorFlow And ONNX Models Into TensorRT Using Python

Table Of Contents

• Description
• How does this sample work?
  • caffe_resnet50
  • onnx_resnet50
  • uff_resnet50
• Prerequisites
• Running the sample
  • Sample --help options
• Additional resources
• License
• Changelog
• Known issues

Description

This sample, introductory_parser_samples, is a Python sample which uses TensorRT and its included suite of parsers (the UFF, Caffe and ONNX parsers), to perform inference with ResNet-50 models trained with various different frameworks.

How does this sample work?

This sample is a collection of three smaller samples, with each focusing on a specific parser. The following sections describe how each sample works.

Caffe Resnet50

This sample demonstrates how to build an engine from a trained Caffe model using the Caffe parser and then run inference. The Caffe parser is used for Caffe2 models. After training, you can invoke the Caffe parser directly on the model file (usually .caffemodel) and deploy file (usually .prototxt).

ONNX Resnet50

This sample demonstrates how to build an engine from an ONNX model file using the open-source ONNX parser and then run inference. The ONNX parser can be used with any framework that supports the ONNX format (typically .onnx files).

UFF Resnet50

This sample demonstrates how to build an engine from a UFF model file (converted from a TensorFlow protobuf) and then run inference. The UFF parser is used for TensorFlow models. After freezing a TensorFlow graph and writing it to a protobuf file, you can convert it to UFF with the convert-to-uff utility included with TensorRT. This sample ships with a pre-generated UFF file.

Prerequisites

1. Install the dependencies for Python.

   python3 -m pip install -r requirements.txt

2. Download and untar the sample data from the TensorRT release tarball to the default location /usr/src/tensorrt/data

   • NOTE: This step can be skipped when using NVIDIA TensorRT container to run the sample as it has the data premounted.

Running the sample

1. Run the sample to create a TensorRT inference engine and run inference:

   python3 <parser>_resnet50.py

   • Where <parser> is either caffe, onnx, or uff.

   • NOTE: If the TensorRT sample data is not installed in the default location, /usr/src/tensorrt/data/, the data directory must be specified. For example: python caffe_resnet50.py -d /path/to/my/data/

2. Verify that the sample ran successfully. If the sample runs successfully you should see output similar to the following: Correctly recognized data/samples/resnet50/reflex_camera.jpeg as reflex camera

Sample --help options

To see the full list of available options and their descriptions, use the -h or --help command line option. For example:

usage: caffe_resnet50.py|uff_resnet50.py|onnx_resnet50.py [-h] [-d DATADIR]

Runs a ResNet50 network with a TensorRT inference engine.

optional arguments:
 -h, --help            show this help message and exit
 -d DATADIR, --datadir DATADIR
                       Location of the TensorRT sample data directory.
                       (default: /usr/src/tensorrt/data)

Additional resources

The following resources provide a deeper understanding about importing a model into TensorRT using Python:

ResNet-50

• Deep Residual Learning for Image Recognition

Parsers

• Caffe Parser
• ONNX Parser
• UFF Parser

Documentation

• Introduction To NVIDIA’s TensorRT Samples
• Working With TensorRT Using The Python API
• Importing A Model Using A Parser In Python
• NVIDIA’s TensorRT Documentation Library

License

For terms and conditions for use, reproduction, and distribution, see the TensorRT Software License Agreement documentation.

Changelog

February 2019 This README.md file was recreated, updated and reviewed.

Known issues

There are no known issues in this sample