<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pubudu  Rathnayake on Medium]]></title>
        <description><![CDATA[Stories by Pubudu  Rathnayake on Medium]]></description>
        <link>https://medium.com/@p.m.sepalarathnayake?source=rss-25ef4162f46e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*qbvz68MPPoDX77zWGQtFRQ.jpeg</url>
            <title>Stories by Pubudu  Rathnayake on Medium</title>
            <link>https://medium.com/@p.m.sepalarathnayake?source=rss-25ef4162f46e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 23:46:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@p.m.sepalarathnayake/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Optimizing TensorFlow Classification Models with TensorRT for Improved Inference Performance — II]]></title>
            <link>https://medium.com/accelr-blog/optimizing-tensorflow-classification-models-with-tensorrt-for-improved-inference-performance-ii-a0da967690dd?source=rss-25ef4162f46e------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0da967690dd</guid>
            <dc:creator><![CDATA[Pubudu  Rathnayake]]></dc:creator>
            <pubDate>Mon, 19 Jun 2023 07:42:49 GMT</pubDate>
            <atom:updated>2023-12-01T07:28:08.815Z</atom:updated>
            <content:encoded><![CDATA[<h3>Optimizing TensorFlow Classification Models with TensorRT for Improved Inference Performance — II</h3><blockquote>This is a series of two articles. You can refer the first article via the link provided below.</blockquote><blockquote><a href="https://medium.com/accelr-blog/optimizing-tensorflow-classification-models-with-tensorrt-for-improved-inference-performance-i-6cf881294ab"><strong>Optimizing TensorFlow Classification Models with TensorRT for Improved Inference Performance — I</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_NfLYC47modBHKFq" /><figcaption>Photo by <a href="https://unsplash.com/@lorenzoherrera?utm_source=medium&amp;utm_medium=referral">Lorenzo Herrera</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Welcome to the second part of our series on accelerating deep learning inference using TensorFlow and TensorRT. In the first article, we discussed the project overview, showcased the performance improvements achieved by converting a TensorFlow classification model to a TensorRT model, and presented benchmark results.</p><p>In this article, we will delve into the technical details of the conversion process for both the TF-TRT and the full conversion to TensorRT methods. By understanding the steps involved in these conversion approaches, you will gain insights into the optimizations and advantages of each method.</p><p>You can find more details in our github repository.</p><p><strong>Github Repo : </strong><a href="https://github.com/accelr-net/TF2TensorRTConversion"><strong>https://github.com/accelr-net/TF2TensorRTConversion</strong></a></p><h3><strong>TF-TRT Conversion</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QjL-ONQZ0aYqj1rYCsYyLQ.jpeg" /><figcaption>Source : <a href="https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html">https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html</a></figcaption></figure><p>The TF-TRT approach allows us to optimize the TensorFlow model without fully converting it to a TensorRT engine. This approach offers the benefits of both TensorFlow and TensorRT, leveraging the optimization capabilities of TensorRT while maintaining the flexibility and compatibility of TensorFlow.Here is a step-by-step breakdown of the TF-TRT conversion process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*jjGH7Vtb9djR-MFHJYIktw.jpeg" /></figure><ol><li>TensorFlow Model Optimization: The first step involves optimizing the TensorFlow model for inference. This can include techniques such as graph freezing, removing unnecessary operations, and quantization to reduce precision.</li><li>Conversion to TF-TRT Model: Next, we use the TF-TRT library to convert the optimized TensorFlow model to a TF-TRT model. TF-TRT combines the best of TensorFlow and TensorRT by dynamically integrating TensorRT optimizations into TensorFlow graph execution. During the TF-TRT conversion process, TensorRT performs various optimizations on the TensorFlow graph. These optimizations include layer fusion, precision calibration, and dynamic tensor memory management to enhance inference performance.</li><li>3. Inference on GPU: The final step involves executing the TF-TRT model on a GPU using the TensorRT runtime. This allows us to use the optimized model and achieve accelerated inference performance.</li></ol><p>Let’s have a glimpse of the technical implementation.</p><h4>Technical Implementation</h4><ol><li>Import the necessary libraries:</li></ol><ul><li>First, import the required libraries such as TensorFlow and TF-TRT.</li></ul><pre>import tensorflow as tf<br>print(&quot;TensorFlow version: &quot;, tf.__version__)<br><br>from tensorflow.python.compiler.tensorrt import trt_convert as trt<br>from tensorflow.python.saved_model import tag_constants</pre><p>2. Load the TensorFlow model:</p><ul><li>Load your pre-trained TensorFlow model using the appropriate TensorFlow APIs or frameworks.</li></ul><pre>SAVED_MODEL_DIR =  &quot;/saved_model/resnet_v1_fp32_savedmodel&quot;</pre><p>3. Create a TF-TRT optimization profile:</p><ul><li>Define an optimization profile that specifies the target precision (FP32, FP16, or INT8) and other configuration parameters for the conversion process.</li></ul><pre>conversion_params = trt.DEFAULT_TRT_CONVERSION_PARAMS._replace(<br>    precision_mode=trt.TrtPrecisionMode.FP32)<br><br>converter = trt.TrtGraphConverterV2(<br>    input_saved_model_dir=SAVED_MODEL_DIR,<br>conversion_params=conversion_params)</pre><p>4. Build the TF-TRT engine:</p><ul><li>Create a TF-TRT engine by applying the optimization profile to the TensorFlow model. This process involves analyzing the model and performing layer fusion, precision calibration, and other optimizations based on the specified profile.</li></ul><pre>converter.convert()</pre><p>5. Serialize and save the TF-TRT engine:</p><ul><li>Serialize the TF-TRT engine into a file format that can be loaded and executed efficiently during inference.</li></ul><pre>converter.save(FP32_SAVED_MODEL_DIR)</pre><h3>TensorRT Conversion</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DpESDg0K08bDNySx4tLg6w.png" /><figcaption>TensorRT’s vertical and horizontal layer fusion and layer elimination optimizations simplify the GoogLeNet Inception module graph, reducing computation and memory overhead. Source : <a href="https://developer.nvidia.com/blog/tensorrt-3-faster-tensorflow-inference/">https://developer-blogs.nvidia.com/wp-content/uploads/2017/12/</a></figcaption></figure><p>The full conversion to TensorRT provides another method to convert the TensorFlow model to a TensorRT engine. Here is a step-by-step overview of the full conversion process:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*oC2NZrUA-ZkVx0hOsKuIGQ.jpeg" /></figure><p>1. TensorFlow to ONNX Conversion: The first step is to convert the TensorFlow model to the ONNX (Open Neural Network Exchange) format. ONNX is an open standard for representing deep learning models, allowing interoperability across different frameworks.</p><p>2. ONNX to TensorRT Conversion: Once the model is in the ONNX format, we utilize the TensorRT’s ONNX parser to convert it into a TensorRT engine. The ONNX parser analyzes the ONNX model and constructs an optimized TensorRT engine for efficient inference.</p><p>During the ONNX to TensorRT conversion, TensorRT performs a series of optimizations specific to the TensorRT engine. These optimizations include layer fusion, precision calibration, kernel auto-tuning, and dynamic memory management to maximize inference performance.</p><p>3. Inference on GPU: Similar to the TF-TRT method, the final step involves executing the converted TensorRT model on a GPU using the TensorRT runtime to leverage the accelerated inference capabilities</p><p>Let’s dive into the technical implementation.</p><h4>Technical Implementation</h4><p><strong>Tensorflow to ONNX</strong></p><blockquote>“tf2onnx” is a library that can be used to convert the Tensorflow model to the ONNX model via Python API or command line.</blockquote><ol><li>Install the tf2onnx library</li></ol><ul><li>Ensure that you have tf2onnx installed in your Python environment. You can install it using pip:</li></ul><pre>pip install tf2onnx</pre><p>2. Convert the TensorFlow model</p><ul><li>Use the tf2onnx conversion command to convert the TensorFlow model to ONNX format. Here’s an example command</li></ul><pre>python -m tf2onnx.convert --saved-model tensorflow-model-path --opset 13 --output model.onnx</pre><ul><li>--saved-model tensorflow-model-path that specifies the path to the TensorFlow model directory or SavedModel.</li><li>--opset 13 indicates the ONNX operator set version to use. You can specify the appropriate version based on your requirements.</li><li>--output model.onnx specifies the output file path for the converted ONNX model.</li></ul><p>3. Execute the conversion command</p><ul><li>Run the above command in your terminal or command prompt to initiate the TensorFlow to ONNX conversion process. Make sure to provide the correct paths and options according to your setup.</li></ul><p>4. Conversion completion</p><ul><li>Once the conversion process is complete, you will have an ONNX model file (model.onnx) representing your TensorFlow model.</li></ul><p><strong>ONNX to TensorRT</strong></p><blockquote>The trtexec command is a powerful tool provided by TensorRT that allows you to optimize and run inference on TensorRT engines.</blockquote><p>Here is an explanation of how to use the trtexec command to convert an ONNX model to a TensorRT engine.</p><ol><li>Install TensorRT</li></ol><ul><li>Ensure that TensorRT is installed on your system. You can download and install the appropriate version of TensorRT from the NVIDIA developer website or if you are using the docker container from the nvidia mentioned in the first article, the tensorrt is already installed and setup in it.</li></ul><p>2. Prepare the ONNX model</p><ul><li>Have the ONNX model file (.onnx) ready for conversion. If you need to convert a TensorFlow model to ONNX, refer to the previous explanation on TensorFlow to ONNX conversion.</li></ul><p>3. Run the trtexec command</p><ul><li>Open a terminal or command prompt and use the trtexec command to perform the conversion and inference. Here&#39;s an example command:</li></ul><pre>trtexec --onnx=model.onnx --saveEngine=engine.trt --fp16 --explicitBatch --batch=8</pre><ul><li>--onnx=model.onnx specifies the path to the ONNX model file that you want to convert to a TensorRT engine.</li><li>--saveEngine=engine.trt defines the output file path where the converted TensorRT engine should be saved.</li><li>--fp16 enables FP16 precision mode, which utilizes 16-bit floating-point arithmetic for faster inference with reduced memory footprint. You can omit this option if you want to use FP32 precision.</li><li>--explicitBatch enables explicit batch mode, allowing you to specify the batch size during inference.</li><li>--batch=8 sets the batch size to 8. Adjust this value according to your specific batch size requirement.</li></ul><p>4. Execute the conversion command</p><ul><li>Run the trtexec command in the terminal or command prompt, providing the correct paths and options based on your setup.</li></ul><p>5. Conversion completion</p><ul><li>Once the conversion process is complete, the TensorRT engine file (engine.trt) will be generated.</li></ul><p>After the conversion, you can use the generated TensorRT engine for accelerated inference on NVIDIA GPUs.</p><p>The trtexec command provides additional options and configurations that you can explore for advanced optimization and profiling of the TensorRT engine. Refer to the TensorRT documentation for more information on available options and their usage.</p><p>For further instruction on preparing dataset and model performance evaluation, Please refer the readme.md on our github repository.</p><p><strong>Github Repo : </strong><a href="https://github.com/accelr-net/TF2TensorRTConversion"><strong>https://github.com/accelr-net/TF2TensorRTConversion</strong></a></p><p>In our project, we initially converted our TensorFlow native model to a TF-TRT model with FP32 (32-bit floating-point) precision. This conversion already provided significant performance improvements compared to the original TensorFlow model. However, TensorRT offers additional optimizations by allowing us to further optimize the TF-TRT models and fully converted TensorRT engines for different precisions, including FP16 (16-bit floating-point), INT8 (8-bit integer), and dynamic tensor quantization.</p><p>By utilizing the TensorRT optimization features, we can achieve even faster inference speeds and improved performance on compatible hardware architectures. Let’s explore the available optimizations:</p><ol><li>FP16 Precision: TensorRT allows us to optimize the model for FP16 precision by utilizing the lower 16-bit floating-point format. This optimization reduces memory consumption and computational requirements while maintaining reasonable accuracy. By leveraging hardware architectures that support FP16 operations, we can achieve faster inference speeds compared to FP32 precision.</li><li>INT8 Precision: To achieve even further optimization, TensorRT offers INT8 precision. However, optimizing for INT8 precision requires an additional calibration step using calibration data. Calibration data consists of representative samples from the dataset on which the model will be used. TensorRT runs the model on the calibration data to determine the optimal quantization parameters for each layer. These parameters are then used to quantize the model to INT8 precision. The INT8-optimized model provides faster inference speeds and reduced memory usage but with a potential trade-off in accuracy.</li><li>Dynamic Tensor Quantization: TensorRT also supports dynamic tensor quantization, which is a form of quantization that optimizes the model for lower precision. It allows the model to dynamically adjust the precision of tensors during inference based on their values. This technique further reduces memory bandwidth requirements and computational overhead, resulting in improved inference performance.</li></ol><p>When exploring these optimization options, it’s crucial to consider the specific requirements of your application. Different optimizations may yield varying trade-offs in terms of accuracy, hardware compatibility, and memory consumption.</p><h4>Conclusion</h4><p>This article explored the technical details of the TF-TRT and full TensorRT conversion methods for accelerating deep learning inference. We discussed the step-by-step process for each approach, highlighting the optimizations and advantages offered by TensorFlow and TensorRT. By understanding these conversion techniques, you can effectively optimize your TensorFlow models for high-performance inference.</p><h4>References</h4><p>[1]: Onnx Documentaion. tf2onnx — Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX. <br><a href="https://github.com/onnx">https://github.com/onnx</a></p><p>[2]: TensorRT Documentaion. <br><a href="https://github.com/NVIDIA/TensorRT">https://github.com/NVIDIA/TensorRT</a></p><p>[3]: Shashank Prasanna<strong>, </strong>Prethvi Kashinkunti, Fausto Milletari<strong> </strong>. TensorRT 3: Faster TensorFlow Inference and Volta Support.<em> </em><a href="https://developer.nvidia.com/blog/tensorrt-3-faster-tensorflow-inference/">https://developer.nvidia.com/blog/tensorrt-3-faster-tensorflow-inference/</a></p><p>[4]: Nvidia Developer. NVIDIA Deep Learning TensorRT Documentation<em> </em><a href="https://docs.nvidia.com/deeplearning/tensorrt/index.html">https://docs.nvidia.com/deeplearning/tensorrt/index.html</a></p><p>[5]: Tensorflow Documentaion. Documentation for TensorRT in TensorFlow (TF-TRT)<br><em> </em><a href="https://github.com/tensorflow/tensorrt">https://github.com/tensorflow/tensorrt</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0da967690dd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/accelr-blog/optimizing-tensorflow-classification-models-with-tensorrt-for-improved-inference-performance-ii-a0da967690dd">Optimizing TensorFlow Classification Models with TensorRT for Improved Inference Performance — II</a> was originally published in <a href="https://medium.com/accelr-blog">ACCELR Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimizing TensorFlow Classification Models with TensorRT for Improved Inference Performance — I]]></title>
            <link>https://medium.com/accelr-blog/optimizing-tensorflow-classification-models-with-tensorrt-for-improved-inference-performance-i-6cf881294ab?source=rss-25ef4162f46e------2</link>
            <guid isPermaLink="false">https://medium.com/p/6cf881294ab</guid>
            <dc:creator><![CDATA[Pubudu  Rathnayake]]></dc:creator>
            <pubDate>Tue, 23 May 2023 03:49:01 GMT</pubDate>
            <atom:updated>2023-12-01T07:42:56.848Z</atom:updated>
            <content:encoded><![CDATA[<h3>Optimizing TensorFlow Classification Models with TensorRT for Improved Inference Performance — I</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DAADnJK3evKGPFWz" /><figcaption>Photo by <a href="https://unsplash.com/it/@lazycreekimages?utm_source=medium&amp;utm_medium=referral">Michael Dziedzic</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Introduction</h4><p>Deep learning models have revolutionized the field of machine learning, enabling breakthroughs in various applications, including image classification, object detection, and natural language processing. However, as models become larger and more complex, their inference time and memory usage can become a bottleneck, especially in real-time edge-computing applications.</p><p>TensorRT, developed by NVIDIA, is a high-performance deep learning inference optimizer and runtime library that is specifically designed for NVIDIA GPUs. It is capable of optimizing and accelerating deep learning models, resulting in significant improvement in inference speed and reduced memory footprint. In this article series, we will explore how we converted a TensorFlow classification model to a TensorRT model to achieve substantial performance gains in our project.</p><p>This first article will provide an overview of the project and showcase the performance improvements that we achieved. In the next article … So, let’s dive in and learn how TensorRT can supercharge our deep-learning models.</p><h4>Project Overview</h4><p>There are two methods available to us when optimizing an existing TensorFlow model using TensorRT.</p><ol><li>Optimizing the TensorFlow model with the TF-TRT library</li><li>Fully converting the model to a TensorRT engine</li></ol><h4>Methods of Optimization</h4><ol><li>Optimizing the TensorFlow model with TF-TRT library:</li></ol><p>TensorRT for TensorFlow (TF-TRT) is a TensorFlow integration library that allows users to optimize their TensorFlow models using TensorRT without fully converting the model to a TensorRT engine. This method provides seamless integration with TensorFlow and enables users to take advantage of TensorRT’s optimizations during inference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RL41TZElADmaxsBKwhiEOA.png" /><figcaption>Image by the authors adapted from <a href="https://blog.tensorflow.org/2021/01/leveraging-tensorflow-tensorrt-integration.html">https://blog.tensorflow.org/</a></figcaption></figure><p>2. Fully converting the model to a TensorRT engine:</p><p>In this method, we will fully convert the TensorFlow model to a TensorRT engine via the ONNX format. This involves first converting the TensorFlow model to the ONNX format, and then using the TensorRT ONNX parser to create a TensorRT engine. This method involves complete optimization of the model using TensorRT’s optimizations, including layer fusion, kernel auto-tuning, and dynamic tensor memory management.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qOVRuIBz1rwxqRxTOxfdQg.png" /><figcaption>Image by the authors adapted from <a href="https://developer.nvidia.com/blog/speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt/">https://developer.nvidia.com/blog/</a></figcaption></figure><h4>The Rationale for Experimenting with Both Methods</h4><p>We decided to experiment with both methods of conversion to evaluate their performance and determine the optimal approach for our specific use case. TF-TRT provides selective optimization, allowing us to choose which parts of the model to optimize with TensorRT, which may be beneficial in certain scenarios. On the other hand, full conversion via ONNX allows for complete model optimization using TensorRT’s optimizations, which may result in higher performance in certain cases. By comparing the performance of both methods, we aim to determine the most effective approach for our specific project requirements.</p><h4>Experiment and Results</h4><p>The <a href="#d334">ResNet50V1 Classification model</a> serves as the baseline for our optimization experiments. In order to evaluate the models we will be using the <a href="https://www.tensorflow.org/datasets/catalog/imagenet2012">ImageNet(ILSVRC-2012-CLS)</a> validation dataset.</p><p>After converting the TensorFlow model to a TensorRT model using both methods, we conducted comprehensive benchmarking tests that are summarized in the chart below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*iqSeMCgWKS8DdRttchGQOg.png" /><figcaption>Performance Result — Figure 1.0</figcaption></figure><p>As can be seen from the benchmark results, both the TF-TRT and the full conversion via ONNX methods achieved higher inference performance compared to the original TensorFlow model. However, the TensorRT model achieved the highest performance across all batch sizes, with the TF-TRT model showing slightly lower performance compared to the full conversion via ONNX.</p><h4>Conclusion of the Test</h4><p>In this article, we have presented two methods of converting a TensorFlow classification model to a TensorRT engine: using TF-TRT and full conversion to TensorRT. Through comprehensive benchmarking tests, we observed that both methods resulted in improved performance compared to the original TensorFlow model. However, full conversion to the TensorRT method showed slightly better performance compared to the TF-TRT method.</p><p>In our next articles, we will further investigate the technical details of these conversion methods, providing insights into their optimizations and advantages.</p><blockquote>You can refer the second article via the link provided below.</blockquote><blockquote><a href="https://medium.com/accelr-blog/optimizing-tensorflow-classification-models-with-tensorrt-for-improved-inference-performance-ii-a0da967690dd">Optimizing TensorFlow Classification Models with TensorRT for Improved Inference Performance — II</a></blockquote><h4>Model and Data Specifications</h4><p>The based TensorFlow model used in the project is a pre-trained <a href="https://tfhub.dev/google/imagenet/resnet_v1_50/classification/5"><em>ResNet50V1 classification model</em></a>, which is obtained from TensorFlow Hub. The weights for this model were obtained by training on the <a href="https://www.tensorflow.org/datasets/catalog/imagenet2012">ImageNet(ILSVRC-2012-CLS)</a> for image classification (“Imagenet”) with TF-Slim’s “Inception-style” preprocessing.</p><h4>System Specifications</h4><p><em>Here are the specifications of the AWS Virtual Machine on which we conducted our experiments.</em></p><ul><li>VM Instance Type: g4dn.xlarge</li><li>vCPU: 4</li><li>RAM: 16 GB</li><li>TensorFlow version: 2.4</li><li>TensorRT version: 7.2.3.4</li><li>CUDA version: 11.03</li><li>cuDNN version: 8.0</li><li>GPU: NVIDIA Tesla T4 -16 GB</li><li>GPU Compute Capability: 7.5</li><li>Host OS: Ubuntu 20.04.5 LTS</li><li>Container Tag: <a href="http://nvcr.io/nvidia/21.04-tf2-py3">nvcr.io/nvidia/21.04-tf2-py3</a> (Release Notes)</li></ul><h4>Test Results</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*p_uqPPj2o_sKcVhBj_RMgg.png" /><figcaption>Detailed Results of Figure 1.0</figcaption></figure><p>Note: Accuracy is shown as a percentage, and performance is shown in images per second (images/sec).</p><h4>References</h4><blockquote>1. <a href="https://www.tensorflow.org/">https://www.tensorflow.org/</a></blockquote><p>2.<a href="https://blog.tensorflow.org/2021/01/leveraging-tensorflow-tensorrt-integration.html">https://blog.tensorflow.org/2021/01/leveraging-tensorflow-tensorrt-integration.html</a></p><p>3.<a href="https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html">https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html</a></p><p>4.<a href="https://medium.com/@abhaychaturvedi_72055/understanding-nvidias-tensorrt-for-deep-learning-model-optimization-dad3eb6b26d9">https://medium.com/@abhaychaturvedi_72055/understanding-nvidias-tensorrt-for-deep-learning-model-optimization-dad3eb6b26d9</a></p><p>5.<a href="https://onnx.ai/get-started.html">https://onnx.ai/</a></p><p>6. <a href="https://developer.nvidia.com/blog/speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt/">https://developer.nvidia.com/blog/speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6cf881294ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/accelr-blog/optimizing-tensorflow-classification-models-with-tensorrt-for-improved-inference-performance-i-6cf881294ab">Optimizing TensorFlow Classification Models with TensorRT for Improved Inference Performance — I</a> was originally published in <a href="https://medium.com/accelr-blog">ACCELR Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>