<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Abhishek Patnia on Medium]]></title>
        <description><![CDATA[Stories by Abhishek Patnia on Medium]]></description>
        <link>https://medium.com/@abhishekpatnia?source=rss-f888c8c5bea2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*K0KeQBfwAJpfsNFB.</url>
            <title>Stories by Abhishek Patnia on Medium</title>
            <link>https://medium.com/@abhishekpatnia?source=rss-f888c8c5bea2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 02:52:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@abhishekpatnia/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Running Deep Learning Models on AWS Lambda]]></title>
            <link>https://becominghuman.ai/running-deep-learning-models-on-aws-lambda-cfd2f76ca048?source=rss-f888c8c5bea2------2</link>
            <guid isPermaLink="false">https://medium.com/p/cfd2f76ca048</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[theano]]></category>
            <category><![CDATA[keras]]></category>
            <dc:creator><![CDATA[Abhishek Patnia]]></dc:creator>
            <pubDate>Thu, 28 Sep 2017 15:54:29 GMT</pubDate>
            <atom:updated>2017-10-01T06:29:54.918Z</atom:updated>
            <content:encoded><![CDATA[<h3>Optimized Theano &amp; Keras on AWS Lambda</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/0*AOleV5PenQk46uF3.png" /><figcaption><a href="http://2ndwatch.com/blog/enter-aws-lambda/">Source</a></figcaption></figure><p>I read this <a href="https://ryan-cranfill.github.io/keras-aws-lambda/">amazing blog post</a> about running Keras + Theano models on AWS Lambda. However, these models have very high latency on Lambda because Theano optimizes models using GCC which is not available in Lambda. <strong>This post will explain how to make GCC available in Lambda and run fully optimized models. You can expect 8x boost in performance.</strong></p><p><em>I expect the readers to have basic familiarity with AWS Lambda, EC2, Keras, and Theano.</em></p><h4>Setup Keras &amp; Theano on AWS Lambda</h4><p>At the end of this section we will have a toy model running. The basic idea is to setup all dependencies on an EC2 instance running the AWS Lambda AMI. Then, from these dependencies, create a Lambda deployment. Lambda AMI will make the deployment consistent with Lambda function execution environment.</p><p>Start by launching an EC2 instance with the appropriate AMI. Get the AMI details from <a href="http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html">this page</a>. I used a C4.4xlarge instance. Log in to the instance and do the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/db512abff1b3fd713ddb0ec70e29919c/href">https://medium.com/media/db512abff1b3fd713ddb0ec70e29919c/href</a></iframe><p>Create a basic handler for testing.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79d423e47729030e80c8e00994bbf9de/href">https://medium.com/media/79d423e47729030e80c8e00994bbf9de/href</a></iframe><p>Copy this handler to the deployment folder of previous section and get the Lambda deployment payload ready.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f6ff5c5c0ac65f5af7b1121c6e978e2f/href">https://medium.com/media/f6ff5c5c0ac65f5af7b1121c6e978e2f/href</a></iframe><pre># Add the following environment variables to Lambda function setup.<br>THEANO_FLAGS=base_compiledir=/tmp/.theano <br>KERAS_BACKEND=theano</pre><p>If the Lambda function is setup correctly the handler code will work. <strong>The latency will be around 16 ms.</strong></p><h4>Setup GCC</h4><p>Next, let us make GCC available.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5103eecbb67082a004e01cecb8b308d0/href">https://medium.com/media/5103eecbb67082a004e01cecb8b308d0/href</a></iframe><p>Change the Lambda Function handler to download GCC payload and bootstrap it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8ddbce8482e2e23c2c8d4fe2cb896bd0/href">https://medium.com/media/8ddbce8482e2e23c2c8d4fe2cb896bd0/href</a></iframe><pre># Add the following environment variables to the Lambda function<br>GCC_EXEC_PREFIX=/tmp/gcc/lib/gcc/<br>THEANO_FLAGS=base_compiledir=/tmp/.theano,cxx=/tmp/gcc/bin/g++</pre><p>At this point, lambda function invocation should fail with GCC complaining about header files OR static/dynamic libraries not found.</p><p><strong>Setup Header Files &amp; Static/Dynamic Libraries</strong></p><p>To fix GCC errors from earlier section do the following on EC2 instance.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c7506d3b52988dcbf77c186c0720aaba/href">https://medium.com/media/c7506d3b52988dcbf77c186c0720aaba/href</a></iframe><p>Change the Lambda function handler to download this new payload and bootstrap it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/60bbc62f06151ccf2eb7b0d379f4fde8/href">https://medium.com/media/60bbc62f06151ccf2eb7b0d379f4fde8/href</a></iframe><pre>#Add the following environment variables to Lambda setup<br>LIBRARY_PATH=$LIBRARY_PATH:/tmp/lib<br>CPATH=$CPATH:/tmp/lib:/tmp/lib/python2.7</pre><p>At this point, everything should work. First lambda invocation will be slow as it downloads GCC and Lib artifacts. <strong>After that the latency will be around 2 ms !!</strong></p><h4>Conclusion</h4><p>I extended the original work and added GCC support. This allows for ~8x boost in model performance. I hope this post is helpful :).</p><p><strong>Disclaimer</strong></p><p>I work for Amazon.com. This post is part of my personal exploration and learning. It does not represent Amazon in any way.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f5f8b%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F8f5f8b&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href">https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href</a></iframe><figure><a href="https://becominghuman.ai/artificial-intelligence-communities-c305f28e674c"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*2f7OqE2AJK1KSrhkmD9ZMw.png" /></a></figure><figure><a href="https://upscri.be/8f5f8b"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*v-PpfkSWHbvlWWamSVHHWg.png" /></a></figure><figure><a href="https://becominghuman.ai/write-for-us-48270209de63"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*Wt2auqISiEAOZxJ-I7brDQ.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cfd2f76ca048" width="1" height="1" alt=""><hr><p><a href="https://becominghuman.ai/running-deep-learning-models-on-aws-lambda-cfd2f76ca048">Running Deep Learning Models on AWS Lambda</a> was originally published in <a href="https://becominghuman.ai/">Becoming Human: Artificial Intelligence Magazine</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>