<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Youssef Ahmed on Medium]]></title>
        <description><![CDATA[Stories by Youssef Ahmed on Medium]]></description>
        <link>https://medium.com/@youssefahmed_41143?source=rss-4fb5c9f5f957------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*51n1sGbHPfG65DEfzwuIOA.jpeg</url>
            <title>Stories by Youssef Ahmed on Medium</title>
            <link>https://medium.com/@youssefahmed_41143?source=rss-4fb5c9f5f957------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 01:29:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@youssefahmed_41143/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deploy a local trained machine learning model on AWS using SAM]]></title>
            <link>https://medium.com/@youssefahmed_41143/deploy-a-local-trained-machine-learning-model-on-aws-using-sam-cbb065067692?source=rss-4fb5c9f5f957------2</link>
            <guid isPermaLink="false">https://medium.com/p/cbb065067692</guid>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <dc:creator><![CDATA[Youssef Ahmed]]></dc:creator>
            <pubDate>Thu, 09 Jun 2022 18:37:17 GMT</pubDate>
            <atom:updated>2022-06-26T14:24:00.127Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MPmQESGXxT_AJ-KOnS-46g.png" /><figcaption>Center_of_the_Milky_Way_Galaxy</figcaption></figure><h4>Introduction</h4><blockquote>Deploying a machine learning model could be a headache. It can take as much time as the building, development, training, and evaluation or even much more in both time and effort. I did not find many blogs about this topic, especially with SAM, but no worries! In this blog I will summarize my deployment experience after almost a week of fixing dependencies, configurations, and policy errors. I will also make some suggestions on the best practices used to deploy your machine learning model and dockerize it, so let’s get started.</blockquote><h4>Why SAM?</h4><p>AWS Serverless Application Model (SAM) makes the deployment of a machine learning model much easier than ever before. It takes care of the full stack from creating (s3 buckets, ecr, lamda functions,..etc) and much more with just simple commands (which can be automated). Making an end-to-end public API ready to be used by your application. Figure 1 below shows the benefits of serverless architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/742/1*Znh_Tyo0pmRBHP3ffUcGDg.png" /><figcaption>Figure 1</figcaption></figure><p>You can find much more about SAM in it’s documentation <a href="https://aws.amazon.com/serverless/sam/">here</a></p><h3>Prerequisites</h3><p>You should have the following installed on your PC to be able to deploy with SAM:</p><ul><li>Python (recommended version &gt; 3.7)</li><li>Docker (recommended version 20.10.12)</li><li>AWS CLI (recommended version 2.7.4) you can install it from <a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html">here</a></li><li>SAM (recommended version 1.33.0) you can install it from <a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html">here</a></li></ul><h3>Steps</h3><p>We will start with a SAM template and edit it with our model and our logic for inference function.. after installing SAM</p><h3><strong>-Init a SAM project</strong></h3><p>(attached below in Figure 2 all of these settings)</p><p>1- Type <strong>sam init </strong>in a terminal</p><p>2- Choose <strong>1</strong> (AWS Quick Start Templates)</p><p>3- Choose <strong>2 </strong>(image option is much better as we have a limit of 10 giga to our docker image size, and only 250 mega for the Zip option)</p><p>4- Choose <strong>5</strong> (amazon/python3.8-base) or the most suitable python version for you ML model</p><p>5- Type your app name ex: <strong>model-x-deploy</strong></p><p>6- Choose<strong> 2 </strong>(PyTorch Machine Learning example)- don’t worry we will edit our requirements file in the next step -</p><p>You will find a generated folder containing the template files and a README.md containing the details of the next steps to deploy the app. <strong>(Note: the names, numbers, and orders of these settings may differ depending on the SAM version, but the end result should be the same)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bJUkIyV-IxCXy-BZpXwTBw.png" /><figcaption>Figure 2</figcaption></figure><h3>-Build the SAM project</h3><p>To build our local image, we first need to edit this template with our dependencies and logic for the inference function.</p><p>1- Open the docker file, you will find it inside the <em>app</em> folder it should be something like below</p><pre>FROM public.ecr.aws/lambda/python:3.8<br><br>COPY app.py requirements.txt ./<br>COPY model /opt/ml/model<br><br>RUN python3.8 -m pip install -r requirements.txt -t .<br><br>CMD [&quot;app.lambda_handler&quot;]</pre><p>Here we pull the required python version image from the ecr.aws lamda registry in order to invoke the lamda handler inside this image. We then copy our app.py, which has the logic for the inference in the lamda_handler function in app.py. We also copy our model (you should place the model physically in the app folder with a folder named model). We copy what is inside this model folder checkpoints into our relative path “opt/ml/model”, which we will use this relative path as a hard coded path inside our code later. We then install all our required dependencies in the requirements.txt file.</p><p>An important word about best practices I found here you should instead of grabbing the requirements from a text file directly, put them directly line by line in the dockerfile. In this way, you will use the cache as much as you can, since every time you add a new dependency in the.txt file, all the other packages’ caches are lost and you have to download them from scratch, unlike if you only add the new package pip install in the docker file itself. Another hint is you should increase the — default-timeout=200 in large packages in order not to to fail a lot due to time out error. A good practice also is to have a specific version for every package, in order not to face any dependency issue in the future releases. Don’t forget to add any local package or other source code or any dependency in the image by using the COPY command. You also need to make sure that the last line is added to make the app. The handler is invoked.</p><p>For the app.py you should put your logic inside the <em>lamda_handler</em> function, and make sure to load the model as a global variable like this</p><p>model_file = ‘/opt/ml/model’<br>model = torch.jit.load(model_file) # pytorch example</p><p>And for the data, you should have this function which has the event variable, which is passed to the lamda_handler (event, context). as shown below</p><pre>def parse_event(event):<br>    if &#39;body&#39; in event.keys():<br>        return json.loads(event[&#39;body&#39;])[&#39;data&#39;]<br>    else:<br>        return event[&#39;data&#39;]</pre><p>Have a look at the template.yaml file you can edit your preferred paths and other parameters, like the edit I made below. But normally you don’t need to edit that much.</p><pre><em><br>  </em>Properties:<br>    Path: /model_x_inference<br>    Method: post</pre><p>2- To build your image use this command <strong>sudo sam build — debug</strong></p><p>3- To test your image locally run <strong>sudo sam local start-api — debug </strong>.Test it by running the following curl command in another terminal</p><pre>curl --request POST \<br>  --url http://127.0.0.1:3000/model_x_inference/ \<br>  --header &#39;content-type: application/json&#39; \<br>  --data &#39;{<br>  &quot;data&quot;: {<br>    &quot;data_key&quot;: &quot;data_value&quot;<br>  }<br>}&#39;</pre><h3>-Deploy the SAM project</h3><p>To Deploy your local image to AWS use this command</p><p><strong>sudo sam deploy — debug — guided</strong></p><p>and then follow the terminal instructions.</p><p><strong>Possible Errors you will face and solutions for it will be listed in this section</strong></p><p>1) “no basic auth credentials” error</p><p>This is the hardest error I have struggled with and spent my time and effort trying to solve: The error means AWS CLI can not find your access keys environment variables in order to use your credentials to deploy your local image.</p><p><strong>basic solution</strong> use <strong>aws configure</strong> command you will see something like this:</p><p>AWS Access Key ID [****************xxxx]:</p><p>AWS Secret Access Key [****************xxxx]:</p><p>You should enter your Keys here and you can generate them using this <a href="https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html">link</a></p><p>The first solution link is <a href="https://stackoverflow.com/questions/34689445/cant-push-image-to-amazon-ecr-fails-with-no-basic-auth-credentials">here</a></p><p>What if you tried these solutions but you still got the same error? (Like me)… No worries, I have another solution for you :)</p><p>As you may know, when running any command that uses Docker or trying to run any local images, it will need sudo access. If you try to build with sam and do not include sudo in the command, it may fail. So when using sudo, you are now accessing the environmental variables of the root user, which, as you probably guess, won’t have the credentials you have just added.</p><p>So the second solution is to run <strong>sudo aws configure </strong>and you will see that the credentials and regions are not set, so here you go that should fix the first error.</p><p>2) “Authorization Token has expired”</p><p>This occurs when you try to push your image using docker repository instead of sam and send the image url to the sam when deploying, because docker credentials expire after only 12 hours for security reasons. The solution is available <a href="https://stackoverflow.com/questions/41379808/authorization-token-has-expired-issue-aws-cli-on-macos-sierra">here</a>.</p><p>3) “A lot of permission errors during to AWS account”</p><p>I was really struggling with this too, thinking the problem was in my SAM or CLI versions or any other dependency, to find out that it was just another permission to have for our ML group of policies, and that was a lot to talk about and get with the head of the software or the Devops team leads to get this permission from their super users’ root accounts. Here I list all the policies and even inline written policies that with these settings and policies I was able to successfully deploy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O7lzHszuQaJdcvy_WGQMzw.png" /><figcaption>figure2 the list of policies</figcaption></figure><p>And for the ML inline policy here is the json of it <a href="https://stackoverflow.com/questions/65137639/lambda-does-not-have-permission-to-access-the-ecr-image">ref link</a></p><pre>{<br> &quot;Version&quot;: &quot;2012–10–17&quot;,<br> &quot;Statement&quot;: [<br> {<br> &quot;Sid&quot;: &quot;VisualEditor0&quot;,<br> &quot;Effect&quot;: &quot;Allow&quot;,<br> &quot;Action&quot;: [<br> &quot;ecr:SetRepositoryPolicy&quot;,<br> &quot;ecr:GetDownloadUrlForLayer&quot;,<br> &quot;ecr:BatchGetImage&quot;,<br> &quot;ecr:CompleteLayerUpload&quot;,<br> &quot;ecr:DescribeImages&quot;,<br> &quot;ecr:DescribeRepositories&quot;,<br> &quot;ecr:UploadLayerPart&quot;,<br> &quot;ecr:ListImages&quot;,<br> &quot;ecr:InitiateLayerUpload&quot;,<br> &quot;ecr:BatchCheckLayerAvailability&quot;,<br> &quot;ecr:GetRepositoryPolicy&quot;,<br> &quot;ecr:PutImage&quot;<br> ],<br> &quot;Resource&quot;: &quot;*&quot;<br> },<br> {<br> &quot;Sid&quot;: &quot;VisualEditor1&quot;,<br> &quot;Effect&quot;: &quot;Allow&quot;,<br> &quot;Action&quot;: &quot;ecr:GetAuthorizationToken&quot;,<br> &quot;Resource&quot;: &quot;*&quot;<br> }<br> ]<br>}</pre><p>4) “cloud formation errors based”</p><p>It seems that SAM sometimes fails to automatically update or delete an old stack, so you should try to manually delete it when you find that the error is related to cloudwatch somehow, like this one “ValidationError Stack:arn aws cloudformation stack is in ROLLBACK_COMPLETE state and can not be updated.” A link to a similar error is <a href="https://stackoverflow.com/questions/57932734/validationerror-stackarn-aws-cloudformation-stack-is-in-rollback-complete-state">here</a>. You can access cloud watch from the main menu and delete any failed stack, and don’t worry if it did not delete the first time. It will ask you for validation that this deletion will delete related functions and modules related to it, then delete it successfully.</p><p>5) “internal server error”</p><p>Finally, you have now deployed your model and when testing it on the public API, it fails with this error, but what does this error really mean? You need to access AWS Cloud Watch logs to check it. Use this link to do so. If the error does not appear yet, it seems to be the name of the environment variable that is the reason. It may change in the future, but for me it is shown when changing the name from <strong>$context.integrationErrorMessage </strong>to<strong> $context.integration.error </strong><a href="https://noise.getoto.net/2020/09/04/troubleshooting-amazon-api-gateway-with-enhanced-observability-variables/">reference link</a></p><p>6) “OSError: [Errno 38] Function not implemented”</p><p>This one is tricky. It happens when you are using any multi-threading functions in Python. It seems that the aws lamda function for Python still has errors with it, so for me the solution was to set num_workers = 0 in a pytorch model. The error is solved. <a href="https://stackoverflow.com/questions/69042763/error-oserror-errno-38-function-not-implemented-accessing-trend-deepsec">reference link</a></p><p>7) “The cold start problem”</p><p>After all of this long journey, don’t panic if you have just tested it after successfully deploying it and find an error like “time out” or another kind of similar error. It happens because of the cold start well known problem. You can read more about it on other blogs or articles. So anyway, be patient and your API will eventually be deployed successfully :)</p><p>I hope you will find this article useful and fix one or more errors that you will face. Feel free to ask me in the comments section and I will try my best to help and add these new errors to this blog if it is a generalized error. Best of luck and happy deploying everyone :)</p><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cbb065067692" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>