<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lim Keng Hin on Medium]]></title>
        <description><![CDATA[Stories by Lim Keng Hin on Medium]]></description>
        <link>https://medium.com/@limkenghin?source=rss-d46d755fc75------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*YF98EaM6TtV1siKJUPjZcQ.jpeg</url>
            <title>Stories by Lim Keng Hin on Medium</title>
            <link>https://medium.com/@limkenghin?source=rss-d46d755fc75------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 19:40:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@limkenghin/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[배고파 — A Korean Food Classifier App]]></title>
            <link>https://medium.com/@limkenghin/%EB%B0%B0%EA%B3%A0%ED%8C%8C-a-korean-food-classifier-app-b372d92fe563?source=rss-d46d755fc75------2</link>
            <guid isPermaLink="false">https://medium.com/p/b372d92fe563</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[heroku]]></category>
            <category><![CDATA[transfer-learning]]></category>
            <dc:creator><![CDATA[Lim Keng Hin]]></dc:creator>
            <pubDate>Wed, 11 Dec 2019 10:35:03 GMT</pubDate>
            <atom:updated>2019-12-14T06:59:14.697Z</atom:updated>
            <content:encoded><![CDATA[<h3>배고파 — A Korean Food Classifier App</h3><h4>A full stack guide to deploy a food classifier application</h4><h4>Source Codes:</h4><p><strong>Google Colab-Model Development:</strong></p><p><a href="https://colab.research.google.com/drive/1g7e8uryxfulCwIBgzO87CRVH_E8SLSXQ">Google Colaboratory</a></p><p><strong>Github —Model Deployment</strong></p><p><a href="https://github.com/2manslkh/korean-food-classification-flask">2manslkh/korean-food-classification-flask</a></p><p><strong>Github — Mobile Application</strong></p><p><a href="https://github.com/benjaminlim00/Im-Hungry">benjaminlim00/Im-Hungry</a></p><h3>The Team</h3><p>Lim Keng Hin — Computer science &amp; Engineering<br><a href="/service/mailto:sutd1002656@hotmail.sg">sutd1002656@</a>gmail.com</p><p><a href="https://github.com/benjaminlim00">Lim Ping An Benjamin</a> — Computer science &amp; Engineering<br><a href="/service/mailto:benjaminlim1996@hotmail.sg">benjaminlim1996@hotmail.sg</a></p><p>Noémie Galzin — Computer science &amp; Engineering<br><a href="/service/mailto:noemie.galzin@etu.utc.fr">noemie.galzin@etu.utc.fr</a></p><p>Guillermo Sáez García — Electric and Biomedical Engineering<br><a href="/service/mailto:guillermo.saez@hotmail.es">guillermo.saez@hotmail.es</a></p><p>María Montero Torres — Electric and Biomedical Engineering<br><a href="/service/mailto:maria.monterotorres@gmail.com">maria.monterotorres@gmail.com</a></p><h3>Introduction</h3><p>Nowadays, health is a subject of matter. While we are being bombarded with information about diets and pills that magically make you thinner, most of the developed countries are suffering from increasing rates of obesity and diabetes, and it doesn’t seem to stop. As students of not only the computer science division but also the biomedical one, we wanted to create a mobile application to aid people in their daily lives. Maintaining a diet is a tedious task as it requires some time and persistency to record food intake. However, with the aid of Artificial Intelligence and computer vision, it is possible to reduce the work done to sustain a balanced diet.</p><p><strong>배고파</strong> is a mobile application which uses an image classification model combined with an food calorie database which provides information of the nutrition provided in each food that users eat daily.</p><h3>Problem Statement</h3><p>Current food classification models do not extend towards classifying Korean food, hence, these existing models will not be as useful in Korea.</p><h3>Project Objective</h3><p>To create a image classification model which is also able to classify Korean Food and deploy the model on a intuitive mobile application.</p><h3>Project Solution</h3><p>We will achieve our objective though 3 main steps:</p><ol><li><strong>Model Development</strong></li><li><strong>Model Deployment</strong></li><li><strong>Application</strong></li></ol><p>Software used in this project:</p><ol><li>Python 3.7</li><li>Tensorflow 2.0</li><li>Flask + Heroku</li><li>React Native + Firebase</li></ol><h3>Video</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FAnIoOzIaKf8%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DAnIoOzIaKf8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FAnIoOzIaKf8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/31c9efa250cadeead94c719de156396c/href">https://medium.com/media/31c9efa250cadeead94c719de156396c/href</a></iframe><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><h3>Model Development</h3><p>To develop a classification model, we will require the appropriate data (aka lots and lots of images of Korean Food). This includes the original dataset and additional (augmented) dataset built upon the original data. Then, we will use a technique called transfer learning and fine tuning to train our classification model. Here, we will take a look at 2 models — Inception-ResNetV2 and MobileNetV2 — and evaluate them.</p><p>Because of limited computing resources, we will only classify 10 Korean Foods in this prototype.</p><p>We utilised google colab to develop our model as they have good computing power. It can be found <a href="https://colab.research.google.com/drive/1g7e8uryxfulCwIBgzO87CRVH_E8SLSXQ">here</a>.</p><h4>Data Collection</h4><p>The data set used is a subset of the food data set “<em>provided by deep learning contest 2016 by</em><a href="http://kr.nvidia.com/"><em> Nvidia Korea</em></a><em>.” </em>The subset corresponds to the previously stated food dishes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sto4E7gfCk3F-1AkCmdIgg.png" /><figcaption>Sample data from Nvidia Korea Image Classification Contest 2016</figcaption></figure><h4>Data Augmentation</h4><p>After filtering the foods that we want, were we only left with 6400 training images and 640 validation images for 10 classes. That only gives us about 640 images per class for our model to train on. For training a image classifier, this is a very small dataset to work with. In order to create more data for our model to train on, we use a technique called data augmentation. The augmented images may look similar, but to the model, they have very different input values. This allows the model to generalise better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vrs9RTau9c7OOWDFQRssgQ.png" /><figcaption>Augmented images</figcaption></figure><h4>Transfer Learning</h4><p>Our main reason for using transfer learning is due to our small dataset size. In image classification training, the model has to learn complex features built upon learning simpler features. This is called layering. Existing models on the internet such as Inception-ResNetV2 and MobileNetV2 are trained on a huge dataset called ImageNet. These models will contain the ability to identify the simple features and hence, will be used as our ‘feature extractor’ in our own model.</p><p>Below are our iterations of transfer learning with the respective base models.</p><p><strong>Version 1 — Inception ResNetV2</strong></p><p>In our first iteration, we used Inception ResNetV2 as it has the best Top-1 Accuracy on the ImageNet Dataset.</p><pre>Train Set: 6400<br>Validation Set: 640<br>Batch size: 64<br>Epochs: 10<br>Training Time: 10387s<br>Number of Classes: 10</pre><pre>Class Labels: {0: &#39;samgupsal&#39;, 1: &#39;bulgogi&#39;, 2: &#39;ojingeo_bokkeum&#39;, 3: &#39;dakbokkeumtang&#39;, 4: &#39;galchijorim&#39;, 5: &#39;jeyuk_bokkeum&#39;, 6: &#39;ramen&#39;, 7: &#39;bibimbap&#39;, 8: &#39;galbijjim&#39;, 9: &#39;kimchi&#39;}</pre><pre>Input Shape: 224 x 224 x 3<br>Image Augmentation: None</pre><pre>Best Validation Accuracy: 72.34%<br>Base Model: Inception ResnetV2</pre><pre>Inception Resnet Paper: <a href="https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14806">https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14806</a><br>TF Hub: https://tfhub.dev/google/imagenet/inception_resnet_v2/classification/4</pre><pre>Model Size: 219328 KB</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zulaYdFdzzaKkKEsVxUfBA.png" /><figcaption>v1 training screenshot</figcaption></figure><p><strong>Version 2 — MobileNetV2 without Image Augmentation</strong></p><p>In this version,we switched to a smaller model as the Inception ResNet is to large to be deployed on a FREE Heroku server. The validation accuracy in this model is significantly lower but it is faster to train.</p><pre>Train Set: 6400<br>Validation Set: 640<br>Batch size: 32<br>Epochs: 1<br>Training Time: 458s</pre><pre>Number of Classes: 10</pre><pre>Class Labels: {0: &#39;samgupsal&#39;, 1: &#39;bulgogi&#39;, 2: &#39;ojingeo_bokkeum&#39;, 3: &#39;dakbokkeumtang&#39;, 4: &#39;galchijorim&#39;, 5: &#39;jeyuk_bokkeum&#39;, 6: &#39;ramen&#39;, 7: &#39;bibimbap&#39;, 8: &#39;galbijjim&#39;, 9: &#39;kimchi&#39;}</pre><pre>Input Shape: 224 x 224 x 3<br>Image Augmentation: None</pre><pre>Best Validation Accuracy: 39.53%</pre><pre>Base Model: MobileNet V2<br>MobilenetV2 Paper: <a href="https://arxiv.org/abs/1801.04381">https://arxiv.org/abs/1801.04381</a><br>Model size: 16632 KB</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vNlqVUguy6Bv1OcjsaXgHg.png" /><figcaption>v2 training screenshot</figcaption></figure><p><strong>Version 3 — MobileNetV2 with Image Augmentation</strong></p><p>In this version we used image augmentation to increase the training dataset size. The validation accuracy improved as a result.</p><pre>Train Set: 6400<br>Validation Set: 640<br>Batch size: 32<br>Epochs: 2<br>Training Time: 891s</pre><pre>Number of Classes: 10</pre><pre>Class Labels: {0: &#39;samgupsal&#39;, 1: &#39;bulgogi&#39;, 2: &#39;ojingeo_bokkeum&#39;, 3: &#39;dakbokkeumtang&#39;, 4: &#39;galchijorim&#39;, 5: &#39;jeyuk_bokkeum&#39;, 6: &#39;ramen&#39;, 7: &#39;bibimbap&#39;, 8: &#39;galbijjim&#39;, 9: &#39;kimchi&#39;}</pre><pre>Input Shape: 224 x 224 x 3<br>Image Augmentation: ImageDataGenerator(<br>rotation_range=10,<br>width_shift_range=0.1,<br>height_shift_range=0.1,<br>brightness_range=[0.7, 1.3],<br>shear_range=0.2,<br>zoom_range=0.5,<br>horizontal_flip=True,<br>vertical_flip = True,<br>rescale=1/255)</pre><pre>Best Validation Accuracy: 48.59%<br>Base Model: MobileNet V2<br>MobilenetV2 paper: https://arxiv.org/abs/1801.04381</pre><pre>Model size: 16629 KB</pre><p><strong>Version 4 — MobileNetV2 with Image Augmentation and Depth Multiplier</strong></p><p>In this version, We try to use an updated version of mobilenetv2 on tfhub, it was able to train the model more efficiently and accurately than the previous versions.</p><pre>Train Set: 6400<br>Validation Set: 640<br>Batch size: 32<br>Epochs: 10 (early stop at 4)<br>Training Time: 665s<br>Number of Classes: 10</pre><pre>Class Labels: {0: &#39;samgupsal&#39;, 1: &#39;bulgogi&#39;, 2: &#39;ojingeo_bokkeum&#39;, 3: &#39;dakbokkeumtang&#39;, 4: &#39;galchijorim&#39;, 5: &#39;jeyuk_bokkeum&#39;, 6: &#39;ramen&#39;, 7: &#39;bibimbap&#39;, 8: &#39;galbijjim&#39;, 9: &#39;kimchi&#39;}</pre><pre>Input Shape: 224 x 224 x 3</pre><pre>Image Augmentation: ImageDataGenerator(rotation_range=10,<br>width_shift_range=0.1,<br>height_shift_range=0.1,<br>brightness_range=[0.7, 1.3],<br>shear_range=0.2,<br>zoom_range=0.5,<br>horizontal_flip=True,<br>vertical_flip = True,<br>rescale=1/255)</pre><pre>Validation Accuracy: 67.19%</pre><pre>Base Model: MobileNet V2<br>MobilenetV2 paper: <a href="https://arxiv.org/abs/1801.04381">https://arxiv.org/abs/1801.04381</a><br>TF Hub: https://tfhub.dev/google/imagenet/mobilenet_v2_035_224/classification/4</pre><pre>Model size: 6963 KB</pre><p><strong>Model Comparison</strong></p><p>The best validation accuracy out of all the versions of mobileNetV2 models we have trained is 67.19%, whereas the original mobileNetV2 trained on ImageNet has an accuracy of 74.7%. This could be due to a small training set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*1Zjfk02Ls0rV7iCmJ3PRJA.png" /><figcaption>MobileNetV2 performance on ImageNet h<a href="https://arxiv.org/abs/1801.04381">ttps://arxiv.org/abs/1801.04381</a></figcaption></figure><p>The best validation accuracy for our modified Inception Resnet V2 is 72.34% while the validation accuracy of Inception Resnet V2 trained on ImageNet is 80%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*vMqpJYn6V_fBR5j1Sn9EgA.png" /><figcaption>InceptionResNetV2 performance on ImageNet <a href="https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14806">https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14806</a></figcaption></figure><p><strong>Model Summaries</strong></p><p>Below is a table containing the summaries of our trained models.</p><pre>+----------+---------------------+------------+----------------+<br>|  Version | Validation Accuracy | Model Size | Training Time  |<br>+----------+---------------------+------------+----------------+<br>|        1 | 71.88%              | 219 MB     | 3 Hours        |<br>|        2 | 39.53%              | 16 MB      | 7.6 Minutes    |<br>|        3 | 48.59%              | 16 MB      | 14.9 Minutes   |<br>|        4 | 67.19%              | 7 MB       | 10 Minutes     |<br>+----------+---------------------+------------+----------------+</pre><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><h3>Model Deployment</h3><p>We have decided to deploy version 4 of our model as it is relatively small (7MB) and has an accuracy of 67.19%.</p><p>The service that we used to deploy our model on is called Heroku. Its ‘free-tier’ has sufficient memory for our prototype.</p><h4>Local Deployment</h4><p>Before deploying on Heroku, we tested hosting our model as a REST API locally on Flask.</p><p>Download the source code <a href="https://github.com/2manslkh/korean-food-classification-flask">here</a>.</p><p>Your folder structure should look like the one below:</p><pre>flask-app<br>│   app.py<br>│   Procfile<br>│   requirements.txt<br>│<br>└────model<br>        my_model.h5<br>        predict.py</pre><h4>app.py</h4><p>This is the main file which will be used to run your Flask server.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4a439de9427ac8ad6eeb6b1810bfa2c/href">https://medium.com/media/e4a439de9427ac8ad6eeb6b1810bfa2c/href</a></iframe><h4>Procfile</h4><p>This will be used for Heroku deployment later on.</p><h4>requirements.txt</h4><p>Heroku will install the packages in this file.</p><h4>model/my_model.h5</h4><p>Your trained model downloaded from Google Colab.</p><h4>model/predict.py</h4><p>A python script to load and use the model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cc6b49720023483260a0594d56ad224c/href">https://medium.com/media/cc6b49720023483260a0594d56ad224c/href</a></iframe><p>Now, host the flask server locally.</p><pre>$ python app.py<br>* Debugger is active!<br>* Debugger PIN: 986-712-972<br>* Running on <a href="http://0.0.0.0:5000/">http://0.0.0.0:5000/</a> (Press CTRL+C to quit)</pre><p>We can access the API using a simple request client in python:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eb6abd409616ef79110c35030d930821/href">https://medium.com/media/eb6abd409616ef79110c35030d930821/href</a></iframe><h4>Deployment on Heroku</h4><ol><li><a href="https://signup.heroku.com/">Create</a> a Heroku account</li><li><a href="https://devcenter.heroku.com/articles/heroku-cli">Download</a> Heroku CLI</li><li>Login to Heroku via the CLI</li></ol><pre>heroku login<br>heroku: Press any key to open up the browser to login or q to exit<br> ›   Warning: If browser does not open, visit<br> ›   <a href="https://cli-auth.heroku.com/auth/browser/***">https://cli-auth.heroku.com/auth/browser/***</a><br>heroku: Waiting for login...<br>Logging in... done<br>Logged in as me@example.com</pre><p>4. Go (or cd) to your project directory in the CLI</p><p>5. Initialize heroku</p><pre>heroku create [YOUR_APP_NAME]<br>Creating ⬢ [YOUR_APP_NAME]... done<br>https://[YOUR_APP_NAME].herokuapp.com/ | <a href="https://git.heroku.com/[YOUR_APP_NAME].git">https://git.heroku.com/[YOUR_APP_NAME].git</a></pre><p>6. Modify Procfile</p><pre>web: gunicorn app:app</pre><p>7. pip install gunicorn</p><pre>pip install gunicorn</pre><p>8. Modify requirements.txt file</p><pre>pip freeze &gt; requirements.txt</pre><p>9. Deploy to Heroku</p><pre>git add .<br>git commit -m Commit 1<br>git push heroku master</pre><p>10. See what is going on on Heroku</p><pre>heroku logs --tail</pre><p>11. Test your app</p><pre><a href="https://[YOUR_APP_NAME].herokuapp.com/predict?data=[DATA]">https://[YOUR_APP_NAME].herokuapp.com/predict?data=[DATA]</a></pre><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><p><em>⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀</em></p><h3>Mobile Application</h3><p>The application uses the hosted prediction model to identity the food name, after which it sends the food name through <a href="https://developer.edamam.com/food-database-api"><em>Edamam’s</em></a><em> </em>food database API to retrieve the nutritional details. It was built using React Native.</p><p>Using the finalized model, the application is able to identify up to 10 different food labels. The following documentation will focus on the implementation of Firebase’s Realtime Database as the choice of database for this application. We have chosen to use the default Firebase package instead of ‘react-native-firebase’ as it was relatively easy to use the default Firebase package, for my particular use case.</p><p><em>This section assumes understanding of react-native code and will skip the setup of the application UI.</em></p><p><strong><em>The source code can be retrieved from this </em></strong><a href="https://github.com/benjaminlim00/Im-Hungry"><strong><em>link</em></strong></a><strong><em>.</em></strong></p><h4>Data structure explanation</h4><p>When a user takes a photo of his/her food or uploads a new photo, the picture will be sent to a python CNN model hosted online which will return a prediction label. We then assign that picture(image) with a predicted food name(name), as well as a UUID(id).</p><p>Right as we retrieve the food name, we send the food name over to <a href="https://developer.edamam.com/food-database-api"><em>Edamam’s</em></a><em> </em>food database API to retrieve the nutritional details and add the nutritional values to the data structure(nutritionData).</p><p>The whole data structure is added in a JSON format onto the Firebase Realtime database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/0*N5-Y6SSJ5hN9iFmE.png" /><figcaption>Example data structure</figcaption></figure><p>Here are the steps on how we connected Firebase to our application.</p><h4>Step 1: Create a Firebase account</h4><p>The first thing you have to do is to setup a free Firebase account <a href="https://accounts.google.com/signin/v2/identifier?passive=1209600&amp;osid=1&amp;continue=https%3A%2F%2Fconsole.firebase.google.com%2F&amp;followup=https%3A%2F%2Fconsole.firebase.google.com%2F&amp;flowName=GlifWebSignIn&amp;flowEntry=ServiceLogin"><em>here</em></a>. Click on the button that says “Add Project” and fill in the project details. It should be a simple and fuss free process as you just have to select the default settings.</p><p>After that, we have to ensure our application is able to access the database as it is usually locked behind authentication methods. As we are just creating an application for testing, we can simply allow all access to it without the use of passwords.</p><p>To do that, we enter Realtime Database under the Database tab that you can see on the left.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*7QIvg9IqSF5qXoS3.png" /><figcaption>We will be using Firebase’s realtime database to store our data</figcaption></figure><p>Modify the rules tab so that we grant all read and write access. The code snippet is written below.</p><pre>{<br>  &quot;rules&quot;: {<br>    &quot;.read&quot;: true,<br>    &quot;.write&quot;: true<br>  }<br>}</pre><p>Now the database is configured and we move on to link the application with the Firebase SDK.</p><h4>Step 2: Setting up Firebase</h4><p>We will be using the default Firebase package. Run this command in the application’s root folder to install the</p><pre>sudo npm i firebase</pre><p>We will be extracting the Firebase initializing process to a single JS file called config.js, to prevent multiple instantiation of our database.</p><pre>import Firebase from &#39;firebase&#39;;<br><strong>const</strong> firebaseConfig = {  apiKey: &#39;---&#39;,  authDomain: &#39;---&#39;,  databaseURL: &#39;---&#39;,  projectId: &#39;---&#39;,  storageBucket: &#39;---&#39;,  messagingSenderId: &#39;---&#39;,  appId: &#39;---&#39;,  measurementId: &#39;---&#39;,};<strong>const</strong> app = Firebase.initializeApp(firebaseConfig);<strong>const</strong> db = app.database();export { db };</pre><p>You can find your configuration details under “Project Overview”, followed by clicking “Add app”. Select “Web” and give a cool nickname to your app. We selected “Web” as React Native runs just on JavaScript and the web option will give us the code snippet in JavaScript.</p><h4>Step 3: Send queries to Firebase</h4><p>We will look at the 3 JS files where we used queries to edit the data in our database.</p><p>Our application calls the Firebase SDK through config.js in 3 different React components. Lets look through it one by one.</p><p>Firstly, <strong><em>CameraScreen.js.</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/50bad7dfd40813a5d2c30cd54aae8072/href">https://medium.com/media/50bad7dfd40813a5d2c30cd54aae8072/href</a></iframe><pre>import { db } from &#39;../config&#39;;</pre><pre><strong>const</strong> itemsRef = db.ref(&#39;/foodRecords&#39;);</pre><p>As we import <em>db</em>, we create a reference called <em>itemRef</em> which points to the <em>foodRecords</em> path in our database. All future queries on <em>itemRef</em> will append and remove data from this node.</p><p>We can see that as the camera shutter button is pressed, <em>_takePicture</em> will be called which calls_retrievePrediction and _retrieveNutrition from api.js. The two api calls allow us to retrieve the name and nutritionData details respectively which we then assign to as key-value pairs in an Javascipt object structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/255/1*pp6GS_pqyQBf37rZ-v8k3w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/262/1*QGCsPhwU4L8nLJpKll61tg.png" /><figcaption>Example of data structure</figcaption></figure><p>We can see that <strong><em>UploadImageScreen.js </em></strong>follows a similar concept as in <strong><em>CameraScreen.js </em></strong>by calling the two apis as well.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7d26dbfb7ecb9cbee356148dd76522d4/href">https://medium.com/media/7d26dbfb7ecb9cbee356148dd76522d4/href</a></iframe><p>In <strong><em>DiaryScreen.js</em></strong>, we retreive the food records from Firebase and show them using a FlatList.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/62578dd1385286bed1a5ea023da2e877/href">https://medium.com/media/62578dd1385286bed1a5ea023da2e877/href</a></iframe><p>Firstly, we create a reference, same as in <strong><em>CameraScreen</em></strong>.</p><pre>import { db } from &#39;../config&#39;;</pre><pre><strong>const</strong> itemsRef = db.ref(&#39;/foodRecords&#39;);</pre><p>We then retrieve from Firebase the records through the <strong><em>retrieveRecord </em></strong>function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*uhzhlq4e8cnlH8pde1ihWg.png" /></figure><p>The variable named data contains all the records within our database and we proceed to iterate through all the records and save them to state. We display the items using the FlatList component.</p><p>Finally, we also have to allow users to be able to delete diary entries within the FlatList by long-pressing on particular entries.</p><p>Observe the <strong><em>confirmDelete </em></strong>function within <strong><em>ModalDetail.js </em></strong>which is modal that pops up when the user long-presses a particular entry.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c3db62cca7515074df0fcf6aa6d791ff/href">https://medium.com/media/c3db62cca7515074df0fcf6aa6d791ff/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8aa80474d466e270aa1c925699e101f2/href">https://medium.com/media/8aa80474d466e270aa1c925699e101f2/href</a></iframe><p>We can see that the function iterates through our database for a record where it has the ‘id’ value equal to the id value of the ListItem being pressed. It then calls .remove() which will delete the record from our database. That’s how simple it is to remove a record from Firebase’s Realtime database.</p><h3>Screenshots</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*1M-xGMnE8a3ugYdgREH3wQ.jpeg" /><figcaption>CameraScreen.js, where you take a photo of a food item</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*dqWMm8ndENYFx-crPxOH-w.jpeg" /><figcaption>UploadScreen.js, where you upload a picture of a meal you forgot</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*ggli_MJj6Y2cebjD9oX98w.jpeg" /><figcaption>UserScreen.js, where you can see the user’s details</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*m5zyNI5PIDmQDVmFZw9XzQ.jpeg" /><figcaption>DiaryScreen.js, where the food history is recorded</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*xV_yK623fJiXI43pSyY3Ug.jpeg" /><figcaption><a href="https://github.com/benjaminlim00/Im-Hungry/blob/master/src/components/ModalDetail.js">ModalDetail.j</a>s, where you can see the nutritional details of a single food item</figcaption></figure><h3>Application in action</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*73AkiDuVozkk4CQgPqpO8g.gif" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b372d92fe563" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying Tensorflow 2.0 Model on Heroku]]></title>
            <link>https://medium.com/@limkenghin/deploying-tensorflow-2-0-model-on-heroku-1b5cd49f8a2?source=rss-d46d755fc75------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b5cd49f8a2</guid>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Lim Keng Hin]]></dc:creator>
            <pubDate>Wed, 04 Dec 2019 10:41:42 GMT</pubDate>
            <atom:updated>2019-12-04T10:53:08.458Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article, we will explore how to deploy a TF 2.0 model on Heroku.</p><h3>Core Requirements</h3><p>Python &gt; 3.6.x<br>Tensorflow 2.0<br>Flask<br>Heroku<br>Git</p><h4>Overview</h4><ol><li>Model Training</li><li>Saving the model</li><li>Deployment on localhost (FLASK)</li><li>Deployment on Heroku</li><li>Common Bugs</li><li>Future Improvements</li></ol><p>The example that we will be using is a Korean Food Classification API that I have deployed on Heroku for an AI project during my exchange studies in Hanyang University.</p><h3>1. Model Training</h3><p>I found that training models on the cloud is much more efficient and less prone to crashes, therefore I used Google Colab to train my model.</p><p>For my project, I used a technique called Transfer Learning to create my own Korean Food Classification Model.</p><p>I applied this technique on 2 pre-trained models: ResNet50V2 and MobileNetV2.</p><h4>Preparing Google Colab</h4><p>By default, Google Colab uses Tensorflow 1.15.0 (TF 2.0 will be the default in future versions)</p><pre>!pip install — upgrade tensorflow</pre><p>Restart the Kernel after this step.</p><p>Next, we will mount out cloud drive onto our cloud project to allow us to read and write data.</p><pre>from google.colab import drive<br>drive.mount(&#39;/content/drive&#39;,force_remount=True)</pre><h4>Preparing Transfer Learning</h4><p>Variables to assign values to:</p><p>IMG_HEIGHT — Height of Image</p><p>IMG_WIDTH — Width of Image</p><p>IMG_CHANNELS — Number of Colour Channels</p><p>NUM_CLASSES — Number of Prediction classes</p><pre>import tensorflow as tf<br>from tensorflow.keras import layers<br>from tensorflow.keras.models import Model</pre><pre># Load MobileNetV2 Model from tf.keras.applications</pre><pre>pretrained_model = tf.keras.applications.MobileNetV2(<br>input_shape=(IMG_HEIGHT, IMG_WIDTH, IMG_CHANNELS), include_top=False)</pre><pre># include_top=False removes the last Dense &#39;prediction&#39; layer</pre><pre>### STACKING METHOD ###<br>x = pretrained_model.layers[-2].output<br>x = layers.Flatten()(x)<br>x = layers.Dense(NUM_CLASSES, activation=&#39;softmax&#39;)(x)<br>model = Model(inputs=pretrained_model.input,outputs=x)</pre><pre># Freeze all weights except last layer<br>for layer in model.layers[:-2]:<br>    layer.trainable=False</pre><pre># Compile model<br>model.compile(optimizer=&#39;adam&#39;,<br>loss=tf.keras.losses.SparseCategoricalCrossentropy(),<br>metrics=[&#39;accuracy&#39;])</pre><pre>model.summary()</pre><p>After preparing the Dataset (image augmentation, validation split, shuffling etc.), we will train the model.</p><pre>model.fit(X_train,Y_train,verbose=1,epochs=10,batch_size=32,validation_data=v_data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zulaYdFdzzaKkKEsVxUfBA.png" /></figure><h3>2. Saving the Model</h3><pre>model.save(&#39;/content/drive/My Drive/data/saved_models/my_model.h5&#39;)</pre><p>The model will be saved in your drive location as specified above.</p><p>Next, we must download the model and proceed to deploy it locally and then on Heroku.</p><h3>3. Deployment on Localhost (FLASK)</h3><p>Create a project folder like this:</p><pre>flask-app<br>│   app.py<br>│   Procfile<br>│   requirements.txt<br>│<br>└────model<br>        my_model.h5<br>        predict.py</pre><h4>app.py</h4><p>This is the main file which will be used to run your Flask server.</p><pre>from flask import Flask,request<br>from model import predict</pre><pre># create the flask object<br>app = Flask(__name__)</pre><pre>@app.route(&#39;/&#39;)<br>def index():<br>    return &quot;Index Page&quot;</pre><pre>@app.route(&#39;/predict&#39;,methods=[&#39;GET&#39;,&#39;POST&#39;])<br>def predict():<br>    data = request.form.get(&#39;data&#39;)<br>    if data == None:<br>        return &#39;Got None&#39;<br>    else:<br>        # model.predict.predict returns a dictionary<br>        prediction = model.predict.predict(data) </pre><pre>    return json.dumps(str(prediction))</pre><pre>if __name__ == &quot;__main__&quot;:<br>    app.run(host=&#39;0.0.0.0&#39;,debug=True)</pre><h4>Procfile</h4><p>This will be used for Heroku deployment later on.</p><h4>requirements.txt</h4><p>Heroku will install the packages in this file.</p><h4>model/my_model.h5</h4><p>Your trained model downloaded from Google Colab.</p><h4>model/predict.py</h4><p>A python script to load and use the model.</p><pre>import tensorflow as tf</pre><pre>model = tf.keras.models.load_model(&#39;model/my_model.h5&#39;)</pre><pre>def predict(data):</pre><pre>    ###  IMAGE PREPROCESSING  ###<br>                  |<br>                  |<br>    #  data--&gt;preprocessed_data #<br>                  |<br>                  |<br>    ###          END          ###</pre><pre>    prediction = model.predict(preprocessed_data)<br>    <br>    # You may want to further format the prediction to make it more<br>    # human readable</pre><pre>    return prediction</pre><p>Now, host the flask server locally.</p><pre>$ python app.py</pre><pre>* Debugger is active!<br>* Debugger PIN: 986-712-972<br>* Running on <a href="http://0.0.0.0:5000/">http://0.0.0.0:5000/</a> (Press CTRL+C to quit)</pre><p>We can access the API using a simple request client in python:</p><pre>import requests<br>import json</pre><pre>def main():</pre><pre>    PARAMS = {&#39;data&#39;:&#39;input_data&#39;}<br>    response = requests.post(url=&#39;http://localhost:5000/predict?&#39;,data=PARAMS)<br>    print(response.text)</pre><pre>if __name__ == &quot;__main__&quot;:<br>    main()</pre><h3>4. Deployment on Heroku</h3><ol><li><a href="https://signup.heroku.com/">Create</a> a Heroku account</li><li><a href="https://devcenter.heroku.com/articles/heroku-cli">Download</a> Heroku CLI</li><li>Login to Heroku via the CLI</li></ol><pre>heroku login<br>heroku: Press any key to open up the browser to login or q to exit<br> ›   Warning: If browser does not open, visit<br> ›   https://cli-auth.heroku.com/auth/browser/***<br>heroku: Waiting for login...<br>Logging in... done<br>Logged in as me@example.com</pre><p>4. Go (or cd) to your project directory in the CLI</p><p>5. Initialize heroku</p><pre>heroku create [YOUR_APP_NAME]<br>Creating ⬢ [YOUR_APP_NAME]... done<br>https://[YOUR_APP_NAME].herokuapp.com/ | https://git.heroku.com/[YOUR_APP_NAME].git</pre><p>6. Modify Procfile</p><pre>web: gunicorn app:app</pre><p>7. pip install gunicorn</p><pre>pip install gunicorn</pre><p>8. Modify requirements.txt file</p><pre>pip freeze &gt; requirements.txt</pre><p>9. Deploy to Heroku</p><pre>git add .<br>git commit -m Commit 1<br>git push heroku master</pre><p>10. See what is going on on Heroku</p><pre>heroku logs --tail</pre><p>11. Test your app</p><pre>https://[YOUR_APP_NAME].herokuapp.com/predict?data=[DATA]</pre><h3>5. Common Bugs / Things to look out for</h3><ul><li>Heroku doesn’t support some versions of Tensorflow.</li><li>Tensorflow Hub is not fully compatible with Heroku yet.</li><li><a href="https://devcenter.heroku.com/articles/python-c-deps">Some packages</a> cannot be installed on Heroku.</li></ul><h3>6. Future Improvements</h3><p>Hopefully Heroku will be fully compatible with Tensorflow Hub models because they tend to be provide better results as compared to the default ones provided by tf.keras.application.</p><p>There you have it, you have successfully deployed your Tensorflow model as a web API on Heroku!</p><p>To see the API used in a React Native environment, check out <a href="https://medium.com/@benjaminlpa1996/react-native-firebase-smart-food-recognition-app-a7bfc513aa8">this article</a> by <a href="https://github.com/benjaminlim00">Benjamin Lim</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b5cd49f8a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>