<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Zaid Alyafeai on Medium]]></title>
        <description><![CDATA[Stories by Zaid Alyafeai on Medium]]></description>
        <link>https://medium.com/@alyafey22?source=rss-14424af03ecf------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Av4jziiLCRXhKyhkNh4ePg.jpeg</url>
            <title>Stories by Zaid Alyafeai on Medium</title>
            <link>https://medium.com/@alyafey22?source=rss-14424af03ecf------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 08:18:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alyafey22/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Fast pix2pix in the Browser]]></title>
            <link>https://medium.com/@alyafey22/fast-pix2pix-in-the-browser-287d9858a5e4?source=rss-14424af03ecf------2</link>
            <guid isPermaLink="false">https://medium.com/p/287d9858a5e4</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Zaid Alyafeai]]></dc:creator>
            <pubDate>Fri, 07 Sep 2018 16:11:49 GMT</pubDate>
            <atom:updated>2019-04-08T20:30:24.678Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*gHtfxUy3HzeAiw4zf5t3Lw.gif" /></figure><h3>Introduction</h3><p>In this post we will make a pix2pix app that runs directly in the browser using Tensorflow.js. The app creates realistic images given a sketch. You can check the <a href="https://zaidalyafeai.github.io/pix2pix/cats.html">demo</a>. If you are not familiar with Tensorflow.js then have a look at this <a href="https://medium.com/tensorflow/a-gentle-introduction-to-tensorflow-js-dba2e5257702">tutorial</a>. You can find all the code <a href="https://github.com/zaidalyafeai/zaidalyafeai.github.io/tree/master/pix2pix">here</a>.</p><h3>What is pix2pix</h3><p>pix2pix is a method that uses GANs to generate realistic images conditioned on certain inputs.</p><p>Typically the training set contains images of size (256, 2* 256)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/0*F8x-YlnvBKp_Znsq.jpg" /><figcaption>from facades dataset</figcaption></figure><p>Each image contains two augmented images. The first image is the target image <strong>A</strong> and the second image is the input/condition image <strong>B</strong>.</p><p>The pix2pix architecture contains two networks. The first one is the discriminator network which tries to guess if the current image is real or fake conditioned on the input image <strong>B</strong>. The second network is called a generator network which tries given the condition image <strong>B</strong> to generate similar images to <strong>A</strong> to fool the discriminator network</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vwCOla_RiuTD6pEEMitTgQ.png" /><figcaption>architecture <a href="https://affinelayer.com/pix2pix/">source</a></figcaption></figure><p>The two networks play a min-max game to optimize the output and generate realistic images given the condition images.</p><p>For more details read this wonderful <a href="https://affinelayer.com/pix2pix/">post</a>.</p><h3>Dataset</h3><h4>Labeled Datasets</h4><p>The pix2pix project page contains many datasets that are already labeled. You can find them <a href="http://efrosgans.eecs.berkeley.edu/pix2pix/datasets/">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UZH4vzlBCa1xmrtO.jpg" /><figcaption><a href="https://github.com/phillipi/pix2pix">Source</a></figcaption></figure><h4>Create your own dataset</h4><p>The pix2pix authors suggest using Holistically Nested Edge Detection (<a href="https://github.com/s9xie/hed">HED</a>) to generate the edges.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/981/0*hTKRLtLvptD6EAXD" /><figcaption><a href="https://github.com/s9xie/hed">Source</a></figcaption></figure><p>It uses a Caffe model that was trained on a large dataset to generate edges of different labels.</p><p>You can run this python <a href="https://github.com/zaidalyafeai/pix2pix/blob/master/scripts/edges/batch_hed.py">script </a>to create .mat files. After that run this Matlab <a href="https://github.com/zaidalyafeai/pix2pix/blob/master/scripts/edges/PostprocessHED.m">script</a> for post-processing to create the final edges</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*3vN9dLe4co8yDGw0wS4pBA.png" /><figcaption>Edges</figcaption></figure><p>A simpler approach is to use Canny edge detector from OpenCV cv2.Canny(img, min, max)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*weRFFG3_30K10chmJw8Aqw.png" /></figure><h3>Pix2Pix Training</h3><p>I trained on a modified version of the <a href="https://github.com/affinelayer/pix2pix-tensorflow">tensorflow </a>implementation. You can run the modified version on colab <a href="https://colab.research.google.com/github/zaidalyafeai/zaidalyafeai.github.io/blob/master/pix2pix/tf_pix2pix.ipynb">here</a>.</p><h4>Important Arugments</h4><ol><li>input_dir defines the input directory of the training dataset</li><li>mode defines the current mode [train, test, export]</li><li>output_dir the location to save output images and checkpoints .</li><li>checkpoint this is used to continue training from a saved checkpoint.</li><li>which_direction choose AtoB if the input is the first image and second image is the target output. BtoA for the opposite.</li><li>ngf, ndf defines the a factor that manages the number of parameters in the generator and discriminator respectively. In the original paper the default is ngf = ndf = 64. In our implementation we use 32 to reduce the number of parameters and enforce faster prediction time.</li></ol><h4>Start Training</h4><p>Set mode=train and define the input_dirand output_dir variables. Start main() function to start training. You can also define some display parameters like display_freq to show the output images, max_epochs for the number of epochs and progress_freq to show the loss.</p><p>Once you start training you will see three important loss values</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/233/1*7M9wlhb7lBy5dI9j39KwNA.png" /></figure><ol><li>discrim_loss defines the ability of the discriminator to recognize fake/real images. The larger the value, the better visual results.</li><li>gen_loss_GAN defines the generator loss which should get lower once the training proceeds.</li><li>gen_loss_L1 defines the difference between the output (fake images) and the target images of the training process.</li></ol><h4>Improve Training</h4><p>Since we are training with less parameters the visual outputs will be much worse than the original paper. To obtain much better results we introduce a simple trick that augments the training dataset with more images and hence create much detailed images.</p><p>The idea is similar to pyramid scaling where we train on different random scales of each image. We choose a random scaling between (286, 300) then we apply random cropping back to (256, 256)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*ACiQ3S697GFHkY5C6xi3tA.png" /></figure><p>Training on different scales helps capturing details when using lesser number of parameters.</p><p>This is highlighted using the following code</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3b7b20a8fc34fe8b2989ea63338b7413/href">https://medium.com/media/3b7b20a8fc34fe8b2989ea63338b7413/href</a></iframe><p>after applying this function we see some better visualizations</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/615/1*0PFpuNOzGax3F9NDTC2r-Q.png" /><figcaption>(left) without scaling, (right) with random scaling we see better textures and realization of details.</figcaption></figure><h4>Less Parameters</h4><p>If we use ndf = ngf = 16 we have much faster predictions but worse visualizations with no smooth details</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*12dnK56VRENbLSnFMab5Rw.png" /><figcaption>(left) ndf=ngf=32 , (right) ndf=ngf=16</figcaption></figure><h3>Export the Model</h3><p>Once you are satisfied with the results you can convert the model to a web format by following the following steps</p><ol><li>Change to mode=export and set checkpoint to the directory where you have the model checkpoints.</li><li>Convert the model to keras using the following <a href="https://github.com/zaidalyafeai/zaidalyafeai.github.io/blob/master/pix2pix/convert_keras.py">script</a> by runningpython convert_keras.py --dir input_dir --out output_dir note that input_dir is the directory where you have the saved checkpoints and output_dir will contain the keras.h5 file.</li><li>Install tensorflow.js pip install tensroflowjs then run the command</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dd813d84515d4971a5ea95a22b554170/href">https://medium.com/media/dd813d84515d4971a5ea95a22b554170/href</a></iframe><p>The tensorflowjs folder is the output directory which will contain the model topology and the weights files.</p><h3>Run on the Browser</h3><p>First import the following JavaScript <a href="https://github.com/zaidalyafeai/zaidalyafeai.github.io/blob/master/pix2pix/tf.min.js">library</a> into your application</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4a36c41205f0c6963141fe9f5ae3804f/href">https://medium.com/media/4a36c41205f0c6963141fe9f5ae3804f/href</a></iframe><p>Now you can load the model using</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4008f22da375d06aac2e0f31b9540d4f/href">https://medium.com/media/4008f22da375d06aac2e0f31b9540d4f/href</a></iframe><p>After that you need to do some preprocessing before running predictions</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c5594e3c3d30792ebdb6533b899176d4/href">https://medium.com/media/c5594e3c3d30792ebdb6533b899176d4/href</a></iframe><p>Basically we are the getting the image data from the canvas then we do scaling and normalization.</p><p>We also need a post-processing function</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1fd17b73396012912daaed5771a949e9/href">https://medium.com/media/1fd17b73396012912daaed5771a949e9/href</a></iframe><p>Basically we undo what we have done in the preprocessing function. Finally, we can do predictions</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5906f8a70775452f6b202f498bc50b69/href">https://medium.com/media/5906f8a70775452f6b202f498bc50b69/href</a></iframe><figure><a href="https://usejournal.com/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_image"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f2IVAl0TbsfES9cFGYr40g.png" /></a></figure><p>📝 Read this story later in <a href="https://usejournal.com/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_read_later_text">Journal</a>.</p><p>🗞 Wake up every Sunday morning to the week’s most noteworthy Tech stories, opinions, and news waiting in your inbox: <a href="https://usejournal.com/newsletter/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_text">Get the noteworthy newsletter &gt;</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=287d9858a5e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Train, Test and Deploy ML models in the Browser using One Notebook]]></title>
            <link>https://medium.com/@alyafey22/train-test-and-deploy-ml-models-in-the-browser-using-one-notebook-ef0e8a7c29e4?source=rss-14424af03ecf------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef0e8a7c29e4</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Zaid Alyafeai]]></dc:creator>
            <pubDate>Thu, 09 Aug 2018 00:17:50 GMT</pubDate>
            <atom:updated>2018-08-09T00:17:50.454Z</atom:updated>
            <content:encoded><![CDATA[<p>In this tutorial we learn how to train, test and deploy machine learning models on the browser for free. Free GPU usage and free hosting. We will only work on a Google Colab notebook which enables you to train on a GPU for free. The end product of this tutorial will be the following <a href="https://t.co/hnUoZlzWGx">notebook</a>.</p><h4>Prerequisites</h4><ol><li>Google Account</li><li>GitHub Account</li></ol><h4>Pipeline</h4><p>The following pipeline summarizes the idea</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/323/1*5_tbp1f1HO24mOQz-Cj21w.png" /><figcaption><strong>Pipeline</strong></figcaption></figure><h4>Setup a GitHub Page</h4><p>First you need to create a GitHub page. Which is a free domain where you can create a web page for free and host your files there. You can create one using this <a href="https://guides.github.com/features/pages/">guide</a>. Once you are done with that you will have a web page with a similar domain to <a href="http://zaidalyafeai.github.io/">mine</a>. Its associated GitHub repository is <a href="https://github.com/zaidalyafeai/zaidalyafeai.github.io">here</a>.</p><h4>Setup Colab</h4><p>Open your Google drive account and create a new Colab file using this <a href="https://colab.research.google.com/">link</a>. Create a new Python 3 notebook. Once Loaded, enable GPU computations by clicking Edit then Notebook Settings. Set the Hardware accelerator to GPU</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*Trr2U9peDM8QPz-v79LSgg.png" /><figcaption><strong>Setup GPU</strong></figcaption></figure><h4>Clone GitHub Repository</h4><p>We will upload the model to our GitHub page to run in the browser. Make sure you first configure your name and email</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff8642e20fc693af323ef38ffdcee2cf/href">https://medium.com/media/ff8642e20fc693af323ef38ffdcee2cf/href</a></iframe><p>Then clone the repository that you created in the previous steps. It will be something similar to <a href="https://github.com/zaidalyafeai/zaidalyafeai.github.io">https://github.com/zaidalyafeai/zaidalyafeai.github.io</a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d80a2914693d477dedbb32fcf025b17f/href">https://medium.com/media/d80a2914693d477dedbb32fcf025b17f/href</a></iframe><p>Change the directory</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/154a16e640d862177e4cf48ed460a3c9/href">https://medium.com/media/154a16e640d862177e4cf48ed460a3c9/href</a></iframe><p>Make a new folder for the project</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/444b3356d82ba2803c494402352f184a/href">https://medium.com/media/444b3356d82ba2803c494402352f184a/href</a></iframe><p>Create two folders for the keras model and the web model</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/03f240d3352ce484c2fb2b1757191291/href">https://medium.com/media/03f240d3352ce484c2fb2b1757191291/href</a></iframe><h4>Create a Model</h4><p>We will create a model using Keras. The model will take two binary inputs and predict the xor output as in the table</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/152/1*8RmcPqC_f03LaqztF9pZrA.png" /><figcaption>XOR table</figcaption></figure><p>Necessary imports</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/81f32d7d942a52200595322bbfd389f4/href">https://medium.com/media/81f32d7d942a52200595322bbfd389f4/href</a></iframe><p>Prepare inputs and their truth values</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1b68fd2a7338c22b73a611aea24ba95e/href">https://medium.com/media/1b68fd2a7338c22b73a611aea24ba95e/href</a></iframe><p>Now, we create the model and compile it with SGD optimizer</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/71bf67cd4ebd468ea2f97fd8f601aada/href">https://medium.com/media/71bf67cd4ebd468ea2f97fd8f601aada/href</a></iframe><p>We then run the model for 1000 epochs</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5211f77f6b14a78fd009ba4109cafa4c/href">https://medium.com/media/5211f77f6b14a78fd009ba4109cafa4c/href</a></iframe><p>Predict the output results</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fbe84d68d73acda47149532d0b33963d/href">https://medium.com/media/fbe84d68d73acda47149532d0b33963d/href</a></iframe><p>Save the model for conversion later</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d03d454bf614d6d1173d01f20606e59c/href">https://medium.com/media/d03d454bf614d6d1173d01f20606e59c/href</a></iframe><h4>Convert the Model to Web Format</h4><p>Install tensorflowjs library for converting the model to web format</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4357badd811646acc2c3944754efe2c3/href">https://medium.com/media/4357badd811646acc2c3944754efe2c3/href</a></iframe><p>Convert the model to web format</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f694a4541ddd325e50975f2e23e1513c/href">https://medium.com/media/f694a4541ddd325e50975f2e23e1513c/href</a></iframe><h4>Create a Web Page</h4><p>The header must import the TensorFlow.js library for running the model on the browser</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c191415a12c6434d0af2be92452d1fa4/href">https://medium.com/media/c191415a12c6434d0af2be92452d1fa4/href</a></iframe><p>Then, create the body of the page for loading the model and making predictions. Basically we try to input the tensor [[0,0]] for proving that it works. The output should be close to zero</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/84a44698d406d2711e096017d646872f/href">https://medium.com/media/84a44698d406d2711e096017d646872f/href</a></iframe><p>Finally, create the body for showing the output tensor</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/074fd3bba3af3db29dfa6353432e5cbb/href">https://medium.com/media/074fd3bba3af3db29dfa6353432e5cbb/href</a></iframe><p>Save the code as a html file</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4757df16f326ceb88ed0a5c842eb7a85/href">https://medium.com/media/4757df16f326ceb88ed0a5c842eb7a85/href</a></iframe><h4>Upload the Page to GitHub</h4><p>First create a token using the following <a href="https://github.com/settings/tokens/new">link</a>. The secret token allows you to push your changes to your repo on GitHub. It expires in a few minutes so if you face an error make sure to generate it. It will be similar to this7ab3a8fe5742bf829f1a832a5f330a8365820 .</p><p>After that, you can push the folder using the following code after you make sure to replace the link with your token and page repo link</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/87906ec2dffa0d087e7cc39cda711433/href">https://medium.com/media/87906ec2dffa0d087e7cc39cda711433/href</a></iframe><p>Finally you can open the page and inspect the results. Check out my <a href="https://zaidalyafeai.github.io/XOR/">page</a>.</p><h4>Resources</h4><ol><li>Learn about TensorFlow.js <a href="https://medium.com/tensorflow/a-gentle-introduction-to-tensorflow-js-dba2e5257702">https://medium.com/tensorflow/a-gentle-introduction-to-tensorflow-js-dba2e5257702</a></li><li>Learn about Colab <a href="https://medium.com/deep-learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d">https://medium.com/deep-learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d</a></li><li>Learn about GitHub Pages <a href="https://pages.github.com/">https://pages.github.com/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef0e8a7c29e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Train a model in tf.keras with Colab, and run it in the browser with TensorFlow.js]]></title>
            <link>https://medium.com/tensorflow/train-on-google-colab-and-run-on-the-browser-a-case-study-8a45f9b1474e?source=rss-14424af03ecf------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a45f9b1474e</guid>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[tensorflowjs]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Zaid Alyafeai]]></dc:creator>
            <pubDate>Mon, 02 Jul 2018 21:23:31 GMT</pubDate>
            <atom:updated>2018-09-27T22:16:32.804Z</atom:updated>
            <content:encoded><![CDATA[<p>We will create a simple tool that recognizes drawings and outputs the names of the current drawing. This app will run directly on the browser without any installations. We will use Google Colab for training the model, and we will deploy it on the browser using TensorFlow.js.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*Xbvo-Diilbltp3AEgtTOig.gif" /></figure><h3>Code and Demo</h3><p>Find the live <a href="https://zaidalyafeai.github.io/sketcher/">demo</a> and the <a href="https://github.com/zaidalyafeai/zaidalyafeai.github.io/tree/master/sketcher">code</a> on GitHub. Also make sure to test the notebook on Google Colab <a href="https://colab.research.google.com/github/zaidalyafeai/zaidalyafeai.github.io/blob/master/sketcher/Sketcher.ipynb">here</a>.</p><h3>Dataset</h3><p>We will use a CNN to recognize drawings of different types. The CNN will be trained on the Quick Draw <a href="https://github.com/googlecreativelab/quickdraw-dataset">dataset</a>. The dataset contains around 50 million drawings of 345 classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/901/1*kMEmKip2dqTQPX4m9Nt8gQ.png" /><figcaption>A subset of the classes</figcaption></figure><h3>Pipeline</h3><p>We will train the model on GPU for free on Google Colab using Keras then run it on the browser directly using <a href="http://js.tensorflow.org/">TensorFlow.js</a>(tfjs) . I created a <a href="https://medium.com/tensorflow/a-gentle-introduction-to-tensorflow-js-dba2e5257702">tutorial</a> on TensorFlow.js. Make sure to read it before continuing. Here is the pipeline of the project</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*i2S-d3c-NcFxCnUhNd_Wfg.png" /><figcaption>The pipeline</figcaption></figure><h3>Train on Colab</h3><p>Google provides free processing power on a GPU. You can see this <a href="https://medium.com/deep-learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d">tutorial</a> on how to create a notebook and activate GPU programming.</p><h4>Imports</h4><p>we will use keras with tensorflow backend</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/74498d842347d471868533bd11f334c1/href">https://medium.com/media/74498d842347d471868533bd11f334c1/href</a></iframe><h4>Load the Data</h4><p>Since we have a limited memory we will not train on all the classes. We will only use <a href="https://raw.githubusercontent.com/zaidalyafeai/zaidalyafeai.github.io/master/sketcher/mini_classes.txt">100</a> classes of the dataset. The data for each class is available on <a href="https://console.cloud.google.com/storage/browser/quickdraw_dataset/full/numpy_bitmap?pli=1">Google Cloud</a> as numpy arrays of the shape [N,784] where N is the number of of the images for that particular class. We first download the dataset</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7dfe857093bc6358ed5854ae6c5ce8f6/href">https://medium.com/media/7dfe857093bc6358ed5854ae6c5ce8f6/href</a></iframe><p>Since our memory is limited we will only load to memory 5000 images per classes. We also reserve 20% of the data unseen for testing</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6316007c17b755d3b2c30169779579a5/href">https://medium.com/media/6316007c17b755d3b2c30169779579a5/href</a></iframe><h4>Preprocess the Data</h4><p>We preprocess the data to prepare it for training. The model will take batches of the shape [N, 28, 28, 1] and outputs probabilities of the shape [N, 100]</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1eeabc405e5bec07f26f55ae753a232a/href">https://medium.com/media/1eeabc405e5bec07f26f55ae753a232a/href</a></iframe><h4>Create the Model</h4><p>We will create a simple CNN. Notice that the simpler the model with lesser number of parameters the better. Indeed, we will run the model after conversion on the browser and we want the model to run fast for prediction. The following model contains 3 conv layers and 2 dense layers.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5eaadacbeec6933b829b9b3f3f54fdf5/href">https://medium.com/media/5eaadacbeec6933b829b9b3f3f54fdf5/href</a></iframe><h4>Fit, Validate and Test</h4><p>After that we train the model for 5 epochs and 256batches with 10% validation split</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eecf38159d8d8db7cba954f7f7785048/href">https://medium.com/media/eecf38159d8d8db7cba954f7f7785048/href</a></iframe><p>Here is the results of the training</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*t5OP5I4f3qcZU0OzGIrcNQ.png" /></figure><p>And the testing accuracy is 92.20% top 5 accuracy.</p><h4>Prepare the model for Web Format</h4><p>After we are satisfied about the accuracy of the model we save it in order to convert it</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fbc8bdebe6bd130ef20a91bfede29b91/href">https://medium.com/media/fbc8bdebe6bd130ef20a91bfede29b91/href</a></iframe><p>we install the tfjs package for conversion</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5be96b14b7cfcfc1c575b41fadbf77c2/href">https://medium.com/media/5be96b14b7cfcfc1c575b41fadbf77c2/href</a></iframe><p>then we convert the model</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f351ef34ce6d456e586f2d6867b527a5/href">https://medium.com/media/f351ef34ce6d456e586f2d6867b527a5/href</a></iframe><p>This will create some weight files and the json file which contains the architecture of the model.</p><p>zip the model to prepare for downloading it to our local machine</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90873f261c9f05342d01423d15f98618/href">https://medium.com/media/90873f261c9f05342d01423d15f98618/href</a></iframe><p>finally download the model</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aab499919eb935ae3d87ea5dae367bec/href">https://medium.com/media/aab499919eb935ae3d87ea5dae367bec/href</a></iframe><h3>Inference on the Browser</h3><p>In this section we show how to load the model and make inference. I will assume that we have a canvas of size 300 x 300 . I will not go over the details of the interface and focus on TensorFlow.js part.</p><h4>Loading the Model</h4><p>In order to use TensorFlow.js first use the following script</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5a6a8dfe968e973e18dd25d7fc182c79/href">https://medium.com/media/5a6a8dfe968e973e18dd25d7fc182c79/href</a></iframe><p>You will need a running server on your local machine to host the weight files. You can create an apache server or host the page on <a href="https://pages.github.com/">GitHub</a> as I did on my <a href="https://github.com/zaidalyafeai/zaidalyafeai.github.io/tree/master/sketcher">project</a>.</p><p>After that, load the model to the browser using</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4008f22da375d06aac2e0f31b9540d4f/href">https://medium.com/media/4008f22da375d06aac2e0f31b9540d4f/href</a></iframe><p>The await keyword waits for the model to be loaded by the browser.</p><h4>Preprocessing</h4><p>We need to preprocess the data before making a prediction. First get the image data from the canvas</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d161a1aa8a60b66a5fc35175ea9c98a0/href">https://medium.com/media/d161a1aa8a60b66a5fc35175ea9c98a0/href</a></iframe><p>The getMinBox() will be explained later. The variable dpi is used to stretch the crop of the canvas according to the density of the pixels of the screen.</p><p>We take the current image data of the canvas convert it to a tensor, resize and normalize</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/edc988e4809580e58ada1e043e68117d/href">https://medium.com/media/edc988e4809580e58ada1e043e68117d/href</a></iframe><p>For prediction we use model.predict this will return probabilities of the shape [N, 100]</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f878895af15875e6678d14842b0fe757/href">https://medium.com/media/f878895af15875e6678d14842b0fe757/href</a></iframe><p>We can then use simple functions to find the top 5 probabilities.</p><h4>Improve Accuracy</h4><p>Remember that that our model accepts tensors of the shape [N, 28, 28,1] . The drawing canvas we have is of size 300 x 300 which might be two large for drawings or the user might draw a small figure. It will be better to crop only the box that contains the current drawing. To do that we extract the minimum bounding box around the drawing by finding the top left and the bottom right points</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a625aadc6b9e10c6acb972a556dea3c2/href">https://medium.com/media/a625aadc6b9e10c6acb972a556dea3c2/href</a></iframe><h3>Testing Drawings</h3><p>Here are some first time drawings and the highest percentage class. I made all the drawings with my mouse. You should get better accuracy using a pen …</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*2geX9YeCUCZO_Uign40g7A.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a45f9b1474e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/train-on-google-colab-and-run-on-the-browser-a-case-study-8a45f9b1474e">Train a model in tf.keras with Colab, and run it in the browser with TensorFlow.js</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sentiment Classification from Keras to the Browser]]></title>
            <link>https://medium.com/@alyafey22/sentiment-classification-from-keras-to-the-browser-7eda0d87cdc6?source=rss-14424af03ecf------2</link>
            <guid isPermaLink="false">https://medium.com/p/7eda0d87cdc6</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Zaid Alyafeai]]></dc:creator>
            <pubDate>Tue, 10 Apr 2018 19:25:21 GMT</pubDate>
            <atom:updated>2018-04-10T19:25:21.485Z</atom:updated>
            <content:encoded><![CDATA[<p>We will create a classifier using Keras to differentiate between positive sentiments and negative sentiments for movies. Then we will load the model to the browser for user prediction.</p><h4>The Code</h4><p>I created a repository on <a href="https://github.com/zaidalyafeai/Browser-Sentiment-Classification">GitHub</a> with the code required to follow the tutorial. If you spot any error or faced any problems please raise an issue there.</p><h4>The Dataset</h4><p>We will use the <a href="https://www.kaggle.com/c/si650winter11/data">dataset</a> for sentiment classification. The dataset contains 7086 statements about movies with labels. label 1 means positive sentiment and 0 means negative sentiment.</p><h4>Preprocessing the data</h4><p>For text analysis we need first to preprocess the data. First of all, upon uploading the data we need to split the sentences into words then populate the training set.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2df7778de1ab4ece7392f834e38c978/href">https://medium.com/media/e2df7778de1ab4ece7392f834e38c978/href</a></iframe><p>Then after that we will need to create the dictionary word_index that will map words to integers for embedding. Note that the dictionary length might be huge so we count the counts of each word in the training set and we only include words that repeat more than some threshold.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f0a726ff468c716f663db0b9091f46a7/href">https://medium.com/media/f0a726ff468c716f663db0b9091f46a7/href</a></iframe><p>The process method will create words out of statements by removing punctuations</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/82ed8169e4c275ee747c9e029153693e/href">https://medium.com/media/82ed8169e4c275ee747c9e029153693e/href</a></iframe><p>After that we will need to map each sentimental statement to a sequence of integers but first get the statement with the largest number of words</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1db072ec14ec4b039267d1fcc50fccbc/href">https://medium.com/media/1db072ec14ec4b039267d1fcc50fccbc/href</a></iframe><p>Now we can create a sequence of integers using our dictionary word_index Note that we create a fixed number of sequences with length max_tokens which has the largest number of tokens. We use pre padding since it gives better results</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d5636e7075446e12bdfc9cb9bc6c97a7/href">https://medium.com/media/d5636e7075446e12bdfc9cb9bc6c97a7/href</a></iframe><p>Now we are ready to create the keras model. The first layer is the embedding layer. Then we follow it with 3 <a href="https://en.wikipedia.org/wiki/Gated_recurrent_unit">GRU</a> layers. Finally we create a dense layer with sigmoid activation. We compile the model using an Adam optimizer</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb958cadc6ee7b8b15cf31bb6880590f/href">https://medium.com/media/fb958cadc6ee7b8b15cf31bb6880590f/href</a></iframe><p>Then we train the model for 5 epochs with 5% split for validation with 32 batch size</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8dd68296b2f53ebf4c04e07208c043ed/href">https://medium.com/media/8dd68296b2f53ebf4c04e07208c043ed/href</a></iframe><p>We reach 97% accuracy on validation set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*V0l012DyGMDERgdHt7fKWw.png" /></figure><p>We can see the model summary using</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cc2f45b0378e3e88af8881b2f69c68ea/href">https://medium.com/media/cc2f45b0378e3e88af8881b2f69c68ea/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*e3lIBA3Cz33FQYiLrOQ89Q.png" /></figure><p>We will save the model using .h5 format</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/95670742fddc754b411d7025cbda077b/href">https://medium.com/media/95670742fddc754b411d7025cbda077b/href</a></iframe><p>Now that we are done with the model we will import it to run on the browser. First we will need to convert it into json format. Before this step you will need to install tensorflowjs tools using</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/98bba9e8638b6821f00017d0856055df/href">https://medium.com/media/98bba9e8638b6821f00017d0856055df/href</a></iframe><p>Convert the keras model into a model understood by tensorflowjs</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dd813d84515d4971a5ea95a22b554170/href">https://medium.com/media/dd813d84515d4971a5ea95a22b554170/href</a></iframe><p>This will create one json file which contains the meta variables and some other variables with names like group1-shard1of1 which contains the computed values of the weights</p><h4>Porting the model to the browser</h4><p>For simple processing we will load the dictionary generated by the keras code</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cca78bfcf084e4d355b2381f3bee606a/href">https://medium.com/media/cca78bfcf084e4d355b2381f3bee606a/href</a></iframe><p>Hence word_index will now contain the same word,index pair as in the previous section. After that we need to create some helper methods to process the text, tokenize it then map it to integers using the dictionary</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a5cb2ad2121ce673b82888b5e6744e0e/href">https://medium.com/media/a5cb2ad2121ce673b82888b5e6744e0e/href</a></iframe><p>Now we can load that into the browser using tensorflow.js</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08c1a5391308f8e4f3849da8c9002516/href">https://medium.com/media/08c1a5391308f8e4f3849da8c9002516/href</a></iframe><p>Finally we combine all the steps using one method</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/25b84d4bf513bd0b2fd1b458ad7a062f/href">https://medium.com/media/25b84d4bf513bd0b2fd1b458ad7a062f/href</a></iframe><p>Using the sample text we get a prediction value 0.98 which is very close to 1.</p><h4>References</h4><ul><li><a href="https://github.com/Hvass-Labs/TensorFlow-Tutorials">Hvass-Labs/TensorFlow-Tutorials</a></li><li><a href="https://github.com/tensorflow/tfjs">GitHub - tensorflow/tfjs: A WebGL accelerated JavaScript library for training and deploying ML models.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7eda0d87cdc6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Gentle Introduction to TensorFlow.js]]></title>
            <link>https://medium.com/tensorflow/a-gentle-introduction-to-tensorflow-js-dba2e5257702?source=rss-14424af03ecf------2</link>
            <guid isPermaLink="false">https://medium.com/p/dba2e5257702</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflowjs]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Zaid Alyafeai]]></dc:creator>
            <pubDate>Tue, 03 Apr 2018 23:24:26 GMT</pubDate>
            <atom:updated>2018-07-23T12:23:40.802Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*6k9B30y848iV-lTwoH8Faw.png" /></figure><p><a href="https://js.tensorflow.org/">Tensorflow.js</a> is a library built on <a href="https://deeplearnjs.org/">deeplearn.js</a> to create deep learning modules directly on the browser. Using that you can create CNNs, RNNs , etc … on the browser and train these modules using the client’s GPU processing power. Hence, a server GPU is not needed to train the NN. This tutorial starts by explaining the basic building blocks of TensorFlow.js and the operations on them. Then, we describe how to create some complicated models.</p><h4>One Note or Two …</h4><p>If you want to play with the code I created an interactive coding session on <a href="https://beta.observablehq.com/@zaidalyafeai/an-intractive-introduction-to-tensorflow-js">Observable</a>. Also, I created many <a href="https://github.com/zaidalyafeai/zaidalyafeai.github.io">mini-projects</a> including simple classification, style transfer, pose estimation and pix2pix translation.</p><h4>Getting Started</h4><p>Since TensorFlow.js runs on the browser you just need to include the following script to the header of the html file</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5a6a8dfe968e973e18dd25d7fc182c79/href">https://medium.com/media/5a6a8dfe968e973e18dd25d7fc182c79/href</a></iframe><p>This will load the latest published version of the bundle.</p><h4>Tensors (The building blocks)</h4><p>If you are familiar with deep learning platforms like <a href="https://www.tensorflow.org/">TensorFlow</a> you should be able to recognize that tensors are <em>n </em>dimensional arrays that are consumed by operators. Hence they represent the building block for any deep learning application. Let us create a scalar tensor</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b26bfb2ef0d6e2dcdd3aaec21775cd9/href">https://medium.com/media/5b26bfb2ef0d6e2dcdd3aaec21775cd9/href</a></iframe><p>This created a scalar tensor. We also can convert arrays to tensors</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6c1ac34432009ff319e2379c7b574bf8/href">https://medium.com/media/6c1ac34432009ff319e2379c7b574bf8/href</a></iframe><p>This creates a constant tensor of the array [2,2]. In other words we converted the one dimensional array to a tensor by a applying the tensor function. We can use input.shape to retrieve the sizeof the tensor.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5c17a3a152bce564099edb9140976a3d/href">https://medium.com/media/5c17a3a152bce564099edb9140976a3d/href</a></iframe><p>This has the shape [2]. We can also create a tensor with specific size. For instance, here we create a tensor of zeros with shape [2,2].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7bfc0a312bacb1fd72ba0b11d8477f61/href">https://medium.com/media/7bfc0a312bacb1fd72ba0b11d8477f61/href</a></iframe><h4>Operators</h4><p>In order to use tensors we need to create operations on them. Let us say we want to find the square of a tensor</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b6011e81bd898441358d8f039040a68/href">https://medium.com/media/5b6011e81bd898441358d8f039040a68/href</a></iframe><p>The value of x2<em> </em>will be [4,9,16]. TensorFlow.js also allows chaining operations. For example, to evaluate the 2nd power of a tensor we use</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9b5c5f092b1c09cc2967ab61a557d66c/href">https://medium.com/media/9b5c5f092b1c09cc2967ab61a557d66c/href</a></iframe><p>The x2 tensor will have value [1,16,81].</p><h4>Tensor Disposal</h4><p>Usually we generate lots of intermediate tensors. For instance, in the previous example after evaluating x2 we don’t need the value of x. In order to do that we call dispose()</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec0838637d4e2623558b453de4ce4d68/href">https://medium.com/media/ec0838637d4e2623558b453de4ce4d68/href</a></iframe><p>Note that we can no longer use the tensor x<em> </em>in later operations. Now, it might be a little inconvenient to do that for every tensor. Actually, not disposing tensors will be an overhead for the memory. TensorFlow.js offers a special operator tidy() to dispose intermediary tensors automatically</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ffb92ea2637466d39c6035d333ded53f/href">https://medium.com/media/ffb92ea2637466d39c6035d333ded53f/href</a></iframe><p>Notice that the value of the tensor y will be disposed since we don’t need it after we evaluate the value of z.</p><h4>Optimization problem</h4><p>Here we learn how to solve an optimization problem. Given a function <em>f(x)</em> we are asked to evaluate <em>x = a</em> that minimizes <em>f(x)</em>. To do that we will need an optimizer. An optimizer is an algorithm to minimize a function by following the gradient. There are many optimizers in the literature like SGD, Adam, etc… These optimizers differ in their speed and accuracy. Tensorflowjs support the most important <a href="https://js.tensorflow.org/api/0.10.0/#Training-Optimizers">optimizers</a>.</p><p>We will take a simple example were <em>f(x) = x⁶+2x⁴+3x²+x+1.</em> The graph of the function is shown below. We see that the minmum of the function is in the interval [-0.5,0] . We will use an optimizer to find the exact value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/463/1*TaONtvklyBcEk4LKV9w_PQ.png" /><figcaption>graph of the function f(x)</figcaption></figure><p>First we define the function to minimize</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c0284eb97ee821c873032aa15fb86f35/href">https://medium.com/media/c0284eb97ee821c873032aa15fb86f35/href</a></iframe><p>Now we can iteratively minimize the function to find the value of the minimum. We will start by an initial value of <em>a = 2</em>. The learning rate defines how fast we jump to reach the minimum. We will use an Adam optimizer</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0bd7eef74e607c483247f43321e4a64/href">https://medium.com/media/e0bd7eef74e607c483247f43321e4a64/href</a></iframe><p>Using a learning rate with value 0.9 we find the value of the minimum after 200 iterations to be -0.16092407703399658.</p><h4>A Simple Neural Network</h4><p>Now we learn how to create a neural network to learn XOR which is a nonlinear operation. The code is similar to <a href="https://gist.github.com/stewartpark/187895beb89f0a1b3a54">keras</a> implementation. We first create the training set which takes two inputs and one output. We will feed a batch of 4 items in each iteration</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a0be34bf642ab658d16a76ccef55fd59/href">https://medium.com/media/a0be34bf642ab658d16a76ccef55fd59/href</a></iframe><p>Then we create two dense layers with two different nonlinear activation functions. We use stochastic gradient descent with cross entropy loss. The learning rate is 0.1</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a9e26dae82e62397e0dae4d91efc1218/href">https://medium.com/media/a9e26dae82e62397e0dae4d91efc1218/href</a></iframe><p>Then we fit the model for 5000iterations</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e9d4cffddf8b9a2fa7f3726dd5a7c6ee/href">https://medium.com/media/e9d4cffddf8b9a2fa7f3726dd5a7c6ee/href</a></iframe><p>Finally we predict on the training set</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7629d84af20cdbb4d84483916fc02e5a/href">https://medium.com/media/7629d84af20cdbb4d84483916fc02e5a/href</a></iframe><p>the output should be [[0.0064339], [0.9836861], [0.9835356], [0.0208658]] which should be expected.</p><h4>CNN Model</h4><p>TensorFlow.js uses automatic differentiation using <a href="http://www.deepideas.net/deep-learning-from-scratch-i-computational-graphs/">computational graphs</a>. We just need to create the layers, optimizer and compile the model. Let us create a sequential model</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff0d3fd13413239fa6cb1b954ad12379/href">https://medium.com/media/ff0d3fd13413239fa6cb1b954ad12379/href</a></iframe><p>Now we can add different layers for the model. Let us add the first convolutional layer with input [28,28,1]</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f5f58ed0e8807cd644f878d677309565/href">https://medium.com/media/f5f58ed0e8807cd644f878d677309565/href</a></iframe><p>Here we created a conv layer that takes input of size [28,28,1]. The input will be a gray image of size<em> </em>28 x 28. Then we apply 8 kernels of size 5x5 and stride equals to 1 initialized with <a href="https://www.tensorflow.org/api_docs/python/tf/contrib/layers/variance_scaling_initializer">VarianceScaling</a>. After that, we apply an activation function which basically takes the negative values in the tensor and replaces them with zeros. Now we can add this <em>convlayer </em>to the model</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e989e00c10f0ac9a463d33d70f0b966/href">https://medium.com/media/7e989e00c10f0ac9a463d33d70f0b966/href</a></iframe><p>Now what is nice about Tensorflow.js we don’t need to specify the input size for the next layer as it will be evaluated automatically after we compile the model. We can also add max-pooling, dense layers , so on. Here is a simple model</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eded7ea30a1612d0e33f30ff461c5299/href">https://medium.com/media/eded7ea30a1612d0e33f30ff461c5299/href</a></iframe><p>We can apply a tensor for any layer to inspect the output tensor. But here is a catch the input needs to be of shape [BATCH_SIZE,28,28,1] where BATCH_SIZE represents the number of dataset elements we apply to the model at a time. Here is an example of how to evaluate a convolutional layer</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9774832b12b21ffa190da9a719632afb/href">https://medium.com/media/9774832b12b21ffa190da9a719632afb/href</a></iframe><p>After inspecting the shape of the output tensor we see it has shape[1,24,24,8]. This is evaluated using the formula</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/932391f54d4cf3802747f3d299b065be/href">https://medium.com/media/932391f54d4cf3802747f3d299b065be/href</a></iframe><p>Which will result in 24 in our case. Returning to our model we realize that we used flatten() which basically convert the input from the shape [BATCH_SIZE,a,b,c] to the shape [BATCH_SIZE,axbxc]. This is important because in the dense layers we cannot apply 2d arrays. Finally, we used the dense layer with output units 10<em> </em>which represents the number of classes we need in our recognition system. Actually this model is used for the recognizing hand-written digits in the so called <a href="http://yann.lecun.com/exdb/mnist/">MNIST </a>dataset.</p><h4>Optimization and Compilation</h4><p>After creating the model we need a way to optimize the parameters. There are different approaches to do that like <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent">SGD</a> and <a href="https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/">Adam</a> optimizer. For instance, we can create an optimizer using</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d361308fcf9dcba5ad89ad28ae7ce73/href">https://medium.com/media/5d361308fcf9dcba5ad89ad28ae7ce73/href</a></iframe><p>This will create an Adam optimizer using the specified learning rate. Now, we are ready to compile the model (attaching the model with the optimizer)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b9478b92d52c238e9d543e9300a35247/href">https://medium.com/media/b9478b92d52c238e9d543e9300a35247/href</a></iframe><p>Here we created model that uses Adam to optimize the loss function that evaluates a <a href="https://en.wikipedia.org/wiki/Cross_entropy">cross entropy</a> of the predicted output and the true label.</p><h4>Training</h4><p>After compiling the model we are ready to train the model on a dataset. We need to use the fit() function for that</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0bdd70ea2e502736ea029d9a17a68a09/href">https://medium.com/media/0bdd70ea2e502736ea029d9a17a68a09/href</a></iframe><p>Note that we are feeding to the fit function a batch of training set. The second variable for the fit function represents the true labels of the model. Lastly, we have the configuration parameters like the batchSize and epochs. Note that epochs represents how many times we iterate over the current batch NOT the whole dataset. Hence we can for example wrap that code inside a for<em> </em>loop that iterates over all the batches of the training set.</p><p>Note that we used the special keyword await which basically blocks and waits for the function to finish executing the code. It is like running a another thread and the main thread is waiting for the fitting function to finish execution.</p><h4>One Hot Encoding</h4><p>Usually the given labels are numbers which represents the class. For instance, suppose we have two classes an orange class and an apple class. Then we will give the orange class label 0 and the apple class label 1. But, our network accepts a tensor of size [BATCH_SIZE,NUM_CLASSES]. Hence we need to use what we call one hot encoding</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c239b6fc2855ac2b9e7af3c54a93cd0e/href">https://medium.com/media/c239b6fc2855ac2b9e7af3c54a93cd0e/href</a></iframe><p>Hence we converted the 1d tensor off labels into a tensor of shape [BATCH_SIZE,NUM_CLASSES].</p><h4>Loss and Accuracy</h4><p>In order to inspect the performance of our model we need to know the loss and the accuracy. In order to do that we need to fetch the results of the model using the <em>history </em>module</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/94fa727f04da749d6fa4b4059b5b197e/href">https://medium.com/media/94fa727f04da749d6fa4b4059b5b197e/href</a></iframe><p>Note that we are evaluating the loss and accuracy of the validationData that was an input to the fit() function.</p><h4>Prediction</h4><p>Suppose that we are done with training a model and it gives good loss and accuracy. It is time to predict the results of unseen data element. Suppose we are given an image that is in our browser or we took directly from our webcam, then we can use our trained model to predict its class. First, we need to convert the image into tensor</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/adf4f4396ab5564306e8244c55ed2a3f/href">https://medium.com/media/adf4f4396ab5564306e8244c55ed2a3f/href</a></iframe><p>Here we created a canvas<em> </em>and retrieved imageData<em> </em>from it and then we converted to a tensor. Now the tensor will have size [28,28,3]<em> </em>but the model takes 4-dimensional vectors. Hence we need to add an extra dimension for the tensor using expandDims</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a782b62f1b1ded0eaa148edf916b934/href">https://medium.com/media/7a782b62f1b1ded0eaa148edf916b934/href</a></iframe><p>Hence the output tensor will have size [1,28,28,3]<em> </em>since we have added a dimension at index 0. Now for prediction we simply use predict()</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cb060b021e6e4c2440e940364d8ba071/href">https://medium.com/media/cb060b021e6e4c2440e940364d8ba071/href</a></iframe><p>The function predict will return the value of the last layer in our network usually a softmax activation function.</p><h4>Transfer Learning</h4><p>In the previous sections we had to train our model from scratch. However, this is an expensive operation since it requires more training iterations. Hence, we use a pretrained model called mobilenet. It is a light-wight CNN that is optimized to run in mobile applications. Mobilenet was trained on ImageNet classes. Basically we have precomputed activations trained on 1,000 different <a href="https://github.com/tensorflow/tfjs-examples/blob/master/mobilenet/imagenet_classes.js">classes</a>.</p><p>To load the model we use the following</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f521d8773c71a0051588df49c1559f04/href">https://medium.com/media/f521d8773c71a0051588df49c1559f04/href</a></iframe><p>We can use inputs , outputs to inspect the structure of the model</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7724aef4476fe366d3e19e0d8c1f592c/href">https://medium.com/media/7724aef4476fe366d3e19e0d8c1f592c/href</a></iframe><p>Hence we need images of size [1,224,224,3] and the output will be a tensor of size [1,1000] which holds the probability of each class in the ImageNet dataset.</p><p>For the sake of simplicity we will take an array of zeros and try to predict the class number out of 1,000 classes</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f9f19bae110205b44066639a997b91de/href">https://medium.com/media/f9f19bae110205b44066639a997b91de/href</a></iframe><p>After running the code I get class = 21 which represents a kite o:</p><p>Now we need to inspect the contents of the model. To do that we can get the models layers and names</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ef0da9111b17c6036336d77e7db2b025/href">https://medium.com/media/ef0da9111b17c6036336d77e7db2b025/href</a></iframe><p>We see that we have 88 layers which would be so expensive to train again on another dataset. Hence, the basic trick is to use this model just to evaluate the activations (we will not retrain) but we will create dense layers that we can train on another number of classes.</p><p>For instance, suppose we need a model to differentiate between carrots and cucumbers. We will use mobilene tmodel to calculate the activation up to some layer we choose. Then we use dense layers with output size 2 to predict the correct class. Hence the mobilenet model will be in some sense ‘freezed’ and we just train the dense layers.</p><p>First, we need to get rid of the dense layers of the model. We choose to extract a random layer let us say number 81 with name conv_pw_13_relu</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2fc7319c0a520264b450ee38bddf2197/href">https://medium.com/media/2fc7319c0a520264b450ee38bddf2197/href</a></iframe><p>Now let us update our model to have this layer is an output</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/16f23958f41ad63d9fbaaff2c0ea9518/href">https://medium.com/media/16f23958f41ad63d9fbaaff2c0ea9518/href</a></iframe><p>Finally, we create the trainable model but we need to know the last layer output shape</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd391e9819c98d7f26876c97d5523c53/href">https://medium.com/media/bd391e9819c98d7f26876c97d5523c53/href</a></iframe><p>We see that the shape [null,7,7,256] Now we can input this to our dense layers</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d65d15887800d9d1acece49d182be88c/href">https://medium.com/media/d65d15887800d9d1acece49d182be88c/href</a></iframe><p>As you can see we created a dense layer with 100 neurons and the output layer with size 2.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae8b57e9ef5dec08050eed49b3d74e3a/href">https://medium.com/media/ae8b57e9ef5dec08050eed49b3d74e3a/href</a></iframe><p>And we can use the previous sections to train the last model using a certain optimizer.</p><h4>References</h4><ul><li><a href="https://js.tensorflow.org/">https://js.tensorflow.org/</a></li><li><a href="https://github.com/tensorflow/tfjs-examples">https://github.com/tensorflow/tfjs-examples</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dba2e5257702" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/a-gentle-introduction-to-tensorflow-js-dba2e5257702">A Gentle Introduction to TensorFlow.js</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>