<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kyle Zeller on Medium]]></title>
        <description><![CDATA[Stories by Kyle Zeller on Medium]]></description>
        <link>https://medium.com/@kzeller_133?source=rss-8905f49af185------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Kyle Zeller on Medium</title>
            <link>https://medium.com/@kzeller_133?source=rss-8905f49af185------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 06:54:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kzeller_133/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Create a Convolutional Neural Network with TensorFlow]]></title>
            <link>https://medium.com/@kzeller_133/a-convolutional-neural-network-implementation-with-tensorflow-ad23b8cc0691?source=rss-8905f49af185------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad23b8cc0691</guid>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[tensorboard]]></category>
            <category><![CDATA[convolution-neural-net]]></category>
            <dc:creator><![CDATA[Kyle Zeller]]></dc:creator>
            <pubDate>Tue, 04 Sep 2018 05:48:02 GMT</pubDate>
            <atom:updated>2018-09-05T03:44:00.814Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*HEKyqR3AiyaSdPkxiRLeuA.png" /><figcaption>Sample from MNST Dataset (Credit: <a href="https://www.researchgate.net/figure/A-random-sample-of-15-handwritten-digits-from-the-MNIST-data-set_fig2_282924675">https://www.researchgate.net/figure/A-random-sample-of-15-handwritten-digits-from-the-MNIST-data-set_fig2_282924675</a>)</figcaption></figure><p>This portion of the blog requires a bit of prior knowledge on what neural networks are and how they function. Here’s two links (<a href="https://medium.freecodecamp.org/building-a-3-layer-neural-network-from-scratch-99239c4af5d3">1</a> &amp; <a href="https://towardsdatascience.com/how-to-build-your-own-neural-network-from-scratch-in-python-68998a08e4f6">2</a>) to a relatively comprehensive introduction to neural networks.</p><h4>Brief Introduction to Convolutional Neural Networks</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EPpYI-llkbtwHgfprtTJzw.png" /><figcaption>ConvNet (Credit: <a href="https://camo.githubusercontent.com/269e3903f62eb2c4d13ac4c9ab979510010f8968/68747470733a2f2f7261772e6769746875622e636f6d2f746176677265656e2f6c616e647573655f636c617373696669636174696f6e2f6d61737465722f66696c652f636e6e2e706e673f7261773d74727565">https://camo.githubusercontent.com/269e3903f62eb2c4d13ac4c9ab979510010f8968/68747470733a2f2f7261772e6769746875622e636f6d2f746176677265656e2f6c616e647573655f636c617373696669636174696f6e2f6d61737465722f66696c652f636e6e2e706e673f7261773d74727565</a>)</figcaption></figure><p>Convolutional Neural Networks (CNNs) take a bit of inspiration from biological processes, primarily that of the visual cortex in the brain. A 1962 <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/">experiment</a> led by Hubel and Wiesel, led to the discovery that certain individual neurons fired only in the presence of edges of a particular orientation and location given a visual stimulus. The experiment also entails the neurons having been layered in a columnar structure attributing to visual perception.</p><h3>ConvNet Structure</h3><p>In short, CNNs can be described as having a convolutional, pooling, and fully-connected layer. Each CNN layer transforms a 3D input volume to a 3D output volume of activations and the neurons in a CNN are arranged as (width, height, depth).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/1*1F8wom5X1DeCj5BWeGfJbw.jpeg" /><figcaption>Neuron arrangement into three dimensions “depth, height, &amp; depth” (Credit: <a href="http://cs231n.github.io/assets/cnn/cnn.jpeg">http://cs231n.github.io/assets/cnn/cnn.jpeg</a>)</figcaption></figure><h4>Convolutional Layers</h4><p>Each convolutional layer has a number of filters, with their own stride and padding, consisting of learnable filters to produce a 2D activation map for filters at every spatial position.</p><p>As a hyperparameter, depth represents the <strong>number of filters</strong> to use, each of which are trying to learn something different within the input. E.g. edges, color blobs, etc. The <strong>stride</strong> of a filter, represents by how much to move the filter across the input at a time (usually 1 or 2). The size of the <strong>zero-padding</strong>, “<em>same</em>” padding will bound the outer edges of the convolved image with 0’s in order to keep the dimensions of the image the same. Meanwhile, “<em>valid</em>” padding will not bound the convolved image with 0’s and thus reduces the dimensionality of the features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/364/1*Fr6Umze2waDjWVHB2yzT4A.png" /><figcaption>Convolution Diagram (Credit: <a href="http://intellabs.github.io/RiverTrail/tutorial/">http://intellabs.github.io/RiverTrail/tutorial/</a>)</figcaption></figure><p>To calculate the spatial size of the output volume, consider the following:</p><ul><li>Let W be the input volume size</li><li>Let F be the receptive field size of the Conv Layer neurons</li><li>Let P be the amount of zero padding</li><li>Let S be the stride of the filter</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/174/1*kGPXiDTez9Wl5v_WZ4AUKw.png" /></figure><h4>Pooling Layers</h4><p>The main function of the pooling layer is to reduce the number of features, while still retaining relevant information that might have been discarded during the dimensionality reduction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*GksqN5XY8HPpIddm5wzm7A.jpeg" /><figcaption>Max Pooling Example (Credit: <a href="http://cs231n.github.io/convolutional-networks/#overview">http://cs231n.github.io/convolutional-networks/#overview</a>)</figcaption></figure><h4>Fully Connected (Dense) Layer</h4><p>The fully connected layer is used to associate features to a single class from a set of possible classes. As shown below, the preceding activation layer is fully connected to each neuron present in this layer and the output is a probability distribution for the likelihood of each class being correct.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*f2WE-Z7M25bXIebkyp6ZgA.png" /><figcaption>Fully Connected Neural Network (Credit: <a href="https://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Colored_neural_network.svg/1200px-Colored_neural_network.svg.png">https://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Colored_neural_network.svg/1200px-Colored_neural_network.svg.png</a>)</figcaption></figure><p>For a much more in depth tutorial on ConvNets, please refer to this <a href="http://cs231n.github.io/convolutional-networks/#overview">link</a>.</p><h3>Implementation</h3><p>This implementation of a ConvNet is for classifying data in the MIST dataset, which is comprised of gray-scale 28 x 28 pixel images of written digits [0–9].</p><p>More information on the dataset can be found <a href="http://yann.lecun.com/exdb/mnist/">here</a>.</p><h4>The Input Pipeline</h4><ol><li>Extract</li><li>Transform</li><li>Load</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d26bbd3d1764da1735c0cbbe20c49fb/href">https://medium.com/media/6d26bbd3d1764da1735c0cbbe20c49fb/href</a></iframe><h4>The Model Architecture</h4><p><strong>Conv Layer 1</strong>:</p><ul><li>32 filters</li><li>Kernel of size 5 x 5 pixels</li><li>Stride of 1 pixel</li><li>“Same” padding</li><li>ReLU activation</li></ul><p><strong>Pooling Layer 1</strong>:</p><ul><li>Max Pool Operation</li><li>Kernel of size 2 x 2 pixels</li><li>Stride of 2 pixels</li></ul><p><strong>Conv Layer 2</strong> is the same as Conv Layer 1, except for there now being twice as many filters as before.</p><p><strong>Pooling Layer 2</strong>, will have the exact same parameters as Pooling Layer 1.</p><p><strong>Fully Connected Layer</strong>:</p><ul><li>Reshape and flatten the output from the previous layer</li><li>Set the number of neurons desired and activation type</li><li>Use dropout if needed, to prevent overfitting</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f9a5a4b5d22b420e234c2a8668bf93b/href">https://medium.com/media/6f9a5a4b5d22b420e234c2a8668bf93b/href</a></iframe><h4>Model Evaluation</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/71e88e485addc938da9237dc8c102981/href">https://medium.com/media/71e88e485addc938da9237dc8c102981/href</a></iframe><p>The full code for this part can be found <a href="https://gist.github.com/ECE-Engineer/74984583156e5cd6a0e3df575cd76358">here</a>.</p><h3>Running the Code Step by step</h3><p>In setting up the code on your machine, I’ve provided a requirements document from which you can simply type the code in to the terminal. (<strong>Window’s users should use the URL provided in a browser for the python installation</strong>)</p><p>The code below covers the following:</p><ul><li>Installing python2 or python3</li><li>Installing pip or pip3</li><li>Installing virtualenv</li><li>Installing all the needed dependencies within the “activated” virtual environment</li><li>Running the code and deactivating the virtual environment</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d2cc8603ab4903142c191351c5838f1f/href">https://medium.com/media/d2cc8603ab4903142c191351c5838f1f/href</a></iframe><h3>Adding Extra Features:</h3><ul><li><a href="https://gist.github.com/ECE-Engineer/4be7e4e202bf9fd5858972fec119c514">Saving the Model for later use</a></li><li><a href="https://gist.github.com/ECE-Engineer/e3a6b07c5c5d7f0846dfabe7877feb2f">Adding Summaries</a></li><li><a href="https://gist.github.com/ECE-Engineer/dbf317e175f3a988e3440fe23e8b1bfa">Adding Regularization</a></li><li><a href="https://gist.github.com/ECE-Engineer/c1d8a2f737a160aa7b0250b86e8d1ac2">Confusion Matrix</a></li><li><a href="https://gist.github.com/ECE-Engineer/fba661b1a8b45b1193942c61e43bd287">Model Loading</a></li></ul><p>The full code for this “second” part can be found <a href="https://gist.github.com/ECE-Engineer/f8ab70ffcf3cf5e9b95e23f281e5e18a">here</a>. (<strong>Please note that the part below cannot be done without saving the summaries of your model</strong>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*PTdF2U-t5cvbdVpQQFrD0g.png" /></figure><h3>Visualizing The Results w/ TensorBoard</h3><ul><li>Reactivate the virtual environment</li><li>Run the following in the terminal:</li></ul><pre>tensorboard --logdir=path/to/log-directory</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sJy256inA4Z8cQkodvac9g.jpeg" /><figcaption>Accuracy (Orange: Training Data, Blue: Testing Data)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FOY7t9jrkqlAnfHIbeFHfg.jpeg" /><figcaption>Loss (Orange: Training Data, Blue: Testing Data)</figcaption></figure><p>The repository for the code can be found <a href="https://github.com/ECE-Engineer/CNN-TensorFlow-Tutorial">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad23b8cc0691" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>