<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aegeus Zerium on Medium]]></title>
        <description><![CDATA[Stories by Aegeus Zerium on Medium]]></description>
        <link>https://medium.com/@eternalzer0dayx?source=rss-d54b574e8271------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*nY6u7CP1ji1bLBRI_skdSQ.jpeg</url>
            <title>Stories by Aegeus Zerium on Medium</title>
            <link>https://medium.com/@eternalzer0dayx?source=rss-d54b574e8271------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 11:46:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@eternalzer0dayx/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Does Bitcoin follow stock market movements ?]]></title>
            <link>https://medium.com/@eternalzer0dayx/does-bitcoin-follow-stock-market-movements-929c87ba09ce?source=rss-d54b574e8271------2</link>
            <guid isPermaLink="false">https://medium.com/p/929c87ba09ce</guid>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[bitcoin]]></category>
            <dc:creator><![CDATA[Aegeus Zerium]]></dc:creator>
            <pubDate>Fri, 04 Jan 2019 19:03:45 GMT</pubDate>
            <atom:updated>2019-01-05T03:39:37.617Z</atom:updated>
            <content:encoded><![CDATA[<h3>Does Bitcoin follow stock market movements ?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9t_WiiWccrOG0EDMiqMK8Q.jpeg" /></figure><p>One of the most pressing questions among the laymen and experts alike was and remains: How would digital currencies respond to a stock market crash ? will the value of digital currencies hit rock bottom too or will they thrive despite the chaos ?<br>With the Federal Reserve’s latest decision to hike up interest rates many investors and financial analysts have been wondering whether it’s worth taking the risk of investing all of your money in one basket namely the stock market especially when the latter’s fluctuations is heavily tied up to governmental monetary policies, if you were to take a glimpse into the cryptocurrency side of social media and Twitter especially you will stumble upon vast amount of tweets encouraging private and public investors to pour money onto the cryptocurrency market specifically Bitcoin, the digital coin that popularized the blockchain technology, in the 19th December the “Buy Bitcoin” hashtag was all over Twitter, the aim of the hashtag was clear marketing Bitcoin as stock market and government independent “asset”, an “asset” that doesn’t follow the whims and desires of governmental policies nor stock market movements. <br>This article is a brief statistical analysis of “Bitcoiners” claim, is Bitcoin truly independent from stock market movements or is the latter’s gravitational pull too strong to be avoided.</p><p>Before diving into our statistical analysis let’s first mention some core facts about bitcoin that the reader must know about:</p><ul><li>Unlike the stock market, the cryptocurrency market is open 24/7 no weekends nor holidays disturb its opening nor force it to close.</li><li>Cryptocurrencies aren’t backed neither by the government nor the central bank.</li><li>In terms of legality, Bitcoin and cryptocurrencies in general are legal in some parts of the world (The United States, European Union …) , restricted in others (India …) or completely banned (Egypt, Morocco, Algeria …).</li><li>Bitcoin’s supply is finite (21 million Bitcoins in total).</li><li>Bitcoin mining is energy consuming, it was ranked as 71st among the highest power usage worldwide.</li><li>There is no unanimously agreed upon financial term that accurately describes Bitcoin, some consider it an asset (the G-20), others consider it a currency etc …</li></ul><p>Some brief facts about our statistical analysis:</p><ul><li>We compare price fluctuations of Bitcoin and Apple stocks to the S&amp;P 500 market index.</li><li>We used the historical data of the last 3 years (from 30/12/2015 to 2018/12/28).</li><li>The fact that the cryptocurrency market is open 24/7 means that the latter will have more historical prices than Apple stocks and S&amp;P 500 market index thus in order to solve this inequality we only used the historical prices that were recorded on the same dates during the three-year period.</li><li>We used Python as a programming language, NumPy for scientific computing, Pandas for data wrangling and Matplotlib for visualization.</li></ul><h3>Financial time series: A technical analysis</h3><p>Using financial time series we will analyze Bitcoin, Apple stocks and S&amp;P 500 market index charts, find patterns and trends and draw conclusions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*3U24zt_lMrPSRZXKGV5vdQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*UDF1AklqOs0zuCLr2dCV0Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/1*AsqzJ41xbVMMzpB2MQmuFA.png" /></figure><p>From the first sight we can notice clear similarities between the Apple chart and the S&amp;P 500 market index chart.</p><p>In terms of noise and trends:</p><p>Both Apple stocks and S&amp;P 500 are characterized by a general (long term) upward trend followed by a bearish looking downward trend, When it comes to the Bitcoin chart there isn’t much similarities between the latter and the S&amp;P 500 chart.</p><p>The Bitcoin chart can be characterized by its stability in the first third followed by sky rocketing prices in the second third and a long term downward trend in the last third.</p><p>Using financial time series alone we could differentiate between the similarities and non similarities between Bitcoin, Apple stocks and the S&amp;P 500 market index.</p><h3>Risk and Return trade off</h3><p>Not only are the returns of an asset/market widely used because of their assumed normality, but also because of the valuable informations they give concerning the asset/market’s performance and the risk associated with it, thus calculating the returns of Bitcoin, Apple stocks and the S&amp;P 500 market index will give us a clear idea about each one’s performance and how similar they are in terms of risk and reward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*EO_vglkrlZzNEkO2wBGAQA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*DqEwk_ekK0ELuiz--k8IiA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/1*OGyBqbQu33LO9wbpv7PlNw.png" /></figure><p>We can easily notice the extreme noise in all of the figures above, with the difference that the return interval on the y-axis of Apple and S&amp;P 500 are virtually the same (Apple’s return interval is slightly bigger) while Bitcoin has a clearly wider return interval, the return on Bitcoin is bigger so is the risk associated with it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/429/1*UHLB0pNZbivQQpLcD5EU2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/459/1*cDHRSAt7Z6y8M2c0k0kH0A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/446/1*fG_GPfPYD0ooCsKkqxW8JA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*7otbAXTvhYS4n7u04Y9AqQ.png" /></figure><p>The probability density function gives us an even clearer look at the distributivity of Bitcoin, Apple stocks and S&amp;P 500 returns, the Bitcoin probability distribution is fat tailed which is not a surprise for an “asset” notoriously known for its extreme volatility (<em>σ</em> = 4.84 %), while the probability distributions of Apple and the S&amp;P 500 are the total opposite thanks to their relatively low standard deviation.</p><p>We can’t possibly talk about market dynamics and the systematic risk associated with it without mentioning the Beta financial metric, it’s by far one of the most important parameters in finance, for those who don’t know Beta is used to measure the volatility, or sensitivity, of a security or a portfolio in comparison to the entire market or a benchmark (S&amp;P 500 in our example), a Beta closer or equal to 1 means that the security is as volatile as the market it more often moves up as the market index moves up and goes down as the market goes down, a Beta exceeding 1 means that the security is more volatile than the market in a bull market the security would outperform the latter while in a bear market the security will suffer heavier losses, a Beta closer or equal to -1 means that the security more often moves up as the market index goes down and goes down as the market index moves up, a zero Beta or Beta closer to zero means that the security more often doesn’t respond to market swings.</p><p>Upon calculating the Beta of Apple stocks and Bitcoin we get the following results:</p><p><em>β Apple </em>= 1.18 and <em>β Bitcoin</em> = 0.21, the two values indicate that Apple stocks swing a little bit more than the S&amp;P 500 while Bitcoin as the value of its Beta indicates is less responsive to the market movements.</p><p>While Beta shows how strongly one asset (or portfolio) responds to the systemic risk of the entire market, the correlation coefficient quantifies the intensity of the linear relationship between two (or more) variables X (Bitcoin and Apple stocks) and Y (S&amp;P 500 market index), a perfectly positive correlation (r =1) means that the two variables have a perfect linear relationship, a perfectly negative correlation (r = -1) means that the two variable are inversely correlated, while a correlation of zero (r = 0) means there is no inherent relationship between the variables X and Y.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/1*KahAZf7wD0gzneWdJnu_fQ.png" /><figcaption>Correlation Matrix</figcaption></figure><p>The correlation matrix above shows the relationship degree between Bitcoin, Apple stocks and S&amp;P 500, with Corr(Apple, S&amp;P 500) equal to 0.65 indicating a fairly strong (but not perfect) relationship between the S&amp;P 500 and Apple stocks fluctuations, while both Corr(Bitcoin, Apple) and Corr(Bitcoin, S&amp;P 500) being near zero implying an extremely weak relationship between Bitcoin and S&amp;P 500 market index movements.</p><p>The core conclusions that we can draw from this statistical analysis are as follows:</p><ul><li>Bitcoin (unlike Apple stocks) is not responsive to market movements.</li><li>Bitcoin is more volatile than the market thus riskier.</li><li>An extremely weak relationship (if not non-existent) between Bitcoin and market movements.</li></ul><h3>Can Bitcoin be used for hedging ?</h3><p>From the conclusions we have drawn you might legitimately ask whether Bitcoin can be used to hedge the systematic risk associated with the market, the answer would’ve been a straight “yes” if Bitcoin wasn’t an extremely volatile “asset” but it all depends on the investor’s mindset and how ready he is to take on the inherent risk of the cryptocurrency market.</p><h3>For contacts:</h3><p>Twitter: <a href="https://twitter.com/AegeusZerium">@<strong>AegeusZerium</strong></a></p><p>Gmail: <a href="/service/mailto:eternalzer0dayx@gmail.com">eternalzer0dayx@gmail.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=929c87ba09ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying Convolutional Neural Networks]]></title>
            <link>https://medium.com/@eternalzer0dayx/demystifying-convolutional-neural-networks-ca17bdc75559?source=rss-d54b574e8271------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca17bdc75559</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Aegeus Zerium]]></dc:creator>
            <pubDate>Sun, 02 Sep 2018 14:53:19 GMT</pubDate>
            <atom:updated>2018-09-02T14:56:50.498Z</atom:updated>
            <content:encoded><![CDATA[<p>An Intuitive Explanation of Convolutional Neural Networks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/0*ojR1NZocRlfxX7bN" /></figure><h3>Definition:</h3><p>Simply put, a <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network"><strong>Convolutional Neural Network</strong></a> is a Deep learning model or a multilayered percepteron similar to Artificial Neural Networks which is most commonly applied to analyzing visual imagery. The founding father of Convolutional Neural Networks is the well known computer scientist working in Facebook <a href="https://en.wikipedia.org/wiki/Yann_LeCun"><strong>Yann LeCun</strong></a> who was the first one to use them to solve the hand written digits problem using the famous <a href="http://yann.lecun.com/exdb/mnist/"><strong>MNIST</strong></a> Dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*VcXyWD1eMk--huEP.jpg" /><figcaption>Yann LeCunn</figcaption></figure><p>Convolutional Neural Networks were inspired by biological processes in that the connectivity pattern between neurons resembles the organization of the animal visual cortex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/0*0vxy0MWQX_3kzcHX.png" /><figcaption>Visual structure of the human brain</figcaption></figure><p>Individual cortical neurons respond to stimuli only in a restricted region of the visual field known as the receptive field. The receptive fields of different neurons partially overlap such that they cover the entire visual field.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*4V0cYpmKNE5SPdVz.jpg" /><figcaption>Computer vision vs Human vision</figcaption></figure><p>As you can see we can’t possibly talk about any type of Neural Networks without mentioning a little bit of neuroscience and how the human body (especially the brain) and its functions have been the primary inspiration for the creation of various Deep learning models.</p><h3>The architecture of ConvNets:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*hUZjFgUOUBXMBdxw.jpg" /><figcaption>ConvNet architecture</figcaption></figure><p>As you can see in the illustration above a ConvNet architecture is very similar to the regular ANN architecture especially in the last layers of the network namely the Fully connected layers area, you will also notice that a ConvNet accepts a volume as an input instead of a vector.</p><p>let’s now explore the layers that constitute a ConvNet and the mathematical operations that the latter goes through to visualize and classify pictures based on the features and attributes it has learnt during the training process.</p><h3>Input Layer:</h3><p>The input layer is mostly an n × m × 3 RGB (short for Red, Green and Blue) image(s) unlike an Artificial Neural Networks which gets fed with a n × 1 vector, nothing hard to grasp here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/598/0*NI447zpXAZE5KT1X.png" /><figcaption>RGB image</figcaption></figure><h3>Convolution Layer:</h3><p>In the <strong>Convolution layer</strong> we compute the output of the dot product between an area of the input image(s) and a weight matrice called a <strong>filter</strong>, the filter will slide through out the whole image repeating the same dot product operation. Two things that should be mentioned:</p><ul><li>The filter must have the same number of channels as the input image.</li><li>it’s commonly known that the deeper you go into the Network the more filters you use the intuition behind it is that the more filters we have the more edge and feature detection you’ll get</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*5SpP-dbwdZvw5knRDlkIeg.gif" /><figcaption>Forward Convolution Operation</figcaption></figure><p>We calculate the dimensions of the output of the Convolution layer:</p><p><strong><em>Output Width:</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/171/1*1gBf5HUCd877BPXuj8puig.png" /></figure><p><strong><em>Output Height:</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/132/1*koILtHpnLxzRi-vCIer09w.png" /></figure><p>where:</p><ul><li><em>W </em>: the width of the input image</li><li><em>H </em>: the height of the input image</li><li><em>Fw</em> : the width of the filter or kernel</li><li><em>Fh </em>: the height of the filter</li><li><em>P </em>: padding</li><li><em>S </em>: stride</li></ul><p>The number of channels of the Convolution layer output equals to the <strong>number of filters</strong> used during the convolution operation.</p><h4>Why Convolutions ?</h4><p>You are probably asking yourself, why do we use convolutions in the first place ? why not flatten the input images from the beginning ? well if we do that we will end up with a massive number of parameters that need to be trained and most of us don’t have the computational power that will solve our computationally expensive task in the fastest way possible. In addition, with the fewer parameters that the ConvNets have we can avoid overfitting.</p><h3>Pooling Layer:</h3><p>There are two widely used types of pooling, average pooling and max pooling where the latter being the most used of the two. The pooling layer is used to reduce the spatial dimensions, but not depth, on a convolutional neural network. When using the max pooling layer we take the highest number (the most responsive area in the image) of the input’s area (an n × m matrice), whereas when we use the average pooling layer we take the mean of the input area instead.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/312/0*-q55Kruj2uQcA9lu.gif" /><figcaption>Max Pooling</figcaption></figure><h4>Why Pooling ?</h4><p>One of the core goals of the pooling layer (max pooling in this case) is to provide spatial variance, which simply means that you or the machine will be capable of recognizing an object as an object even when its appearance varies in some way. for more in depth explanation of the pooling layer check this rigourous <a href="http://yann.lecun.org/exdb/publis/pdf/boureau-icml-10.pdf"><strong>paper</strong></a> by Yann LeCunn.</p><h3>Non-linearity Layer:</h3><p>In the Non-linearity layer we use the ReLU activation function most if not all the time instead of the Sigmoid or Tan-H activation function. The ReLU activation function returns 0 for every negative value in the input image while it returns the same value for every positive value in the input image (for more in depth explanation of activation functions please check this <a href="https://blog.goodaudience.com/artificial-neural-networks-explained-436fcf36e75"><strong>article</strong></a> of mine).</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*6HyqifN4M_bJ7DTJ0RFRJA.jpeg" /><figcaption>ReLU activation function</figcaption></figure><h3>Fully Connected Layer:</h3><p>In the FC layer we flatten the output of the last Convolution layer and connect every node of the current layer with the other node of the next layer, Fully connected layer is just another word for the regular Artificial neural network as you will see in the image below. The operations in the fully connected layer are exactly the same as in any artificial neural network:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/176/1*lGaY3cig02OSbuO1S_4Nhg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Zd5ScCO-meZl9yrCw6ZC0Q.jpeg" /><figcaption>Flattening of the Convolution layer</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/0*BinB4K8AxFwMKDbp" /><figcaption>Fully connected layer</figcaption></figure><p>The layers and operations discussed above are the core components of every Convolutional neural network.</p><p>Now that we’ve discussed the operations that a ConvNet goes through in a forward pass let’s jump to the operations that a ConvNet goes through in a backward pass.</p><h3>Backpropagation:</h3><h3>Fully Connected Layer:</h3><p>in the Fully connected layer backpropagation works exactly the same as in any regular artificial neural network, in backpropagation (using gradient descent as an optimization algorithm) we use partial derivatives namely the derivative of the loss function with regard to the weights, in order to calculate the latter we use a well known operation in calculus called <strong>The Chain rule</strong> where we multiply (in the backpropagation context) the derivative of the loss function w.r.t the activated output with the derivative of the activated output w.r.t the non-activated output with the derivative of the non-activated output w.r.t to the weights.</p><p>I was going to write in front of each partial derivative its mathematical notation but unfortunately Medium does not natively support writing mathematical expressions which is quite infuriating to be honest, anyway here is how we can summarize what we’ve just said in plaintext above with this mathematical expression below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*Kym9BnY0OFctBcaAXbnioQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/0*N6Z2x-IllT9IsJWr" /><figcaption>Backpropagation illustration</figcaption></figure><p>after calculating the gradient we substract it from the initial weights to get newly optimized ones:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/148/1*vIor1lKMwyxOVmF5QD15rw.png" /></figure><p>where:</p><ul><li><em>θi+1</em> : optimized weights</li><li><em>θi </em>: initial weights</li><li><em>α</em> : learning rate</li><li><em>∇J(θi)</em> : gradient of the loss function</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*rBQI7uBhBKE8KT-X.png" /><figcaption>Gradient descent</figcaption></figure><p>In the animation below, gradient descent is applied to linear regression, you can clearly see that the more the cost function gets minimized the better the linear model fits the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/0*Dv_Mp6ZYKWLA1DAh.gif" /><figcaption>Gradient descent applied to linear regression</figcaption></figure><p>note that you should be careful with choosing the value of the learning rate, a very high learning rate could cause the gradient to overshoot the target minimum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*QwE8M4MupSdqA3M4.png" /><figcaption>Small learning rate vs Big learning rate</figcaption></figure><p>In all optimization tasks ,whether in physics, economics or Computer science, partial derivatives are overwhelmingly used, partial derivatives are primarily used to calculate the rate of change of a dependent variable <em>f(x,y,z)</em> with regard to one of its independent variables while the rest of the variables remain constant. for example imagine you own a share of a company, the stocks of the latter will go up or down based on multiple factors (security, politics, sales revenue etc …), to implement partial derivatives on your situation you would calculate how much the stock price of your company change if security (for example) get affected while others factors remain constant and repeat the same process with each and every other factor.</p><h3>Pooling Layer:</h3><p>In the Max Pooling layer the gradient gets backpropagated through the maximum values only since changing them slightly won’t affect the output. In the process we replace the maximum values before max pooling with 1 and set all the non maximum values to zero then use the <strong>Chain rule</strong> to multiply the gradient by them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/0*jDAl5F-y3qzvcObd.png" /><figcaption>Backpropagation through the Pooling layer</figcaption></figure><p>Unlike the max pooling layer, in the average pooling layer the gradient passes through all the inputs (before average pooling) the maximum and the non maximum ones.</p><h3>Convolution layer:</h3><p>You are probably asking yourself right now, if the forward pass of a convolution layer is a convolution then what is its backward pass ? luckily, its backward pass is also a convolution (as you can clearly see below) so you don’t need to worry about learning new set of hard to grasp mathematical operations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*CkzOyjui3ymVqF54BR6AOQ.gif" /><figcaption>Backpropagation through the Convolution layer</figcaption></figure><p>where:</p><ul><li><em>∂hij</em>: the derivative of the loss function w.r.t the output of the convolution layer</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*VruqyvXfFMrFCa3E9U6Eog.png" /></figure><p>This is in a nutshell how backpropagation works in a Convolution layer.</p><p>Now that you have a robust theoretical understanding of Convolutional Neural Networks let’s build our first ConvNet with TensorFlow.</p><h3>Convolutional Neural Network with TensorFlow:</h3><h3>What is Tensorflow ?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/0*d9WoXIzIR0l4ZQ0S.png" /></figure><p><a href="https://opensource.com/article/17/11/intro-tensorflow"><strong>TensorFlow</strong></a> is an open source software library for numerical computation using data-flow graphs. It was originally developed by the Google Brain Team within Google’s Machine Intelligence research organization for machine learning and deep neural networks research.</p><h3>What is a Tensor ?</h3><p>A <a href="https://en.wikipedia.org/wiki/Tensor"><strong>tensor</strong></a> is an organized multidimensional array of numerical values. The order (also degree or rank) of a tensor is the dimensionality of the array needed to represent it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*_D5ZvufDS38WkhK9rK32hQ.jpeg" /><figcaption>Types of tensors</figcaption></figure><h3>What is a Computational Graph ?</h3><p><a href="http://www.cs.columbia.edu/~mcollins/ff2.pdf"><strong>Computational graphs</strong></a> are a powerful formalism that have been extremely fruitful in deriving algorithms and software packages for neural networks and other models in machine learning. The basic idea in a computational graph is to express some model — for example a feedforward neural network — as a directed graph expressing a sequence of computational steps. Each step in the sequence corresponds to a vertex in the computational graph; each step corresponds to a simple operation that takes some inputs and produces some output as a function of its inputs.</p><p>In the illustrated graph below we have two inputs <em>w1=x</em> and <em>w2=y</em>, the inputs will flow through the graph where each node in the graph is a mathematical operation to give us the following outputs:</p><ul><li><em>w3 = cos(x)</em> where the operation is the Cosine trigonometric function</li><li><em>w4 = sin(x)</em> where the operation is the Sine trigonometric function</li><li><em>w5 = w3∙w4</em> where the operation is multiplication</li><li><em>w6 = w1/w2</em> where the operation is division</li><li><em>w7 = w5+w6</em> where the operation is addition</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/0*m4ZKS_aJNxzOtyy6.png" /></figure><p>Now that we understand what a computational graph is, let’s build our own in tensorflow, we will build the same one above.</p><h3>Code:</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a81261dd47a15929bfb7244bc16fcba5/href">https://medium.com/media/a81261dd47a15929bfb7244bc16fcba5/href</a></iframe><h3>Visualization with Tensorboard:</h3><h3>What is Tensorboard ?</h3><p><a href="https://github.com/tensorflow/tensorboard"><strong>TensorBoard</strong></a> is a suite of web applications for inspecting and understanding your TensorFlow runs and graphs, it’s one of the biggest edges that Google’s TensorFlow has over Facebook’s <a href="https://pytorch.org/"><strong>Pytorch</strong></a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/452/1*BIy_Sqob3hsC-oQkdd9UTg.jpeg" /><figcaption>The previous code visualized in Tensorboard</figcaption></figure><p>Now that you have a robust understanding of ConvNets, TensorFlow and TensorBoard, let’s build our first ConvNet that will recognize hand written digits using the <strong>MNIST</strong> dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/255/0*Sq_ASZgwAnhSrIpG" /><figcaption>MNIST dataset</figcaption></figure><p>The architecture of our Convnet will be a set of convolution, max-pooling and non linearity operation layers similar to the <a href="http://yann.lecun.com/exdb/lenet/"><strong>LeNet-5</strong></a> architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*Gze2mRYj8tl1AiGI.gif" /><figcaption>Convolutional Neural Network 3D Simulation</figcaption></figure><h3>Code:</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0ebbf68e177a2ff488a7347aba6cb628/href">https://medium.com/media/0ebbf68e177a2ff488a7347aba6cb628/href</a></iframe><p>The code is lengthy but shouldn’t be intimidating if you break it piece by piece.</p><p>In case you run the program your results should be like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8663f9d0d5956527480c1f9dbff3a08e/href">https://medium.com/media/8663f9d0d5956527480c1f9dbff3a08e/href</a></iframe><p>We have just finished building our first convolutional neural network, as you can see in the results above, the accuracy has dramatically increased from the first step to the last step, but still there is more room for improvement of our ConvNet.</p><p>let’s now visualize our ConvNet in Tensorboard:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IG7juGoP2xRsyEZP8hA6hA.png" /><figcaption>ConvNet Visualization</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*qfK6YdI0miD3wNhwrhISGA.jpeg" /><figcaption>Accuracy and Loss evaluation</figcaption></figure><h3>Conclusion:</h3><p>Convolutional Neural Networks are powerful deep learning models that are applied in a wide range of fields such as radiology, the use of ConvNets will only increase as the data gets bigger and the problems become more sophisticated and challenging.</p><h3>Note:</h3><p>You can find the Jupyter Notebook of this article in:</p><ul><li><a href="https://github.com/AegeusZerium/DeepLearning/blob/master/Deep%20Learning/Demystifying%20Convolutional%20Neural%20Networks.ipynb">https://github.com/AegeusZerium/DeepLearning/blob/master/Deep%20Learning/Demystifying%20Convolutional%20Neural%20Networks.ipynb</a></li></ul><h3>References:</h3><ul><li><a href="https://en.wikipedia.org/wiki/Convolutional_neural_network">https://en.wikipedia.org/wiki/Convolutional_neural_network</a></li><li><a href="https://en.wikipedia.org/wiki/Yann_LeCun">https://en.wikipedia.org/wiki/Yann_LeCun</a></li><li><a href="http://yann.lecun.com/exdb/mnist/">http://yann.lecun.com/exdb/mnist/</a></li><li><a href="https://opensource.com/article/17/11/intro-tensorflow">https://opensource.com/article/17/11/intro-tensorflow</a></li><li><a href="https://en.wikipedia.org/wiki/Tensor">https://en.wikipedia.org/wiki/Tensor</a></li><li><a href="http://www.cs.columbia.edu/~mcollins/ff2.pdf">http://www.cs.columbia.edu/~mcollins/ff2.pdf</a></li><li><a href="https://github.com/tensorflow/tensorboard">https://github.com/tensorflow/tensorboard</a></li><li><a href="http://yann.lecun.com/exdb/lenet/">http://yann.lecun.com/exdb/lenet/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca17bdc75559" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Artificial Neural Networks Explained]]></title>
            <link>https://medium.com/good-audience/artificial-neural-networks-explained-436fcf36e75?source=rss-d54b574e8271------2</link>
            <guid isPermaLink="false">https://medium.com/p/436fcf36e75</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Aegeus Zerium]]></dc:creator>
            <pubDate>Tue, 24 Jul 2018 19:20:32 GMT</pubDate>
            <atom:updated>2018-08-14T18:31:03.081Z</atom:updated>
            <content:encoded><![CDATA[<p>in this post we will explain everything you need to know about Artificial Neural Networks in a theoretical and programmatic way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*u8HrE8_hjKq05i2o7RkM-Q.png" /></figure><h3>Definition:</h3><p><a href="https://en.wikipedia.org/wiki/Artificial_neural_network"><strong>Artificial neural networks</strong></a> (ANNs) or connectionist systems are computing systems vaguely inspired by the biological neural networks that constitute animal brains. Such systems “learn” to perform tasks by considering examples, generally without being programmed with any task-specific rules.</p><h3>The Architecture of an Artificial Neural Network:</h3><p>ANN is a set of connected neurons organized in layers:</p><ul><li><strong>input layer:</strong> brings the initial data into the system for further processing by subsequent layers of artificial neurons.</li><li><strong>hidden layer:</strong> a layer in between input layers and output layers, where artificial neurons take in a set of weighted inputs and produce an output through an activation function.</li><li><strong>output layer:</strong> the last layer of neurons that produces given outputs for the program.</li></ul><h3>Types of ANNs:</h3><h3>Perceptron:</h3><p>The simplest and oldest model of an ANN, the Perceptron is a linear classifier used for binary predictions. This means that in order for it to work, the data must be linearly separable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*5l08QfsUsrsOxcPzfDoStg.png" /></figure><h4>Its Architecture:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*uz3wd5YeVYlU2JR8rE9VDA.png" /></figure><h3>Multi-layer ANN:</h3><p>More sophisticated than the perceptron, a Multi-layer ANN (e.g.: Convolutional Neural Network, Recurrent Neural Network etc …) is capable of solving more complex classification and regression tasks thanks to its hidden layer(s).</p><h4>Its Architecture:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/1*LaEgAU-vdsR_pClMcgbikQ.jpeg" /></figure><h3>Activation Functions:</h3><h4>Definition:</h4><p>In artificial neural networks, the <a href="https://en.wikipedia.org/wiki/Activation_function"><strong>activation function</strong></a> of a node defines the output of that node given an input or set of inputs.</p><h4>Sigmoid:</h4><p>A <a href="https://en.wikipedia.org/wiki/Sigmoid_function"><strong>sigmoid function</strong></a> is a mathematical function having a characteristic “S”-shaped curve or sigmoid curve. Often, sigmoid function refers to the special case of the logistic function which generate a set of probability outputs between 0 and 1 when fed with a set of inputs. The sigmoid activation function is widely used in binary classification.</p><h4>Equation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*dSMcIYpmEmPbELNy_gpQSQ.png" /></figure><h4>Graphical Representation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*mFBXsfKO8qt6lce4Sy08Bg.jpeg" /></figure><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fcb97a11da1a4ff91de7d5bb189ca77e/href">https://medium.com/media/fcb97a11da1a4ff91de7d5bb189ca77e/href</a></iframe><h4>Tan-h:</h4><p>An alternative to the logistic sigmoid is the <a href="https://theclevermachine.wordpress.com/tag/tanh-function/"><strong>hyperbolic tangent</strong></a>, or tan-h function. Like the logistic sigmoid, the tan-h function is also sigmoidal (“S”-shaped), but instead outputs values that range [-1, 1]. Thus strongly negative inputs to the tan-h will map to negative outputs.</p><h4>Equation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/364/1*DRrPpxBBF0xx1rftF3rBmg.png" /></figure><h4>Graphical Representation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/0*38Uwm554uykOmYyn.png" /></figure><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0b46c03f7a1f05007d0646df7c943bd0/href">https://medium.com/media/0b46c03f7a1f05007d0646df7c943bd0/href</a></iframe><h4>Softmax:</h4><p>Unlike the Sigmoid activation function, the <a href="http://dataaspirant.com/2017/03/07/difference-between-softmax-function-and-sigmoid-function/"><strong>Softmax activation function</strong></a> is used for multi-class classification. Softmax function calculates the probabilities distribution of the event over ’n’ different events. In general way of saying, this function will calculate the probabilities of each target class over all possible target classes. Later the calculated probabilities will be helpful for determining the target class for the given inputs.</p><h4>Equation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/176/1*Si0jIxOcjDx9zEsw1nIS0w.png" /></figure><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/62cda5eff5ff2e2cf5d09af87ab5fc10/href">https://medium.com/media/62cda5eff5ff2e2cf5d09af87ab5fc10/href</a></iframe><h4>ReLU:</h4><p>Instead of the sigmoid activation function, most recent artificial neural networks use <a href="https://github.com/Kulbear/deep-learning-nano-foundation/wiki/ReLU-and-Softmax-Activation-Functions"><strong>rectified linear units</strong></a> (ReLUs) for the hidden layers. A rectified linear unit has output 0 if the input is less than 0, and raw output otherwise. That is, if the input is greater than 0, the output is equal to the input.</p><h4>Equation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/217/1*P01OjrP0p13CL5Qv3rCNYA.png" /></figure><h4>Graphical Representation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/0*xnBZ-6o4UN2vatKY.png" /></figure><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a88235e086718bce13eee7ce4a662334/href">https://medium.com/media/a88235e086718bce13eee7ce4a662334/href</a></iframe><h4>Leaky ReLU:</h4><h4>Definition:</h4><p>The <strong>Leaky ReLU</strong> activation function works the same way as the ReLU activation function except that instead of replacing the negative values of the inputs with 0 the latter get multiplied by a small alpha value in an attempt to avoid the “dying ReLU” problem.</p><h4>Equation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/271/1*Gy-XsAiAeyAvF15Iv9DIjg.png" /></figure><h4>Graphical Representation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mgXY0QyfTsiJxEc-.png" /></figure><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ad6a3f311178b80ada317a2410bb48ea/href">https://medium.com/media/ad6a3f311178b80ada317a2410bb48ea/href</a></iframe><h4>Why do we use activation functions ?</h4><p>You are probably asking yourself after reading the whole Activation Function section about the useflness of the latter, why even bother using it ? why don’t we just use the initial dot product output without feeding it to the activation function ?</p><p>Well, without an activation function we will fail to introduce non-linearity into the network.</p><p>An activation function will allow us to model a response variable (target variable, class label, or score) that varies non-linearly with its explanatory variables.</p><p>non-linear means that the output cannot be reproduced from a linear combination of the inputs.</p><p>another way to think of it: without a non-linear activation function in the network, an artificial neural network, no matter how many layers it has, will behave just like a single-layer perceptron, because summing these layers would give you just another linear function.</p><h4>Which activation function is the “best” to use ?</h4><p>There are many activation functions out there but which one is the best to use? This question has been asked by many beginners in the deep learning field (including myself).</p><p>The short answer for this question is: There is no “best” activation function.</p><p>The long answer is: The use of an activation function depends on the task you are dealing with, is the task a regression or a classification problem? if it’s a classification problem then is it a binary classification or a multiclass classification task? it all comes down to the task you are dealing with.</p><p>for example we can’t possibly use the Sigmoid or tan-h activation functions for a multiclass classification task we would better be using the Softmax activation function instead. right? as it would output a vector of probabilities of each class. another example, which activation would be better to use in the hidden layer part of a multi-layered neural net? if you use the sigmoid activation function then you would be hit by the vanishing gradient problem (the vanishing gradient problem is when the gradient becomes so small in the earlier layers of a deep neural network to the point that it barely affects the weights of the earlier layers thus failing at optimizing our initial weights.), should we use the tan-h activation function then? not necessarily, it’s true that the tan-h function is an enhanced version of the sigmoid function but the vanishing gradient problem is still persistently there. So, the solution to our persistent problem is the the ReLU activation function, but how does it solve the vanishing gradient problem? the fact that the derivative of the ReLU activation function is 1 if its inputs are positive will assure that the gradient will not get infinitesimally smaller passing from one layer to another but what if our inputs are negative? here we get stuck with what we call the “dying ReLU” problem (the dying ReLU problem is when the ReLU activation function always outputs the same value (zero) for any input. That means that it takes no role in discriminating between inputs. Once a ReLU ends up in this state, it is unlikely to recover, because the function gradient at 0 is also 0, so gradient descent learning will not alter the weights.), here comes the Leaky/Parametric ReLU to rescue and instead of outputting a flat out zero for the negative values the Leaky ReLU multiplies the negative values by an alpha parametric value (α=0.01).</p><p>As you can see from the previous explanation, the use of a certain activation function depends on the task you are dealing with and the problems you face while building your neural network.</p><h3>Forward Propagation:</h3><h4>Definition:</h4><p><strong>Forward propagation</strong> is the process of feeding the Neural Network with a set of inputs to get their dot product with their weights then feeding the latter to an activation function and comparing its numerical value to the actual output called “the ground truth”.</p><h4>Calculations:</h4><p>Let’s demonstrate how the output gets calculated in a forward pass in a 3–4–1 Artificial Neural Network:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/396/0*H0EHarAJuYVb7g1Q.png" /></figure><p><strong>The initial weights of the first and second layer:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*1_kmyvpXdN_CjgDxSU2LTg.png" /></figure><p><strong>Input:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/125/1*sVjvA4rnpgwamVGJKLLfSw.png" /></figure><p><strong>output:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/75/1*2O051dWsvQ1D_DpOQRxdcg.png" /></figure><p>The activation function of the hidden layer will be the ReLU function and the activation function of the output layer will be the Sigmoid function.</p><p>let’s begin our calculculations:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/58e67ad8d4c1e8a0c94e50d0b7cf9f1e/href">https://medium.com/media/58e67ad8d4c1e8a0c94e50d0b7cf9f1e/href</a></iframe><h4>Cross entropy error:</h4><p><a href="http://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html"><strong>Cross-entropy loss</strong></a>, or log loss, measures the performance of a classification model whose output is a probability value between 0 and 1. Cross-entropy loss increases as the predicted probability diverges from the actual label. So predicting a probability of .012 when the actual observation label is 1 would be bad and result in a high loss value. A perfect model would have a log loss of 0.</p><h4>Equation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/1*wo_X1smoiVhyHUdf07e8WA.png" /></figure><blockquote><em>Where ‘yi’ is the predicted probability value for class ‘i’ and ‘y′i’ is the true probability for that class.</em></blockquote><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c372acbdfb8046cbf9266c95b310c13e/href">https://medium.com/media/c372acbdfb8046cbf9266c95b310c13e/href</a></iframe><h4>Conclusion:</h4><p>This is in a nutshell how a forward propagation works and how a neural network generates its predictions, the forward propagation step has always been less mathematically intense than the backpropagation step where most students and beginners find legitimately harder to grasp.</p><h3>Back-propagation:</h3><h4>Definition:</h4><p><a href="https://en.wikipedia.org/wiki/Backpropagation"><strong>Backpropagation</strong></a> is a method used in artificial neural networks to calculate a gradient that is needed in the calculation of the weights to be used in the network.</p><h4>Calculations:</h4><p>first let’s lay some important derivatives:</p><h4>Cross entropy error derivative:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/204/1*QRXUKvANPlAaXDnn3WAGYg.png" /></figure><h4>Sigmoid derivative:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*58KFstbBZYT6tEFO7wt41g.png" /></figure><h4>Graphical Representation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*FH6EVaNz5Oc0vJPA.jpg" /></figure><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/11d2be7cacb26c50ae6695a158fb0698/href">https://medium.com/media/11d2be7cacb26c50ae6695a158fb0698/href</a></iframe><h4>ReLU derivative:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*1w2CuUhQ-cuqyjqpD7TH_Q.png" /></figure><h4>Graphical Representation:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*g0yxlK8kEBw8uA1f82XQdA.png" /></figure><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/06ff4fbc936e47c665d9e31c905e2ab5/href">https://medium.com/media/06ff4fbc936e47c665d9e31c905e2ab5/href</a></iframe><p>Let’s now demonstrate the calculations that go underneath the Backpropagation process:</p><p>An illustration of how a neural network backpropagates its error:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/0*LSl42BywihLidUiH" /></figure><h4>Programmatically:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3b286e6fcce50eec7686ab5f8f67da1f/href">https://medium.com/media/3b286e6fcce50eec7686ab5f8f67da1f/href</a></iframe><p>let’s now predict the output with the new updated weights.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf63116f7e27272c47c1fd96066f60fb/href">https://medium.com/media/bf63116f7e27272c47c1fd96066f60fb/href</a></iframe><p>Congratulations! you’ve just done forward and backpropagation with just a few lines of code.</p><h3>Building a one hidden layer Neural Network:</h3><p>let’s now build a fully functional multilayer artificial neural network, our neural network will tackle the XOR task (where the third column of the table is irrelevant).</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*WlCd6kgxqb2yQpcutLhYwQ.png" /><figcaption>The classification task</figcaption></figure><p>the ANN’s architecture will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*-a-flCLHLCGM0-7TOcNJnQ.png" /><figcaption>The ANN we’ll build</figcaption></figure><h3>Code:</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1dfb68e7ffca869eea5cde0f52bc4005/href">https://medium.com/media/1dfb68e7ffca869eea5cde0f52bc4005/href</a></iframe><p>Your results should be like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4eb4114fdb72b212583511c5707771c2/href">https://medium.com/media/4eb4114fdb72b212583511c5707771c2/href</a></iframe><p><strong><em>Vizualization of the loss function:</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15686780ec8e495da5ea6298efda9cf2/href">https://medium.com/media/15686780ec8e495da5ea6298efda9cf2/href</a></iframe><p>As you can see in the results the cross entropy loss has dramatically decreased from the first 1000th iteration to the last 1000th iteration which showcase how good our neural network is doing to find the optimal weights.</p><p>the prediction output of our Neural Network is 0.00069824 which is very close to the ground truth 0.</p><h3>Note:</h3><p>I strongly recommend you check out the notebook to visualize the plots that are missing in this blog post.</p><p>Link to Jupyter Notebook: <a href="https://github.com/AegeusZerium/DeepLearning/blob/master/Deep%20Learning/Artificial%20Neural%20Network.ipynb">https://github.com/AegeusZerium/DeepLearning/blob/master/Deep%20Learning/Artificial%20Neural%20Network.ipynb</a></p><h3>Conclusion:</h3><p>With the appearance of Big Data, more Computational power and more challenging and mathematically intense tasks, Neural Networks will be used more than ever.</p><h3>References:</h3><ul><li><a href="https://en.wikipedia.org/wiki/Artificial_neural_network">https://en.wikipedia.org/wiki/Artificial_neural_network</a></li><li><a href="https://en.wikipedia.org/wiki/Activation_function">https://en.wikipedia.org/wiki/Activation_function</a></li><li><a href="https://en.wikipedia.org/wiki/Sigmoid_function">https://en.wikipedia.org/wiki/Sigmoid_function</a></li><li><a href="https://theclevermachine.wordpress.com/tag/tanh-function/">https://theclevermachine.wordpress.com/tag/tanh-function/</a></li><li><a href="http://dataaspirant.com/2017/03/07/difference-between-softmax-function-and-sigmoid-function/">http://dataaspirant.com/2017/03/07/difference-between-softmax-function-and-sigmoid-function/</a></li><li><a href="https://github.com/Kulbear/deep-learning-nano-foundation/wiki/ReLU-and-Softmax-Activation-Functions">https://github.com/Kulbear/deep-learning-nano-foundation/wiki/ReLU-and-Softmax-Activation-Functions</a></li><li><a href="http://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html">http://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html</a></li><li><a href="https://en.wikipedia.org/wiki/Backpropagation">https://en.wikipedia.org/wiki/Backpropagation</a></li></ul><h3>For Contacts:</h3><ul><li>Gmail: <a href="/service/mailto:EternalZer0DayX@gmail.com">EternalZer0DayX@gmail.com</a></li><li>Facebook: <a href="https://www.facebook.com/belghouat.khalil">https://www.facebook.com/belghouat.khalil</a></li><li>Twitter: <a href="http://twitter.com/EternalBlue__">@EternalBlue__</a></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F7b0181%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F7b0181%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/748a06e9019942f2677bba917ef55804/href">https://medium.com/media/748a06e9019942f2677bba917ef55804/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=436fcf36e75" width="1" height="1" alt=""><hr><p><a href="https://medium.com/good-audience/artificial-neural-networks-explained-436fcf36e75">Artificial Neural Networks Explained</a> was originally published in <a href="https://medium.com/good-audience">Good Audience</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>