<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Venktesh Subramaniam on Medium]]></title>
        <description><![CDATA[Stories by Venktesh Subramaniam on Medium]]></description>
        <link>https://medium.com/@venkteshsubramaniam?source=rss-28b126a36b67------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*V3AQAnrHtDgUb3UD.</url>
            <title>Stories by Venktesh Subramaniam on Medium</title>
            <link>https://medium.com/@venkteshsubramaniam?source=rss-28b126a36b67------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 04:29:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@venkteshsubramaniam/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Vagrant setup with Big Data Stack in Windows Subsystem for Linux]]></title>
            <link>https://medium.com/@venkteshsubramaniam/vagrant-setup-with-big-data-stack-in-windows-subsystem-for-linux-5dfcc5cd67b1?source=rss-28b126a36b67------2</link>
            <guid isPermaLink="false">https://medium.com/p/5dfcc5cd67b1</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[vagrantfile]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[vagrant]]></category>
            <dc:creator><![CDATA[Venktesh Subramaniam]]></dc:creator>
            <pubDate>Thu, 10 Feb 2022 12:37:25 GMT</pubDate>
            <atom:updated>2022-02-10T12:37:25.658Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article, we would explore the process of setting up Vagrant for running Big data stack on windows using WSL (Windows Subsystem for Linux). It is a bit tricky to get Vagrant working in WSL. Here we explore setting up Vagrant in WSL and provisioning a VM using docker. The article does not explore provisioning using VirtualBox. That is reserved for another article in the future.</p><p><strong>What is Vagrant ?</strong></p><p>Vagrant is a tool that provides a recipe for creating and managing virtual environments and tackles the age-old argument of “it works on my machine”. It helps create reproducible environments and helps tackle dependency issues. It usually is accomplished by specifying the type of machine to provision and the configuration of the machines in a file called Vagrantfile which is usually written in <strong>Ruby. </strong>The following can be considered as the core components of the Vagrant tool:</p><ul><li><strong>Box: </strong>It is a provisioned virtual machine</li><li><strong>Provider: </strong>A provider is a setup on which the virtual environment runs. It could be VMWare, VirtualBox or even a docker container.</li><li><strong>Provisioning tools: </strong>They could be simple shell scripts or more sophisticated tools like Chef or Puppet which are used to setup the virtual environment.</li></ul><p>Without further delay let us jump to the steps.</p><pre>wsl --install -d Ubuntu</pre><pre>or</pre><pre>wsl --install -d Debian</pre><p>Then we setup vagrant and virtualbox</p><pre>$   sudo apt update</pre><p>To install VirtualBox and vagrant run</p><pre>$    sudo apt install virtualbox</pre><pre>$    sudo apt install vagrant</pre><p>Running Vagrant using WSL also requires Vagrant to setup in windows environment if one wants to run Vagrant files outside WSL specific paths. Install Vagrant 2.2.6 in windows. Then to enable windows access export the following environment variable</p><pre>export VAGRANT_WSL_ENABLE_WINDOWS_ACCESS=&quot;1&quot;</pre><p>Install docker inside the wsl terminal using instructions here</p><p><a href="https://docs.docker.com/engine/install/ubuntu/">https://docs.docker.com/engine/install/ubuntu/</a></p><pre>$ sudo dockerd</pre><p>Press control+Z</p><p>Then type bg into the terminal.</p><p>Clone the repo <a href="https://github.com/VenkteshV/vagrant-hadoop-hive-spark">https://github.com/VenkteshV/vagrant-hadoop-hive-spark</a></p><p>Switch to sudo mode “sudo -s”</p><p>Now start the vagrant VM using the command:</p><pre>sudo vagrant up --provider=docker</pre><p>RUn any of your favourite big data stack by logging into the VM using the command “vagrant ssh”.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5dfcc5cd67b1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Support Vector Machine — with Math — part 1]]></title>
            <link>https://medium.com/analytics-vidhya/support-vector-machine-with-math-part-1-df5a4bf69913?source=rss-28b126a36b67------2</link>
            <guid isPermaLink="false">https://medium.com/p/df5a4bf69913</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Venktesh Subramaniam]]></dc:creator>
            <pubDate>Sun, 17 Nov 2019 14:15:12 GMT</pubDate>
            <atom:updated>2019-11-21T12:15:51.417Z</atom:updated>
            <content:encoded><![CDATA[<h3>Support Vector Machine — with Math — part 1</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*SRLhDoAORm9biKsQgnv15w.png" /><figcaption>Source : google</figcaption></figure><p>SVM is one of the popular supervised machine learning algorithms. It is used for both regression and classification task. It is a discriminative classification algorithm. <br>Naïve Bayes is a generative classifier — a model is learnt that describes the distribution of each underlying class and the generative model is used to probabilistically generate labels for new points. But as termed SVM is a discriminative classifier, rather than modelling each class we find a line or curve(in 2 dimensions) or manifold(in higher dimensions) to separate classes from each other.<br>If you can relate to the diagram the formal definition of SVM is “The main objective of SVM is to find the optimal linearly separating hyperplane which maximizes the margin.”</p><p><strong>INTUITION BEHIND SVM:</strong></p><p>Consider the simple case of a classification task, in which the two classes of points are well separated</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/586/1*I7YaYkGKKxS4SIP_Z4jRGA.png" /></figure><p>A linear discriminative classifier would attempt to draw a straight line separating the two sets of data, and thereby create a model for classification. For 2 D data like that shown here, this is a task we could do by hand. But immediately we see a problem there is more than one possible dividing line that can perfectly discriminate between the two classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*FE-WVP0lNHL1qr3ASQSkDg.png" /></figure><p>Evidently our simple intuition of “drawing a line between classes” is not enough, and we need to think a bit deeper.</p><p>Support vector machines offer one way to improve on this. The intuition is this: rather than simply drawing a zero-width line between the classes, we can draw around each line a margin of some width, up to the nearest point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*V1rN0xWnwenseovADgytdA.png" /></figure><p>In support vector machines, the line that maximizes this margin is the one we will choose as the optimal model. Support vector machines are an example of such a maximum margin estimator.</p><p>We now we can use sklearn’s SVC (Support vector classifier ) which can be used to separate classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/859/1*OUe71-3qf2LmJ69TVdwy7g.png" /></figure><p>In the diagram in the previous line we see a dividing line that maximizes the margin between the set of points. Few of the training points just touch the margin.. These points are the pivotal elements of this fit, and are known as the support vectors, and give the algorithm its name. In Scikit-Learn, the identity of these points are stored in the support_vectors_attribute of the classifier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*-XsOP1Y9lZfmG-plTmIWKQ.png" /></figure><p>A key to this classifier’s success is that for the fit, only the position of the support vectors matter<br>Any points further from the margin which are on the correct side do not modify the fit.<br> Technically, this is because these points do not contribute to the loss function used to fit the model, so their position and number do not matter so long as they do not cross the margin.<br>So, when we are choosing optimal hyperplane we will choose one among set of hyperplane which is highest distance from the closest data points(support vectors). If optimal hyperplane is very close to data points then margin will be very small and it will generalize well for training data but when an unseen data will come it will fail to generalize well as explained above. So our goal is to maximize the margin so that our classifier can generalize well for unseen instances.</p><h3><strong>Summary of how SVM training works</strong></h3><p>So in the training phase, we fix a subset of training points from which we compute the similarity of any test point. These selected training points are called support vectors, since only these points will support our decision of selecting the class of a test point. Our hope is that our training phase finds as few as support vectors so that we must compute fewer number of similarities.<br>Now once we have selected support vectors, we assign a weight to each support vector, which basically tells how much importance we want to give to that support vector while making our decision. We don’t give importance to only a single training point, instead we give each support vector a separate importance.<br> To take decision , we just compute a weighted sum of similarities from test point to each of the support vector and compute the class based on this weighted sum.</p><h3><strong>Concepts to know before deep diving into SVM.</strong></h3><p><strong>Length of a vector:<br></strong>The length of a vector x is called its norm, which is written as ||x||. The Euclidean norm formula to calculate the norm of a vector x = (x1,x2,…,xn) is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*fPK9o6mToEbGSPie4gfL-A.png" /><figcaption>vector norm</figcaption></figure><p><strong>Direction of a vector:</strong></p><p>The direction of a vector x = (x1,x2x1,x2) is written as w, and is defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*9FE63gTPRFlTRArKZxPJPQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CCLQ4fpegX8_KOfLzlAshQ.png" /><figcaption>direction of a vector example</figcaption></figure><p>If we look at the figure , we can see that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/308/1*j0xkkUgU4a9JDDSFNa2CkA.png" /></figure><p>Thus, the direction vector w can also be written as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/279/1*7FgCVxi7e0hvRZ2GZZbyww.png" /></figure><p>The norm of a direction vector is always 1. Therefore w is also called unit vector.</p><h3><strong>Dot product of 2 vectors:</strong></h3><p>The dot product of two vectors returns a scalar. It gives us some insights into how the two vectors are related.<br>The figure below shows two vectors x and y and the angle θθ between them. The geometric formula of dot product is defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/278/1*_gaK09_qzk-V0r0E534asw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hz5NYgUpK7QXK_2naP6row.png" /></figure><p>But from above diagram we see that</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/108/1*Y9W0DlDWtIgti50tabaiLA.png" /></figure><p>Therefore we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/272/1*4jDiH_kW_VGI7Tv3pYhwAg.png" /></figure><p>substituting cos(theta) in formual we get :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*xYOJOrRtmqkFj1i8f1e_rg.png" /></figure><p>This is the algebraic formula of dot product. In general, dot product can be computed as the following for two n-dimensional vectors:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/202/1*RedHMJBznw7NNT8lx4zY9A.png" /></figure><h3><strong>Hyperplane</strong></h3><p>We have seen the term hyperplane earlier . But what is it exactly ? It is used to separate higher dimensional data (3D).<br>Let’s look at the two-dimensional case first. The two-dimensional linearly separable data can be separated by a line. The equation of the line is y=ax+b. We rename x with x1and y with x2 and we get:</p><p>ax1−x2+b=0</p><blockquote>If we define x = (x1,x2) and w = (a,−1), we get:<br>w⋅x+b=0<br>This equation is derived from two-dimensional vectors. But in fact, it also works for any number of dimensions. This is the equation of the hyperplane.</blockquote><p>Once we have the hyperplane, we can then use the hyperplane to make predictions. We define the hypothesis function h as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/1*6zHeRTvtEiUNF725ATtIBA.png" /></figure><p>The point above or on the hyperplane will be classified as class +1, and the point below the hyperplane will be classified as class -1.<br>So basically, reiterating the goal of the SVM learning algorithm is to find a hyperplane which could separate the data accurately. There might be many such hyperplanes. And we need to find the best one, which is often referred as the optimal hyperplane.</p><p><strong>Metrics to compare hyperplanes:</strong></p><p>Let’s first consider the equation of the hyperplane w⋅x+b =0 . We know that if the point (x,y) is on the hyperplane, w⋅x+b=0. If the point (x,y) is not on the hyperplane, the value of w⋅x+b could be positive or negative. For all the training example points, we want to know the point which is closest to the hyperplane. We could calculate β=|w⋅x+b| (perpendicular distance). To formally define the problem:<br>Given a dataset , we compute β for each training example, and B is the smallest β we get.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*t8-DHCXh5Rk8TwMaTxKx7Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/279/1*omLleLqJX7g1czwkBufFmg.png" /></figure><p>If we have s hyperplanes, each of them will have a Bi value, and we’ll select the hyperplane with the largest Bi value.</p><p>The problem with the previous metric is that it could fail to distinguish between a good hyperplane and a bad one. Because we take the absolute value of w⋅x+b, we could get the same value for a correct and an incorrect hyperplane. We need to adjust this metric.<br>We could use the information of the label y. Let’s define f=y(w⋅x+b), and the sign of f will always be positive if the point is correctly classified and will be negative if incorrectly classified.<br>To make it formal, given a dataset D, we compute f for each training example, and F is the smallest f we get. In literature, F is called the functional margin of the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/264/1*nK9YHjHqQGvZJ_fl6V4CbQ.png" /></figure><p>When comparing hyperplanes, the hyperplane with the largest F will be favorably selected.<br>It looks like we found the correct metric. However, this metric suffers from a problem called scale variant. For example, we have two vectors w1=(1,4) and w2=(10,20)). Since they have the same unit vector u=(0.6,0.8), the two vectors w1 and w2represent the same hyperplane. However, when we compute F, the one with w2 will return a larger number than the one with w1. We need to find a metric which is scale invariant.</p><p>Now let’s see the final version which would be scale invariant too:We divide f by the length of the vector w. We define:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/271/1*zrEOzRRKZZTvQ7ysTlVt0w.png" /></figure><p>To make it formal, given a dataset D, we compute γ for each training example, and M is the smallest γ we get. In literature, M is called the geometric margin of the dataset</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*TF_4WxrtO776o-5Hnoa11g.png" /></figure><p>When comparing hyperplanes, the hyperplane with the largest M will be favorably selected.<br>We now have a perfect metric for comparing different hyperplanes. Our objective is to find an optimal hyperplane, which means we need to find the values of w and b of the optimal hyperplane.<br>The problem of finding the values of w and b is called an optimization problem.</p><p>In the next part I will come up with how to derive the svm optimization problem (hint duality).</p><p><strong>References:</strong></p><ul><li><a href="https://shuzhanfan.github.io/2018/05/understanding-mathematics-behind-support-vector-machines/">Understanding the mathematics behind Support Vector Machines</a></li><li><a href="https://jakevdp.github.io/PythonDataScienceHandbook/">Python Data Science Handbook</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df5a4bf69913" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/support-vector-machine-with-math-part-1-df5a4bf69913">Support Vector Machine — with Math — part 1</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The thundering herd — Distributed Systems rate limiting]]></title>
            <link>https://medium.com/@venkteshsubramaniam/the-thundering-herd-distributed-systems-rate-limiting-9128d20e1f00?source=rss-28b126a36b67------2</link>
            <guid isPermaLink="false">https://medium.com/p/9128d20e1f00</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <dc:creator><![CDATA[Venktesh Subramaniam]]></dc:creator>
            <pubDate>Tue, 06 Aug 2019 15:32:13 GMT</pubDate>
            <atom:updated>2019-08-06T15:32:13.363Z</atom:updated>
            <content:encoded><![CDATA[<h3>The thundering herd — Distributed Systems rate limiting</h3><p>Imagine a bull running towards you when you are relaxing on a farm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*gJU3bmk09A_6o5co44_zFg.jpeg" /><figcaption>Source: Google</figcaption></figure><p>What would you do? Easy “I would just get out of the way” you would say. Well how about now?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/333/1*WJreBQ0JTjrbuvT20_VZag.jpeg" /><figcaption>Source: google</figcaption></figure><p>If it’s not clear it’s a herd of raging bulls. Unless you have extreme luck you would likely be trampled. That’s the exact situation of the servers when a huge amount of requests greater than it’s compute power comes in. This is commonly known as “<strong>thundering herd”</strong> problem. In this blog, I will try to discuss the various scenarios and possible solutions from my understanding through various references.</p><p>The thundering herd problem can occur when there is a cascading failure — say you have 3 servers running and a load balancer</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*81TahBbPSvKZzfsF9Ce9pw.png" /></figure><p>Let us say each server can handle a certain number of requests(say 500) per second and the load balancer distributes the requests to handle the load. But what if the server 1 goes down while handling 500 requests ( after completing say 200 requests). So the rest of the 300 requests would be sent to server 2 . Let’s say server 2 comparatively has lesser computing power or even if it has the same computing power it is handling more than it’s designated capacity (QPS). So the server 2 crashes after a while ( after handling 200 requests). Now the first 200 + requests directed at server 2 goes to server 3. Now remember server 3 may additionally also have to handle new requests. So there we have the thundering herd problem . By now let’s say if you are using JVM based languages there is a high probability of out of memory error and server 3 crashes now. So there you have the scenario of “<strong>cascading failure” </strong>. Now, you may ask why not scale up (using terraform) and add another server? But in this situation of cascading failure there is a high probability that server would crash too despite higher computing power and memory allocations.</p><p>So what is a solution for this problem? I hope everyone remembers the various buffering algorithms we have read about in college :</p><p>Token Bucket</p><p>Leaky Bucket</p><p>Read them up again. For now, Let us say we have a queue in between to buffer the requests once a server goes down and then process them by distributing across available servers in limited quantities. If the QPS(queries per second) exceed the queue capacity , then we have to return a temporary error to the user (“try later”) as it is better to serve some users than none at all.</p><p>With the above idea token bucket can be implemented this way . For each request we can categorize the user by maintaining an in memory map or in redis, with the user as key and a token (which would be the number of requests remaining for the user). On each request from the same user the token is decreased by one and when it reaches zero we throw temporary error to user. Well this is space intensive when there are many users. I am sure highly scalable systems have smarter solutions.</p><p>There are many such scenarios that would cause thundering herd problems. Like the big billion day. But in that case, there is a way of handling the load as flipkart could predetermine the load approximately and auto-scale or pre-scale using terraform.</p><p>But what if some video trends, or some image of a celebrity trends. In this case it is difficult to determine the load beforehand. Even auto scale might not work out or is not economical depending on the load. In these cases metadata like number or likes,comments could be approximated to avoid placing load by constant querying and calculations.</p><p>Do customer facing applications only face this problem? No. If you are executing batch jobs(cron) then you might have this problem too. If you face constant issues in your cron jobs which curates data from it’s respective sources and updates to a store or file uploads and backups then it might be due to all those cron jobs being scheduled at the same time. One way to solve this is to interleave them and reduce the quantity of data and queries in each of those jobs. “<strong>Jitter” </strong>really helps here. In layman’s terms jitter breaks periodicity.</p><p>There are many more sophisticated solutions and i would recommend looking into various resources for further understanding</p><p><strong>REFERENCES:</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FxrizarXJgC8%3Fstart%3D968%26feature%3Doembed%26start%3D968&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DxrizarXJgC8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FxrizarXJgC8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d06a0d7b8606d1d1b4cf8b5289be0db/href">https://medium.com/media/4d06a0d7b8606d1d1b4cf8b5289be0db/href</a></iframe><p><a href="http://highscalability.com/blog/2012/4/17/youtube-strategy-adding-jitter-isnt-a-bug.html">http://highscalability.com/blog/2012/4/17/youtube-strategy-adding-jitter-isnt-a-bug.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9128d20e1f00" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Federated learning an introduction]]></title>
            <link>https://medium.com/secure-and-private-ai-writing-challenge/federated-learning-an-introduction-93bc0167f916?source=rss-28b126a36b67------2</link>
            <guid isPermaLink="false">https://medium.com/p/93bc0167f916</guid>
            <category><![CDATA[federated-learning]]></category>
            <category><![CDATA[secure-and-private-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Venktesh Subramaniam]]></dc:creator>
            <pubDate>Mon, 01 Jul 2019 17:57:50 GMT</pubDate>
            <atom:updated>2019-07-03T04:33:23.778Z</atom:updated>
            <content:encoded><![CDATA[<p>Privacy in an increasing concern in the smart phones era. Every smart device owner is generating lots of data and this data has a lot of potential to improve the way we all communicate and also perform our day to day tasks. For instance let’s say based on our messages and emails we type using the keyboard , if we could get better suggestions for the messages we type or even story writing (like GPT2 ) won’t our lives be a lot easier ? as it would save us the time of typing same responses for FAQ’s. But there is a catch here right? Training a LSTM or any attention based model on the data generated by user would give more personalized recommendations rather than training it on a large generic corpus. The problem here is the data would leave the device and has to be fed to a model running on a central server. We would not want the things we type on our smart phones to be accessible to a central server.</p><p>One solution is to randomize every text before sending it to server(differential privacy). But another effective method is federated learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*IHeKZH_oFyaCjk-B.gif" /><figcaption>Source: <a href="http://vision.cloudera.com/wp-content/uploads/2018/11/2018-10-31-181344-federated_learning_animated_labeled.gif">http://vision.cloudera.com/wp-content/uploads/2018/11/2018-10-31-181344-federated_learning_animated_labeled.gif</a></figcaption></figure><p>Before we start on federated learning a common question that would have crossed the mind is “wait, if the data leaving the device is the problem why not have the entire model setup in the device and train it locally?”. Well that won’t work out well for 2 reasons:</p><ol><li>The data on a single client is very limited.</li><li>Other clients are not contributing to the model training and we risk chances on overfitting to device data and the model is not very intelligent.In short the suggestion might become more of an annoyance rather than a convenience.</li></ol><p>So, what if we can still train a centralized model on decentralized data without invading privacy. That’s where FL comes in. A more intuitive explanation of FL would be “bringing the model to data rather than the data to the model as in traditional approach”.</p><h3><strong>How FL Works ?</strong></h3><p>The server first initializes the model. This could be done with any of the neural network initialization methods. It could also be a pretrained model. Then the server sends the model to the client devices(selected at random). Each device trains the model locally and computes an update that must be made to the global model. The server receives the updates and averages them by including a weighting factor for each update which is computed based on the training set each client used. Then the global model is updated using some form of gradient descent(like SGD). This entire process flow amounts to one round. Several such rounds are performed until the model converges.</p><p>To avoid multiple round trips the parameter of number of users should be chosen carefully.</p><p>Well, the above methodology sounds pretty cool. But there are some problems in FL(federated learning):</p><ol><li>The distribution the data is sampled from varies from client to client. In traditional distributed learning there are ways to ensure that the data is well represented. But in FL since users are sampled at random it is very much possible that two datasets of 2 random clients are completely from different distributions.Also each user might not have same amount of samples. One could generate only 10 samples while another could generate larger number based on usage.</li><li>Communication problems arise as devices may belong to different networks (lesser bandwidth etc.)</li></ol><p>And one more problem we didn’t discuss while discussing the how works is privacy.</p><p>While it is true that the data never left the device in the process discussed above there is a possibility of reconstruction attack. A coordinating device maybe an adversary and if that adversary gets access to the gradients it is possible to reconstruct the original data from the gradients(eg: reconstruction of faces in the facial recog. domain).</p><p>There are several ways to tackle this.</p><ol><li><strong>Gradient clipping : </strong>When gradient clipping is performed it limits the amount of influence a single update(an individual device) can influence the final update. This prevents the adversary from learning an individual based on the contribution.</li><li>Another way is to not sample from a fixed number of users but involving different users randomly.</li><li>A popular way is to add some noise to the individual updates (local differential privacy) or to the final update(global diff. privacy). The noise added to individual updates are added in such a way that when aggregating them they cancel out and it is possible to obtain a proper final update.</li></ol><p><strong>CONCLUSION:</strong></p><p>I hope this gives an insight into how FL works. One fun fact is google board uses FL to give personalized suggestions. Don’t worry about the model training draining the battery as it happens only when the device is plugged in .</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93bc0167f916" width="1" height="1" alt=""><hr><p><a href="https://medium.com/secure-and-private-ai-writing-challenge/federated-learning-an-introduction-93bc0167f916">Federated learning an introduction</a> was originally published in <a href="https://medium.com/secure-and-private-ai-writing-challenge">Secure and Private AI Writing Challenge</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Things learnt and tried during my kaggle digit recognizer submission-Part-1(PCA explained)]]></title>
            <link>https://medium.com/@venkteshsubramaniam/things-learnt-and-tried-during-my-kaggle-digit-recognizer-submission-part-1-pca-explained-1fb6db78c402?source=rss-28b126a36b67------2</link>
            <guid isPermaLink="false">https://medium.com/p/1fb6db78c402</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Venktesh Subramaniam]]></dc:creator>
            <pubDate>Mon, 11 Feb 2019 13:52:42 GMT</pubDate>
            <atom:updated>2019-02-11T13:52:42.747Z</atom:updated>
            <content:encoded><![CDATA[<p>Note: This blog is a note to self and for those who are interested. The blog may not be interesting to those who are already familiar with all the concepts mentioned.</p><p>So when i started with the well known digit recognizer problem in kaggle(recognition of handwritten digits) I didn’t start right away with a neural network approach but rather from the basic feature engineering.I will summarize the methods i used and also mistakes made and the submission that increased my rank by<strong> 1149 places.</strong></p><p>One of the prime aspects of machine learning is to focus primarily on feature engineering. By feature engineering, I mean methods for generating, extracting or altering features in the data set. Of course some competition winning solutions use a black box like ensembles of gradient boosting but i wanted to use this as an opportunity to learn. And feature engineering is very important and will also help to explain your model better. Even neural networks can be explained by gradient checking to assert or verify whether you are right about your model(more on this in another post).</p><p>Ok so without further delay to the main subject. I started with PCA(principal component analysis) for dimensionality reduction.There are other methods like TSNE but i will be explaining how i used PCA and also i will try my best to give an intuitive explanation of it.</p><p><strong>PRINCIPAL COMPONENT ANALYSIS:</strong></p><p>SO what is PCA?</p><p><em>PCA is a linear transformation algorithm that seeks to project the original features of our data onto a smaller set of features ( or different co-ordinate axis) while still retaining most of the information. To do this the algorithm tries to find the most appropriate directions/angles ( which are the principal components ) that maximize the variance in the new coordinate-axis.</em></p><p>Ok, so above definition doesn’t seem so intuitive. Let me explain with a real world example which i would say i formulated after reading a stackexchange answer.</p><p>Let’s say you go to a showroom to buy a pair of shoes or through online shopping. You would first look for desirable features right? Depending on whether you would use them for running or walking you would look for certain characteristics like sole strength, brand, waterproof easy wear and so on.</p><p>We can compose a whole list of different characteristics of each shoe in the display. But many of them will measure related properties and so will be redundant. If so, we should be able to summarize each shoe with fewer features. This is what PCA does.But it doesn’t mean PCA finds repetitive characteristics and discards them but instead it constructs new characteristics that summarizes our shoe and these features are computed using old features.</p><p>For example a new feature maybe computed as (sole_strength-shoe_weight) (linear combination of features).We can find the best features using PCA, that summarize the list of shoes so well.</p><p>Why is PCA so useful? Well, when we are purchasing a shoe we are looking for some properties (characteristics) that strongly differ across shoes and brands. If we come up with a property that is the same for most of the shoes. This would not be very useful, would it? Shoes are very different, but the new property makes them all look the same! This would certainly be a bad summary. Instead, PCA looks for properties that show as much variation across shoes as possible.</p><p>We look out for the properties that would allow you to <strong>reconstruct</strong>, the original shoe characteristics. Again, imagine that we come up with a property that has no relation to the original characteristics; if you use only this new property, there is <strong>no way to reconstruct the original ones</strong>. This, again, would be a bad summary. So PCA looks for properties that allow to reconstruct the original characteristics as well as possible.</p><p>The above two goals are equivalent.So the gist is that a simple object can be often described by a lot of variables. So to describe it with simple and lesser number of variables is by choosing a proper co-ordinate system.</p><p>That’s exactly what PCA analysis does. The term “<strong>principal component</strong>” denotes new variables (or coordinate systems) we choose to describe our data in a more concise or convenient way. All principal components must satisfy two conditions:</p><ol><li>They must be perpendicular (or mathematically, Orthogonal) to each other.</li></ol><p>This means principal components are NOT linear correlated between each other.</p><ol><li>And that’s why PCA can reduce the number of features with out losing information(remember we don’t discard old features but rather new features are a linear combination of old ones).Variables in raw data are not independent, and correlated variables cause redundancy.</li></ol><p>So, now to the code :</p><pre>import numpy as np <em># linear algebra</em><br>import pandas as pd <em># data processing, CSV file I/O (e.g. pd.read_csv)</em><br><br>import plotly.offline as py<br>py.init_notebook_mode(connected=True)<br>import plotly.graph_objs as go<br>import plotly.tools as tls<br>import seaborn as sns<br>import matplotlib.image as mpimg<br>import matplotlib.pyplot as plt<br>import matplotlib<br>%matplotlib inline<br><br><em># Import the 3 dimensionality reduction methods</em><br>from sklearn.manifold import TSNE<br>from sklearn.decomposition import PCA<br>from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA</pre><pre>train = pd.read_csv(&#39;../input/train.csv&#39;)<br>train.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QyozaYngOw45PBFRuDyqBw.png" /></figure><pre>print(train.shape)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Uw_VF0XU-knn9uZ9siZFHg.png" /></figure><p>The MNIST set consists of 42,000 rows and 785 columns. There are 28 x 28 pixel images of digits ( contributing to 784 columns) as well as one extra label column which is essentially a class label to state whether the row-wise contribution to each digit gives a 1 or a 9. Each row component contains a value between one and zero and this describes the intensity of each pixel.</p><p>Before we start off, let’s conduct some cleaning of the train data by saving the label feature and then removing it from the dataframe</p><pre><em># save the labels to a Pandas series target</em><br>target = train[&#39;label&#39;]<br><br>train = train.drop(&quot;label&quot;,axis=1)</pre><p>SO, PCA mainly depends on variance and it may be useful to observe how the variances look like for the digits in the dataset. Therefore to achieve this, let us calculate the eigenvectors and eigenvalues of the covarience matrix as follows:</p><pre># Standardize the data<br>from sklearn.preprocessing import StandardScaler<br>X = train.values<br>X_std = StandardScaler().fit_transform(X)</pre><pre># Calculating Eigenvectors and eigenvalues of Cov matirx<br>mean_vec = np.mean(X_std, axis=0)<br>cov_mat = np.cov(X_std.T)<br>eig_vals, eig_vecs = np.linalg.eig(cov_mat)<br># Create a list of (eigenvalue, eigenvector) tuples<br>eig_pairs = [ (np.abs(eig_vals[i]),eig_vecs[:,i]) for i in range(len(eig_vals))]</pre><pre># Sort the eigenvalue, eigenvector pair from high to low<br>eig_pairs.sort(key = lambda x: x[0], reverse= True)</pre><pre># Calculation of Explained Variance from the eigenvalues<br>tot = sum(eig_vals)<br>var_exp = [(i/tot)*100 for i in sorted(eig_vals, reverse=True)] # Individual explained variance<br>cum_var_exp = np.cumsum(var_exp) # Cumulative explained variance</pre><p>Plotly visualisation package can be used to produce an interactive chart to showcase individual explained variance and cumulative explained variance:</p><pre>trace1 = go.Scatter(<br>    x=list(range(784)),<br>    y= cum_var_exp,<br>    mode=&#39;lines+markers&#39;,<br>    name=&quot;&#39;Cumulative Explained Variance&#39;&quot;,<br>#     hoverinfo= cum_var_exp,<br>    line=dict(<br>        shape=&#39;spline&#39;,<br>        color = &#39;goldenrod&#39;<br>    )<br>)<br>trace2 = go.Scatter(<br>    x=list(range(784)),<br>    y= var_exp,<br>    mode=&#39;lines+markers&#39;,<br>    name=&quot;&#39;Individual Explained Variance&#39;&quot;,<br>#     hoverinfo= var_exp,<br>    line=dict(<br>        shape=&#39;linear&#39;,<br>        color = &#39;black&#39;<br>    )<br>)<br>fig = tls.make_subplots(insets=[{&#39;cell&#39;: (1,1), &#39;l&#39;: 0.7, &#39;b&#39;: 0.5}],<br>                          print_grid=True)</pre><pre>fig.append_trace(trace1, 1, 1)<br>fig.append_trace(trace2,1,1)<br>fig.layout.title = &#39;Explained Variance plots - Full and Zoomed-in&#39;<br>fig.layout.xaxis = dict(range=[0, 80], title = &#39;Feature columns&#39;)<br>fig.layout.yaxis = dict(range=[0, 60], title = &#39;Explained Variance&#39;)<br># fig[&#39;data&#39;] = []<br># fig[&#39;data&#39;] += (go.Scatter(x= list(range(784)) , y=cum_var_exp, xaxis=&#39;x2&#39;, yaxis=&#39;y2&#39;, name = &#39;Cumulative Explained Variance&#39;))</pre><pre># fig[&#39;data&#39;] += [go.Scatter(x=list(range(784)), y=var_exp, xaxis=&#39;x2&#39;, yaxis=&#39;y2&#39;,name = &#39;Individual Explained Variance&#39;)]</pre><pre># fig[&#39;layout&#39;] = layout<br># fig[&#39;layout&#39;] += layout2<br>py.iplot(fig, filename=&#39;inset example&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Iy2CznHvvQTkCxbI8Ubmzg.png" /></figure><p>It can be seen from above plot , out of our 784 features or columns approximately 90% of the Explained Variance can be described by using just over 200 over features. So if one wanted to implement a PCA on this, extracting the top 200 features would be a very good choice as they already account for the majority of the data. In the section below, I will use Sklearn toolkit and its built-in PCA method</p><pre>target = train[&#39;label&#39;]<br>train = train.drop(&#39;label&#39;, axis=1)<br>from sklearn.decomposition import PCA<br>n_components = 60<br>pca = PCA(n_components=n_components).fit(train.values)</pre><pre>eigenvalues = pca.components_.reshape(n_components, 28, 28)</pre><pre># Extracting the PCA components ( eignevalues )<br>eigenvalues = pca.components_</pre><p>Next let us visualize the eigenvalues</p><pre>n_row = 5<br>n_col = 12</pre><pre># Plot the first 28 eignenvalues<br>plt.figure(figsize=(13,12))<br>for i in list(range(n_row * n_col)):<br>    offset =0<br>    plt.subplot(n_row, n_col, i + 1)<br>    plt.imshow(eigenvalues[i].reshape(28,28), cmap=&#39;jet&#39;)<br>    title_text = &#39;Eigenvalue &#39; + str(i + 1)<br>    plt.title(title_text, size=6.5)<br>    plt.xticks(())<br>    plt.yticks(())<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D4nWzmCSgDqtFbtDrEC7sA.png" /></figure><p>Next let us visualize some values</p><pre>plt.figure(figsize=(14,12))<br>for digit_num in range(0,70):<br>    plt.subplot(7,10,digit_num+1)<br>    grid_data = train.iloc[digit_num].as_matrix().reshape(28,28)  # reshape from 1d to 2d pixel array<br>    plt.imshow(grid_data, interpolation = &quot;none&quot;, cmap = &quot;afmhot&quot;)<br>    plt.xticks([])<br>    plt.yticks([])<br>plt.tight_layout()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bUuXfBYuPWHG3iXNVnSPaA.png" /></figure><p>Next i choose features with variance above 0.83</p><pre>pca = PCA(n_components= 0.8388)<br>pca.fit(X_std)<br>X_5d = pca.transform(X_std)<br>Target = target</pre><p>So now with my features selected I used knn classifer to fit and predict the digits.</p><pre>from sklearn.model_selection import train_test_split<br>del train<br>train = pd.read_csv(&#39;../input/train.csv&#39;)<br>test = pd.read_csv(&#39;../input/test.csv&#39;)<br>X = (train.values[:,1:])<br>Y = (train.values[:,0])<br>X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size  = 0.666666, random_state  = 0)<br>X_test.shape</pre><p>And</p><pre>from sklearn.neighbors import KNeighborsClassifier<br>from sklearn.preprocessing import PolynomialFeatures<br>poly = PolynomialFeatures(include_bias = False)<br>xtrain = pca.fit_transform(X_train)<br>xtest = pca.transform(X_test)<br>xtrainpoly = poly.fit_transform(xtrain)<br>xtestpoly = poly.transform(xtest)<br>clf = KNeighborsClassifier(n_neighbors=16)</pre><pre>clf = clf.fit(xtrainpoly, y_train)<br>output_label = clf.predict(xtestpoly)</pre><pre>output = pd.DataFrame(output_label,columns = [&#39;Label&#39;])<br>output.reset_index(inplace=True)<br>output[&#39;index&#39;] = output[&#39;index&#39;] + 1<br>output.rename(columns={&#39;index&#39;: &#39;ImageId&#39;}, inplace=True)<br>output.to_csv(&#39;output.csv&#39;, index=False)<br>output.head()</pre><p>So this gave me a score of 0.3 something on leaderboard.(Note: I additionally used polynomialfeatures to improve my accuracy. Please look into that and ping me if u need the concept behind it.</p><p>Ok, so that’s it in part-1. In part 2 ill tell how i climbed to rank 1159 something.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1fb6db78c402" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimization Algorithms — from mini-batch,weighted averages to ADAM,part-1]]></title>
            <link>https://medium.com/@venkteshsubramaniam/optimization-algorithms-from-mini-batch-weighted-averages-to-adam-part-1-3a3109a3ed11?source=rss-28b126a36b67------2</link>
            <guid isPermaLink="false">https://medium.com/p/3a3109a3ed11</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[optimization-algorithms]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[mini-batch]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Venktesh Subramaniam]]></dc:creator>
            <pubDate>Sun, 03 Feb 2019 20:36:18 GMT</pubDate>
            <atom:updated>2019-02-03T20:50:31.952Z</atom:updated>
            <content:encoded><![CDATA[<h3>Optimization Algorithms — from mini-batch,weighted averages to ADAM,part-1</h3><p>This article is a note to myself for future reference and those who maybe interested.I thank deeplearning.ai’s 2nd course on deep learning specialization for the knowledge imparted .</p><p>Deep learning methods work on huge amount of data.So the training could be slow.But with good optimization algorithms there is a good opportunity to speed up the training process.Now, in this article i would like to begin with minibatch gradient descent.</p><h4>BATCH AND MINIBATCH GRADIENT DESCENT:</h4><p>Let’s say we have m training samples.Usually they are stacked as a vector like:</p><p>X = [x(1),x(2)……….x(m)] ,dimensions -(n,m)</p><p>and as for the labels</p><p>Y = [y(1),y(2)…….y(m)] , dimensions-(1,m)</p><p>Surely vectorization would help in speeding up the computation</p><p>Y = W . X +b(where W,X,Y are vectors that is m examples of X stacked horizontally)</p><p>if m is relatively small. But what if m is in millions or more.Even vectorization wouldn’t help, wouldn’t be fast enough. Why?</p><p>This is because gradient descent(batch gradient descent that is) would require processing of the entire training set before taking a step in gradient descent. But what if we could make slight improvements, i.e make gradient descent make progress even without fully processing the entire batch of training examples? This is where minibatch gradient descent comes to the rescue.</p><p>The training set can be split into smaller sibling training sets.These smaller sets are called minibatches. for example let’s say m = 5,000,000</p><p>Then if the training batch is split into minibatches, each containing 1000 examples, then we would have 5000 minibatches.</p><p>X{1} = [x(1)……..x(1000)]</p><p>X{2} = [x(1001)……..x(2000)]</p><p>where X{1} is the first minibatch, X{2} is the second minibatch and so on.</p><p>Similarly Y can also be split up into minibatches.Batch gradient descent runs on the entire training sample whereas mini-batch gradient descent rusn on one minibatch at a time(X{t}). So, to run mini-batch gradient descent</p><pre>for t =  1...5000 ( as we have 5000 mini-batches)<br>  Z[1] = W[1] X{t} + b[1] ( where t denotes the mini-batch,we perform vectorized computation on the minibatch i.e first 1000 examples. [1] denotes the first layer)</pre><pre>A[1] = g[1](Z[1])<br>.<br>.<br>.<br>A[L] = g[L](Z[L])<br>(the computation taking place layer by layer can also be vectorized)</pre><p>Now for computing cost for this scenario it would be :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/343/1*7_INfKqYvisZeicxpEE5RQ.png" /></figure><p>where here 1000 denotes the number of examples in one minibatch, and t denotes the minibatch. If regularization is used then there would be an addition of frobenius form, but we will get to that part in another article.</p><p>Now, backpropagation can be used to compute gradients and it is done with respect to J{t}(i.e w.r.t X{t}.Y{t}). and weight update rule becomes,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/228/1*tBJdRITQhbR-q4YFJLIzYw.png" /></figure><p>The above forward and backward propagation amounts to one pass through the training set.The snippet written above amounts to one epoch, which refers to one pass of the training set.When m = 5,000,000, batch gradient descent enables to take one step in gradient descent after passing through the entire training set.While in mini-batch gradient descent, i epoch amounts to taking 5000 gradient descent steps(as there are 5000 mini-batches).</p><h3>Intuition as to why mini-batch gradient descent works:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*Ak3eF6lqIRgerLfD4MTuYA.png" /></figure><p>In batch gradient descent the cost is expected to decrease on every iteration.If the cost goes up even a bit in a iteration then something is wrong and needs to be corrected(like the learning rate).</p><p>Whereas, in the mini-batch gradient descent when we plot the progress against the cost function, we find out that the cost does not decrease in every iteration.This is because each iteration is trained with X{t} and Y{t} i.e. each iteration sees a different mini-batch.So, the plot of J{t} is like below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/1*UWLFwIL0-jl5SVBKXmWDIg.png" /></figure><p>Of Course, the trend is downwards but a little noisy.The noise i.e oscillations maybe due to the reason that some min-batches maybe harder to train than others,which in turn increases the cost.Example: X{1},Y{1} could be an easy mini-batch, whereas X{2},Y{2} could be a hard one thereby increasing the cost.</p><p>Choosing the mini-batch size is very important.</p><pre>If mini-batch size = m (batch gradient descent)</pre><pre>If mini-batch size = 1 (stochastic gradient descent)(here every example is it&#39;s own mini-batch)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/325/1*uscs8X-CYiNX6Iyl6fPeNA.png" /></figure><p>Above is a contour plot . Blue lines indicate the batch gradient descent which take less noisy steps to the minimum.The violet lines denote the steps taken by the stochastic gradient descent which are a little noisy but fast.But the risk in stochastic gradient descent is that one wrong training example could lead to the wrong path.Stochastic gradient descent never converges,it leads to the path of the minimum and wanders around that region, but never converges at the minimum.</p><p>In practice, mini-batch size would be somewhere between 1 and m.This is because , batch gradient descent takes too long on every iteration if the dataset is large.On the other hand the main disadvantage with stochastic gradient descent is not the noise as it can be reduced with lower learning rate, but the fact that we would lose speedup due to vectorization as we are processing a single training example.So choosing in between gives faster learning.</p><p>Ok so that’s the end of first part.I am attaching the code snippet to compute mini-batch.Please leave your feedback.This is my first article on medium.</p><pre>def random_mini_batches(X, Y, mini_batch_size = 64, seed = 0):<br>    &quot;&quot;&quot;<br>    Creates a list of random minibatches from (X, Y)<br>    <br>    Arguments:<br>    X -- input data, of shape (input size, number of examples)<br>    Y -- true &quot;label&quot; vector (1 for blue dot / 0 for red dot), of shape (1, number of examples)<br>    mini_batch_size -- size of the mini-batches, integer<br>    <br>    Returns:<br>    mini_batches -- list of synchronous (mini_batch_X, mini_batch_Y)<br>    &quot;&quot;&quot;<br>    <br>    np.random.seed(seed)           <br>    m = X.shape[1]                  # number of training examples<br>    mini_batches = []<br>        <br>    # Step 1: Shuffle (X, Y)<br>    permutation = list(np.random.permutation(m))<br>    shuffled_X = X[:, permutation]<br>    shuffled_Y = Y[:, permutation].reshape((1,m))</pre><pre># Step 2: Partition (shuffled_X, shuffled_Y). Minus the end case.<br>    num_complete_minibatches = math.floor(m/mini_batch_size) # number of mini batches of size mini_batch_size in your partitionning<br>    for k in range(0, num_complete_minibatches):<br>        mini_batch_X = shuffled_X[:,k*mini_batch_size:(k+1) *mini_batch_size]<br>        mini_batch_Y = shuffled_Y[:,k*mini_batch_size:(k+1) *mini_batch_size]<br>        mini_batch = (mini_batch_X, mini_batch_Y)<br>        mini_batches.append(mini_batch)<br>    <br>    # Handling the end case (last mini-batch &lt; mini_batch_size)<br>    if m % mini_batch_size != 0:<br>        mini_batch_X = shuffled_X[:,math.floor(m/mini_batch_size)*mini_batch_size:(math.floor(m/mini_batch_size)+1) *mini_batch_size]<br>        mini_batch_Y = shuffled_Y[:,math.floor(m/mini_batch_size)*mini_batch_size:(math.floor(m/mini_batch_size)+1) *mini_batch_size]<br>        mini_batch = (mini_batch_X, mini_batch_Y)<br>        mini_batches.append(mini_batch)<br>    <br>    return mini_batche</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3a3109a3ed11" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>