<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ashik Poovanna on Medium]]></title>
        <description><![CDATA[Stories by Ashik Poovanna on Medium]]></description>
        <link>https://medium.com/@NoblesseCoder?source=rss-3fdaf344b12f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*GHjdKfVMnFWLvKkTEGP8zg.jpeg</url>
            <title>Stories by Ashik Poovanna on Medium</title>
            <link>https://medium.com/@NoblesseCoder?source=rss-3fdaf344b12f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 10:05:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@NoblesseCoder/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Solving Multi Armed Bandits (MAB) problem via ε-greedy agents]]></title>
            <link>https://medium.com/data-science/solving-multi-armed-bandits-mab-problem-via-%CE%B5-greedy-agents-298de2e69971?source=rss-3fdaf344b12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/298de2e69971</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ashik Poovanna]]></dc:creator>
            <pubDate>Tue, 12 May 2020 04:27:06 GMT</pubDate>
            <atom:updated>2020-05-12T04:27:06.899Z</atom:updated>
            <content:encoded><![CDATA[<h3>Solving Multi-Armed Bandits (MAB) problem via ε-greedy agents</h3><h4>In this article, We’ll design a Multi-Armed Bandit problem (as described in Reinforcement Learning: An Introduction — Sutton [1]) &amp; analyze how ε-greedy agents attempt to solve the problem.</h4><p>(This is my implementation &amp; revision of concepts from Chapter-2 of [1])</p><h3><strong>What are Multi-Armed Bandits?</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/0*jFV1aQ88ZsXajRgO.png" /><figcaption>A Slot Machine (1- armed bandit)</figcaption></figure><p>Before we talk about MABs, we need to understand what a <strong>1-armed bandit</strong> is.</p><p>A <strong>Slot machine </strong>that you see in a typical casino is another name for the 1-armed bandit. The name 1-armed comes from the lever/arm that one needed to pull to stop the spinning reels (Nowadays buttons to do the job) and bandits cause they loot you, duh!</p><p>These Slot machines have an RNG (Random Number Generator) chip implanted in the machine that generates a random combination of symbols. So a gambler invests some amount in the machine and plays the game. Some rare combinations are rewarded while most are not. All the payouts are done while maintaining an average payback percentage in the long run.</p><h4><strong><em>Should I gamble on slots [2], [3], [4]?</em></strong></h4><ol><li>Let&#39;s assume that the bandit has an average payback percentage = <strong>x %</strong>. The RNG chip is designed to maintain this average over its lifetime. The total number of possible combinations and the winning combinations are selected accordingly.</li><li>The bandit returns <strong>x%</strong> of the total money invested on the machine. It holds <strong>(100-x)%</strong>. That is the casino’s cut. To maximize its profits the casino traps more and more people into playing the game. The increase in investments results in increased profits.</li><li>Ex- If there is a slot machine with a 90% payback rate (or a 10% hold rate), then a total investment of 100$ on that machine will result in the bandit making a profit of only 10$. But if the total investment is 1,000,000$ then the bandit will end up looting 100,000$ in any case.</li><li>Naive Gamblers end up losing money on bandits cause they assume the machine to be sampling without replacement. So they think that by playing the same machine multiple times in succession they increase their chance of winning significantly. They end up withdrawing more and more money and fall prey to these ruthless bandits. Sadly, this is not the case since at each play the probability of getting a win remains the same.</li></ol><p>Now <strong>MABs</strong> (or k-armed bandits) can be thought of as multiple slot machines with different payback percentages or as a single slot machine with k-arms and each arm having a different payback percentage. The aim of a player/agent is to come up with a <strong>policy</strong> (In this case it is non-associative) for choosing an <strong>action</strong> (here the action is to decide which arm to pull) at each time step (t) such that the total reward at the end of some fixed time period (T) is maximized.</p><p>The MAB problem is a subclass of a more general class of <strong>Stochastic Scheduling problems</strong> [5] that deals with optimal resource allocations. Also its a single state MDP (Markov Decision process) problem.</p><p>Real-World Applications of MABs can be read here [6].</p><h3><strong>Designing the experiment</strong></h3><p>We are going to make some assumptions to simplify the experiment -</p><ol><li>We’ll assume that the k-armed bandits are <strong>stationary</strong>. What this essentially means is that the payout rates do not change over time.</li><li>We’re going to consider the problem in a <strong>non-associative</strong> setting i.e actions are independent of state.</li><li>Let the number of arms (k) =10. The action here is deciding which arm to pull (0–9).</li><li><strong><em>Some Notations</em></strong>: <strong>t</strong>- timestep, <strong>a</strong>- action (0–9), <strong>q*(a)</strong>- True value of the action (i.e the average reward associated with action(a)), <strong>Q_t(a)</strong>- Estimated action-value for action(a) at timestep (t), <strong>R_t</strong>- Rewards, <strong>A_t</strong>- Action taken at time (t).</li></ol><p>To simulate the workings of the MAB problem, [1] suggests sampling the true average action-value for all arms(actions) from a Standard Normal Distribution N(0,1). So the arm having a higher average action-value leads to better payouts in the long run. These values are hidden from the agents/players. To solve the problem we have to figure this out by playing the bandits multiple times. What is available to the agents are the immediate rewards and these are sampled from Normal distributions N(q*(a),1) for the respective arms/actions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*6i9t3y7zXeIqtnJwoTrcOw.jpeg" /><figcaption>Example Reward Distributions for the 10 arms (White dots represent the respective q*(a)s)</figcaption></figure><p>What makes this problem non-trivial is that at any point in time we don’t know what the true average action-values (q*(a)) for all actions are. If we knew that we could just choose the action (a) having the max q*(a). So we need to come up with some ways to estimate q*(a) at each time step.</p><h3><strong>Solving the problem</strong> (Agents)</h3><p>One way to solve the problem is via a <strong>Pure Greedy Action Selection Method</strong>. In this method, the agent always exploits the current knowledge (Q_t(a)s) to maximize the immediate reward (as shown in the below equation).</p><p><strong>A_t = argmax(Q_t(a)) , ∀ a ∈ A — (1)</strong></p><p>In <strong>Action-value estimation</strong> methods, the aim is to estimate the true action-values for each action (i.e Q_t(a)) and using this to decide which action to take in the given time step using equation (1).</p><p>Now according to [1] for the <strong>Stationary k-armed bandit problems</strong>, action-value estimation can be done using the two methods described below-</p><ol><li><strong><em>Sample Average Method (SAM)</em></strong>: Notice that here you require extra space to store past rewards for average calculations at each time step.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EoxUDEqHvFWNk5CTCGfPCQ.jpeg" /></figure><p><em>2. </em><strong><em>Incremental Update method (IUM)</em></strong>: It is a Mathematical Optimization of the previous SAM. Notice how the equation below is memory efficient since you don’t need to store history of past rewards and there is no need for average re-calculations at each time step. (I’ve used this in my code)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CWzjcxfWTlnzsNuLb8kyWA.jpeg" /></figure><ul><li>A point to note is that all the above-mentioned methods are biased by the initial selection of action-values. This bias allows one to provide prior info about what kinds of rewards are to be expected. Ex- <strong>optimistic bias</strong> (high initial action-values) <strong>encourages exploration earlier on &amp; then decreases it</strong>. The downside of bias is that it becomes a parameter that needs to be set by the user.</li></ul><p>The methods presented above creates a dilemma. If you solely chose the action based on the action-value estimates you will end up with what we call a <strong>pure greedy agent </strong>(The pure greedy agent always exploits the current knowledge to maximize its immediate reward). But now the question arises what if there was another untried arm that could have led to better total reward in the end but ended up not being selected (or explored) at the start due to lower initial action-values. This is where <strong>ε-greedy</strong> agents come in. They try to handle this dilemma.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/0*6bw0gugVn9sl2JrR" /><figcaption>From [1] ε-greedy algorithm</figcaption></figure><p>As described in the figure above the idea behind a <strong>simple ε-greedy bandit algorithm</strong> is to get the agent to explore other actions randomly with a very small probability (ε) while at other times you go with selecting the action greedily. It can be asymptotically shown that the estimated action values converge to true action-values, but the practical effectiveness of the algorithm itself is unknown.</p><p>Another alternative to the dilemma of explorations v/s exploitation is to use the <strong><em>Upper Confidence Bound (UCB)</em> Action-Selection Method</strong>. In this approach during exploration instead of randomly selecting from available action space, UCB takes into account the uncertainties (with (c&gt;0) determining the confidence levels of the estimate that decide the degree of exploration) in those estimates. But this method is not practically found to scale well beyond the MAB problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YshjXYS-qcIKUXvlWeRWEQ.jpeg" /></figure><h4><strong>Code</strong></h4><ol><li>Install the <a href="https://github.com/NoblesseCoder/gym-karmedbandits">k-armed bandits environment</a>.</li><li>Head over to <a href="https://github.com/NoblesseCoder/reinforcei">reinforcei</a>. This repo contains the code for the agents and their respective demos.</li></ol><h4><strong>Solution Analysis</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*JULmxMqsJevScRkU4m2e8Q.png" /></figure><p>The plot above compares the performance of different agents on the same 10-armed bandit problem. The statistics are obtained by averaging over 1000 such runs.<br>Some other inferences that hold for asymptotic/more episodes, different experimental settings &amp; might not be visible here:</p><ol><li>ε=0.01 is bound to perform better than ε=0.1 in the long run. And both perform better than ε=0.</li><li>Instead of static (ε) designing the (ε) to reduce/decay over time leads to better performance.</li><li>If rewards are noisier i.e variance in the sampling distributions are increased ε-greedy agents do way better than pure-greedy agents. Whereas if we were to reduce the variance to 0 pure greedy would be the way to go since at each time step the estimated q(a) would equal true q*(a).</li><li>High initial biases are useful to encourage exploration in the early stages of stationary MAB problems.</li><li>UCB performs better than epsilon-greedy for stationary MAB problem.</li></ol><h4><strong>Side Note</strong></h4><p>Thank you for reading the article. I hope it was useful. Any corrections or advice is welcome. I hope to improve my understandings of RL from the grass-root levels. I plan to cover Non-Stationary MAB and Gradient Bandit Algorithm in the next article.</p><h4>References</h4><p>[1] <a href="https://mitpress.mit.edu/books/reinforcement-learning-second-edition">https://mitpress.mit.edu/books/reinforcement-learning-second-edition</a> : All the algorithms and equations &amp; derivations except the code are from this book.</p><p>[2] <a href="https://youtu.be/7Wkubf1PrWg">https://youtu.be/7Wkubf1PrWg</a></p><p>[3] <a href="https://youtu.be/BFlRH99TQOw">https://youtu.be/BFlRH99TQOw</a></p><p>[4] <a href="https://youtu.be/YBQhCP1IdWo">https://youtu.be/4wzg-8QKC5s</a></p><p>[5] <a href="https://en.wikipedia.org/wiki/Stochastic_scheduling">https://en.wikipedia.org/wiki/Stochastic_scheduling</a></p><p>[6]<a href="https://www.researchgate.net/publication/332604222_A_Survey_on_Practical_Applications_of_Multi-Armed_and_Contextual_Bandits">https://www.researchgate.net/publication/332604222_A_Survey_on_Practical_Applications_of_Multi-Armed_and_Contextual_Bandits</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=298de2e69971" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/solving-multi-armed-bandits-mab-problem-via-%CE%B5-greedy-agents-298de2e69971">Solving Multi Armed Bandits (MAB) problem via ε-greedy agents</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tic-Tac-Toe Learner AI]]></title>
            <link>https://medium.com/data-science/tic-tac-toe-learner-ai-208813b5261?source=rss-3fdaf344b12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/208813b5261</guid>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Ashik Poovanna]]></dc:creator>
            <pubDate>Sat, 03 Aug 2019 08:15:37 GMT</pubDate>
            <atom:updated>2019-11-15T21:00:18.044Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://en.wikipedia.org/wiki/Tic-tac-toe">Tic-Tac-Toe</a> is a simple game for two players that we enjoyed playing as kids (especially in boring classrooms). The game involves 2 players placing their respective symbols in a 3x3 grid. The player who manages to place three of their symbols in horizontal/vertical/diagonal row wins the game. If either player fails to do so the game ends in a draw. If both the people always play their optimal strategies the game always ends in a draw.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*YgslEHb5SVUr3rLw7-vagA.png" /><figcaption>Ex: Tic-Tac-Toe Board</figcaption></figure><p>Since the grid is small &amp; there are only two players involved the number of possible moves for every board state is limited thus allowing Tree-based search algorithms like Alpha-Beta pruning to provide a computationally feasible &amp; exact solution to building a Computer-based Tic-Tac-Toe player.</p><p>In this article, we look at an approximate (Learning-based) approach to the same game. Even though a better algorithm exists (i.e. <a href="https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning">Alpha-beta pruning</a>), the approximate method provides an alternative method that might be useful if the complexity of the board were to increase. Also, the code changes to incorporate that would be minimal.</p><p>The idea is to pose the <strong>Tic-Tac-Toe game as a well-posed learning problem</strong> as mentioned in <a href="https://www.amazon.com/Learning-McGraw-Hill-International-Editions-Computer-dp-0071154671/dp/0071154671/ref=mt_paperback?_encoding=UTF8&amp;me=&amp;qid=">Tom Mitchell’s Machine Learning Book</a> (Chapter -1). The learning system is described in brief in the following section.</p><h3>Tic-Tac-Toe Learning System</h3><p>The basic idea behind the learning system is that the system should be able to improve its performance (P) w.r.t to a set of tasks (T) by learning from training experience (E). The training experience (E) can be a direct (predefined set of data with individual labels) or indirect feedback (No labels for each training example). In our case:-</p><ul><li><strong>Task (T)</strong>: Playing Tic-Tac-Toe</li><li><strong>Performance (P)</strong>: Percentage of Games won against Humans</li><li><strong>Experience (E)</strong>: Indirect feedback via solution trace (Game History) generated from games played against itself (a clone)</li></ul><p><strong>Learning from Experience(E)</strong>: Ideally, a function (Ideal Target Function) needs to be learned that gives the best move possible for any given board state. In our problem, we represent our ITF to be a Linear function (V) that maps a given board state to real value (score). Then we use an approximation algorithm (<a href="https://danieltakeshi.github.io/2015-07-29-the-least-mean-squares-algorithm/">Least Mean Squares</a>) to estimate the ITF from the solutions traces.</p><ol><li>V(boardState) → R, (R-Score value for a given boardState.)</li><li>V_hat(boardState) ←(W.T) * X , (W- Weights of the Target function, X- Features extracted from given boardState.)</li><li><strong>LMS training rule to update weights</strong>: Wi ←Wi + lr * (V_hat(boardState)-V_hat(Successor(boardState))) * X, (i- ith training example, lr- Learning rate)</li></ol><p>The score (R) for each non-final board state is assigned with the estimated score of the successor board state. The final board state is assigned a score based on the end result of the game.</p><p>3. V(boardState) ←V_hat(Successor(boardState))<br>4. V(finalBoardState) ←100 (Win) | 0 (Draw) | -100 (Loss)</p><h3>Implementation</h3><p>The Final Design is split into four modules (Ch-1, <a href="https://www.amazon.com/Learning-McGraw-Hill-International-Editions-Computer-dp-0071154671/dp/0071154671/ref=mt_paperback?_encoding=UTF8&amp;me=&amp;qid=">Tom Mitchell’s Machine Learning Book</a>):-</p><ol><li><strong>Experiment Generator</strong>: Its job is to generate new problem statements at the beginning of every training epoch. In our case, it just returns an empty initial board State.</li><li><strong>Performance System</strong>: This module takes as input the problem provided by the Experiment Generator &amp; then uses the improved Learning algorithm to produce a solution trace of the game at every epoch. In our case, this is done by simulating a Tic-Tac-Toe game between two players (cloned programs). These Players make move decisions using the current Target function.</li><li><strong>Critic</strong>: This module takes the solution trace and outputs a set of training examples to be input to the Generalizer.<br>Training Examples ←[&lt;Features(boardState) , R &gt;,&lt;&gt;….]</li><li><strong>Generalizer</strong>: This module uses the training examples provided by the critic to Update/Improve the Target Function by learning the desired weights using the LMS weight update rule at each epoch.</li></ol><p>The following is a small video showing the training &amp; testing phases ( BTW: My video editing skills are not good).</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FVASUwzqs6OU%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DVASUwzqs6OU&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FVASUwzqs6OU%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/622044190cdff121c98026ebc0c7dca3/href">https://medium.com/media/622044190cdff121c98026ebc0c7dca3/href</a></iframe><p>The code for the above implementation can be found at this Github gist: <a href="https://gist.github.com/NoblesseCoder/cccf260ecc3e2052a0a1a013a7f7ac54">https://gist.github.com/NoblesseCoder/cccf260ecc3e2052a0a1a013a7f7ac54</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=208813b5261" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/tic-tac-toe-learner-ai-208813b5261">Tic-Tac-Toe Learner AI</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An Anime inspired ACL injury]]></title>
            <link>https://medium.com/@NoblesseCoder/an-anime-inspired-acl-injury-393083989ade?source=rss-3fdaf344b12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/393083989ade</guid>
            <category><![CDATA[sports]]></category>
            <category><![CDATA[health]]></category>
            <category><![CDATA[anime]]></category>
            <category><![CDATA[recovery]]></category>
            <category><![CDATA[basketball]]></category>
            <dc:creator><![CDATA[Ashik Poovanna]]></dc:creator>
            <pubDate>Mon, 03 Jun 2019 00:59:43 GMT</pubDate>
            <atom:updated>2019-06-03T01:01:12.915Z</atom:updated>
            <content:encoded><![CDATA[<p><em>The following is a short read on ACL Injury &amp; Recovery.</em></p><p>I don’t know how many of you out there watch anime. But those who do would have definitely heard <strong>Kuroko no Basuke</strong>. In this anime, a couple of Japanese High Schoolers pull out some insane NBA level (Actually they overpower NBA) basketball moves. I tried to perfect copy one of the player&#39;s tricks that involved me shooting for the hoops from behind the board (Side Note: I play point guard &amp; shooting guard positions). I don’t remember now if it was a faulty jump or a bad landing, but it felt as if my femur had come out of the joint. The pain was terrible. The whole knee region got swollen up. My friends took me to a nearby Hospital, but the doctor there seemed oblivious to the symptoms of the swelling &amp; tear. Even I as a non-medic knew that there was a fluid leak &amp; it had to be drained.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ft-bYhkcA1sxxCp2HkAHpw.jpeg" /><figcaption>Link: <a href="https://myanimelist.net/anime/11771/Kuroko_no_Basket">https://myanimelist.net/anime/11771/Kuroko_no_Basket</a></figcaption></figure><p>Somehow I managed to reach my parents&#39; place the next day. There was a lot of instability in my movements. The doctor over there drained the joint fluid (It nearly filled six syringes) &amp; put my knee in a cast &amp; recommended a rest period of two weeks. But the instability continued to show up in my knee movements. So the doctor referred me to Dr. Rajkumar Amaravati (St. John’s Hospital, Bangalore) as he mainly deals with Orthopedic &amp; Sports Surgeries. The Doctor explained with the MRI results that there was a complete tear of the ACL coupled with a partial Meniscus Tear and a Surgery needed to be done as soon as possible. The same was also confirmed via Lachman &amp; Anterior Drawer Tests.</p><h3>What is an ACL Tear?</h3><p>The ACL or Anterior Cruciate Ligament is one of the pair of major ligaments in the knee that crosses from the underside of the femur to the top of the tibia. They assist in controlling excessive motion by limiting the mobility of the knee joint. They are basically prone to tear when any kind of excessive rotation (like pivoting &amp; then jumping in basketball) causes the femur &amp; tibia to move in the opposite direction thus causing a tear.</p><p>ACL Tear is considered to be a common sports injury especially in games like Basketball which involves a combination of pivoting, sudden stopping &amp; irregular jumps that puts immense stress on the muscles &amp; ligaments in the joint area.</p><p>The degree of tear can be analyzed from MRI scans but the common symptoms involve — a feeling of instability (giving-away) in the knee movements, Severe pain in the joint, rapid swelling (caused by the Joint fluid leak). In my case, all these symptoms added up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*6oY2iBKIFygQgVWebgm7Xw.jpeg" /></figure><h3>Arthroscopic Reconstructive Surgery</h3><p>In my case, it was an autograft Reconstructive surgery (supported by Arthroscopic video feeds ) in which the torn ACL is completely removed &amp; is replaced by a piece of tissue extracted from my Hamstrings. The Meniscus had to be tied as well (In my case this was experimental &amp; I had to fill up surveys for a case study on the same).</p><p>Before the surgery, the Doctor had asked me to read up on the procedure &amp; come up with some questions. The internet has some scary stuff when you try to look upon health-related problems. There was stuff like if the fluid was not drained in time than it might lead to cancer &amp; so on (Nowadays anything you search in the net ultimately routes you to cancer). I just stuck to the physiology reports from then on.</p><p>I wasn’t a tiny bit scared about the surgery, it was the post-surgery pain that I dreaded. I checked in to the OT at 12:30 &amp; came out at 4:00. I don’t remember much about the surgery as I was sedated (Before the surgery I actually wanted to see the surgery in the monitor while it was being performed). Then the next few days were the most physically uncomfortable days of my life. There was excruciating pain &amp; stiffness in the whole leg. For the first few hours, I felt nothing, but when the effectiveness of the anesthetics ran out then the pain started erupting out of nowhere. I couldn’t scratch it &amp; I couldn’t move it. The ice packs helped a bit. But I couldn’t sleep for three whole days. I only felt better when the Physiotherapy started and I could do some minor movements with my legs.</p><h3>Physiotherapy &amp; Recovery</h3><p>Physiotherapy is as important as the surgery for complete &amp; Successful recovery. The exercises lasted for almost a year. They come in various levels &amp; the progress is periodically tested. These exercises were essential to get the leg muscles back into shape &amp; strengthen the ligaments. The one set of exercises I clearly remember involved the use of these colored bands (Each color indicated a certain tension of the bend). The exercise basically involved a set of alternative straightening &amp; bending of the knee. They evaluated the angle of the bends weekly to check my progress. During this period I walked with the support of knee braces.</p><p>After a few months once the stitches were removed I started Swimming. It was suggested that swimming would help with a faster recovery. Slowly within a period of eighteen months, I got back to playing basketball &amp; began performing activities at a regular pace.</p><h3>Post-Recovery Performance</h3><p>I am able to do my daily Activities with absolutely no loss in performance whatsoever. But with Basketball &amp; knee intense games I probably perform up to 95% of my original levels. One more thing besides physical recovery is the mental one. For a period of time, I was afraid of playing thinking that the tear might happen again. This is one feeling that we have to nip in</p><p>Two years later I had a sensation of deja vu with my other leg (Partial Tear Not so severe). But this is another story by itself.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=393083989ade" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Defence Against the Data Arts : Python v/s R]]></title>
            <link>https://medium.com/data-science/defence-against-the-data-arts-python-v-s-r-5f4529c1d90f?source=rss-3fdaf344b12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f4529c1d90f</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[r-programming]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Ashik Poovanna]]></dc:creator>
            <pubDate>Fri, 20 Apr 2018 17:57:58 GMT</pubDate>
            <atom:updated>2018-04-22T11:56:31.865Z</atom:updated>
            <content:encoded><![CDATA[<h4>Scene (Harry Potter reference) - <strong>Dumbledore</strong> wields a wand having an abstract <strong>python</strong> core and Grindelwald uses a wand with a classical <strong>R</strong> core.Both are equally skilled.<strong>Will the wand-core create a difference of power in the data world???</strong> Lets find out.</h4><p>Before we begin the greatest duel in the history of Defence Against the Data Arts lets checkout their respective wands…</p><p>On one side we have the <strong>2 </strong>time world wand champion (2007 &amp; 2010)-<strong>Python </strong>.It is currently<strong> </strong>ranked <strong>4th</strong> according to TIOBE index and has moved up a place from 2017.It was created by a famous wizard who goes by the name <strong>Guido van Rossum</strong> (1991) to support charms using multiple programming paradigms , readability and support high levels of abstraction.</p><p>For those interested in the Python wand-lore -&gt; <a href="https://en.wikipedia.org/wiki/Zen_of_Python">https://en.wikipedia.org/wiki/Zen_of_Python</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*WaBY8zKKbTTmh9zi.png" /></figure><p>The other mighty elder wand <strong>R</strong> has moved 3 places up the TIOBE index and has claimed the <strong>12th</strong> seat as of this April 2018. It was created by the two wizards Ross Ihaka &amp; Robert Gentleman with the <strong>only aim of winning a statistical duel</strong>.It provides charms for statistical computing and graphics. Its special enchantments allow it to perform statistical modelling, classical statistical tests, time-series analysis, classification, clustering and graphical techniques.</p><h3><strong>Round One: Fan base &amp; Scientific Community</strong></h3><blockquote>You basically go where the crowd goes.</blockquote><p>When I talk about Python in this article it is not about the generic language capabilities as a whole(That would be unfair) but instead a subset of packages(numpy,scipy etc..) built using the same to meet the requirement of ever growing statistical problems in this age of Big Data.</p><p><strong>Python is more popular than R</strong> according to TIOBE ratings.</p><p>What makes python more popular is its <strong>generic programming styles (Zen of Python)</strong>, <strong>faster learning curve </strong>and the most important reason of all is the <strong>excess of high level packages and frameworks for Machine &amp; Deep Learning (SK-learn,Keras,Pytorch,Tensorflow etc…)</strong>.You perform Data Analysis with an end goal of making a decision so if the decision making(DM) tool itself is in Python you would prefer to do the analysis part in the same as it is easier to integrate with the DM algos in the future. R also has equivalent frameworks but are not as well known.</p><h3>The scientific Community is good in both the cases.But again popularity herds us like sheep towards the Pythonic side.Python Wins!!!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/0*i_qnxBNzMmtwCfmZ." /><figcaption>credits:xkcd</figcaption></figure><h3><strong>Round Two: Beginner&#39;s Luck</strong></h3><p>As I mentioned earlier Python is very intuitive where as R has a very steep learning curve.The first time I used python I loved it. Everything was so easy.I thought I could conquer the software world with it.But I was naive.The C course in the next semester showed me my place.(Who teaches C just after Python?).</p><p>Even though I had better programming knowledge by the time I came across it ,R was very troublesome.In the beginning it took tripple the amount of time to cookup a code(checking out packages and different constructs were a pain) than it would take for an equivalent Python code.As you familiarise yourself with the R coding style it gets better but I still suffer due to non-intuitive constructs in R.Its probably feel so because I’m <strong>addicted to</strong> <strong>python.</strong></p><p><strong>Ipython notebook makes coding cleaner and easier (magic cmds).It supports both R and Python Kernels.</strong></p><p><strong>Nonetheless I favour Python in this round.</strong></p><h3>Round Three: Beauty &amp; the Beast</h3><p>Anyone who has used both ggplot2(&amp; others) and matplotlib.pyplot will probably agree with me when I state that the latter is a crude beast when compared to the former.There are other Python visualisation packages like seaborn which are good but not upto R’s standards.</p><p><strong>R sweeps this round!!!</strong></p><h3>Round Four: Data Analysis Power</h3><p>If we are talking in terms of statistical computing and modelling powers <strong>R is clearly superior to Python</strong>.It has a lot more verified statistical computing packages that are yet to be implemented in Python.Python is closing the gap every hour.</p><p>So the question arises- <strong>Is R a dying language?</strong></p><p>No this is not true.<strong>R is the language of the learned</strong>-<strong>Statisticians ,Researchers in medicine &amp; life sciences </strong>etc find R to be a better tool than python.The reason could be thought of as a result of superior statistical packages and customs.</p><p>The question mostly arises in our mind due to a <strong>booming hype created around Deep Learning and Neural Networks</strong> where Python dominates the area and creates a feeling that R is loosing its place in the field as Deep Learning is closely associated with Data Science(&amp; Analytics).<strong>Its like a Decoy effect.</strong></p><p>I conducted a poll in Facebook (AI &amp; DL group) to verify the above reasoning and that resultant stats agreed with the reasoning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/1*LmK0g3aG3ckVk4VoD4V8Nw.png" /></figure><p><strong>So R manages to triumph over python in this round.</strong></p><h3>Deciding the Winner ?</h3><p>Like the wizards themselves (before the famous duel-Harry potter reference) there is no clear proof indicating the superiority of one over the other.</p><p>Like a diplomatic politician I’d say that if your aim is classical statistical analysis then go with R and if your aim is Machine Learning than go with Python as the support is better and coding easier.</p><p>But I believe that for a Data Science and ML enthusiast its <strong>good to be a jack of both the trades</strong>.</p><p>I tried integrating fictional aspects to bring humour to this article.Hope you enjoy it.Please do comment if there are any important characteristics that I could have missed as I cooked up the article in my class hours.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f4529c1d90f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/defence-against-the-data-arts-python-v-s-r-5f4529c1d90f">Defence Against the Data Arts : Python v/s R</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A peek into Genetic Algorithms]]></title>
            <link>https://becominghuman.ai/a-peek-into-genetic-algorithms-6cf2d9a01b1a?source=rss-3fdaf344b12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/6cf2d9a01b1a</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python3]]></category>
            <category><![CDATA[genetic-algorithm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Ashik Poovanna]]></dc:creator>
            <pubDate>Mon, 12 Mar 2018 19:26:39 GMT</pubDate>
            <atom:updated>2019-11-15T21:08:23.573Z</atom:updated>
            <content:encoded><![CDATA[<p>Last week we had a class on genetic algorithms.I had heard that these class of Evolutionary algorithms help in deciding or optimising parameters of ANNs,but I had never really paid attention to the logic or the math behind them.So as the class proceeded I found the idea of these algorithms to be very different from the machine learning algorithms I had coded so far ,where you have train-dev-test sets and you perform the task of function approximation on the same.So I wen’t through the corresponding chapter in Tom Mitchell and surfed the net for some interesting bits &amp; blobs.This article is a result of that.</p><h3><strong>Evolutionary Algorithms &amp; Biological Inspiration.</strong></h3><p>Evolutionary algorithms are a set of generic meta-heuristic algorithms<br>which solve optimisation problems by imitating aspects of biological evolution.<strong>Genetic algorithms </strong>are a subset of Evolutionary algorithms inspired by <strong>Charles Darwin&#39;s</strong> work on evolution by natural selection.They are meta heuristic search algorithms relying on bio-inspired operators such as <strong>mutation</strong>,<strong>crossover</strong> and <strong>selection.</strong>Genetic algorithms in particular became popular through the work of John Holland in the early 1970s, and particularly his book <em>Adaptation in Natural and Artificial Systems</em> (1975).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/514/0*_tbeJDGnioXKSmf-.gif" /><figcaption>Meiosis &amp; Mitosis (pic:quia)</figcaption></figure><p>The above picture depicts the process of <strong>Meiosis</strong> and<strong> Mitosis</strong> where in the former you have <strong>crossovers of chromosomes</strong> forming <strong>new genotypes</strong> and the latter involves <strong>copying of genetic information</strong> to new <strong>offspring</strong>.The other biological phenomenon of <strong>mutation</strong> can be thought of as <strong>errors</strong> during the copying process.Down the lane we can observe that the mutation,crossover and selection of <strong>encoded bit strings of features</strong> of the data are<strong> analogous</strong> to these biological operations to a certain extent.</p><h3>Top 3 Most Popular Ai Articles:</h3><blockquote><a href="https://becominghuman.ai/digital-processes-inspiring-analog-paintings-a358eb7801a0">1. How I used machine learning as inspiration for physical paintings</a></blockquote><blockquote><a href="https://becominghuman.ai/ms-or-startup-job-which-way-to-go-to-build-a-career-in-deep-learning-2d62a5b10f9e">2. MS or Startup Job — Which way to go to build a career in Deep Learning?</a></blockquote><blockquote><a href="https://becominghuman.ai/top-100-medium-com-c2695ab3270c">3. TOP 100 medium articles related with Artificial Intelligence / Machine Learning’ / Deep Learning (until Jan 2017)</a></blockquote><h3><strong>Genetic Algorithms(GA): The Learning process</strong></h3><p>GA is also referred to as <strong>Generate-Test Beam &amp; Search</strong> Algorithm.To explain this my professor told us to imagine it as searching for an object in a darkroom by throwing a beam of light randomly with the hope of hitting the object.We need to understand some basic Jargon before proceeding in the same:-</p><p><strong>1.Hypotheses(h)</strong>- With respect to GA they are usually fixed length bit strings or symbolic expressions and in case of genetic programming they are computer programs.Bit strings are preferred for easier manipulation during crossover and mutation.</p><p><strong>Feature Representation</strong>-Lets assume that we are given a constrained equation- <strong>a+2b+3c=14 where a,b,c&gt;0 </strong>and we have to find value of a,b &amp; c.For this the features are variables a,b,c.We can clearly see that the max value that any of them can take is 14 which can be represented by 4 bits.So we can represent hypotheses of the population as a binary string with fixed length of 4 bit per feature(total length=12 bits) .Let a hypothesis have a=1,b=2,c=2.</p><p>The bit string of the hypothesis is- 0001 0010 0010</p><p><strong>2.Population(P)</strong>- It is a set of initial Hypotheses which undergo operations every generation to give rise to a probably better(evolved) population.The size of population is constant every generation.</p><p><strong>3.Generation</strong>- It refers to the number of iterations of the algorithm(Stages of population modification).Ex-1st iter results in the formation of 1st generation.</p><p><strong>4.Selection</strong>-It is the process of probabilistically selecting <strong>n(1-r) </strong>members out of a population(<strong>p</strong>) whose size is <strong>n</strong> to be added to new population(<strong>P’</strong>) whose size is also <strong>n</strong> .Probability of selecting a hypotheses is given by-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NrpWMBQlXPI2dUPw.png" /></figure><p><strong>5.Fitness function</strong>-It is a criterion used to decide selection of hypothesis for further generations(or operations).For example in case of solving linear equations the fitness function can be one that gives absolute difference from the value(see code).</p><p><strong>6.Crossover(Analogous to meiosis operation)</strong>-Probabilistically selecting <strong>r*n/2 pairs</strong> of hypotheses from P according to equation-1 and then applying crossover operator to generate 2 offspring per pair and adding them to P’.Which bits are to be selected from which hypothesis is given by the <strong>crossover mask</strong>.There are multiple types of crossover operators namely-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KQr16hiOpv92V4eysLiigw.jpeg" /><figcaption>crossover operators</figcaption></figure><p><strong>7</strong>. <strong>Mutation(Analogous to biological Mutation)</strong>-Invert one(<strong>point mutation</strong>)/many randomly selected bit in hypothesis.</p><h3>Genetic Algorithm Flowchart</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gG4H_qS2MXa8a2RxK-962A.jpeg" /></figure><p><strong>Advantages of GA-</strong></p><ol><li>Inherently parallel</li><li>Good in noisy environments</li><li>Gives a set of possible or close by Solution set.</li><li>Easy to understand.</li></ol><p><strong>Disadvantages-</strong></p><ol><li>They don’t scale well with complexity.</li><li>They may not converge to a perfect solution.</li></ol><p>Thus they are better used as a last resort and mainly for optimising parameters and topology .I tried coding an example of using GA to solve linear equations.Link- <a href="https://gist.github.com/NoblesseCoder/d35e19e441ed014ea69ff800be28e748">https://gist.github.com/NoblesseCoder/d35e19e441ed014ea69ff800be28e748</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f5f8b%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F8f5f8b&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href">https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href</a></iframe><figure><a href="https://becominghuman.ai/artificial-intelligence-communities-c305f28e674c"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*2f7OqE2AJK1KSrhkmD9ZMw.png" /></a></figure><figure><a href="https://upscri.be/8f5f8b"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*v-PpfkSWHbvlWWamSVHHWg.png" /></a></figure><figure><a href="https://becominghuman.ai/write-for-us-48270209de63"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*Wt2auqISiEAOZxJ-I7brDQ.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6cf2d9a01b1a" width="1" height="1" alt=""><hr><p><a href="https://becominghuman.ai/a-peek-into-genetic-algorithms-6cf2d9a01b1a">A peek into Genetic Algorithms</a> was originally published in <a href="https://becominghuman.ai/">Becoming Human: Artificial Intelligence Magazine</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Slice Of Life: Top 5 Anime Movies]]></title>
            <link>https://medium.com/@NoblesseCoder/slice-of-life-top-5-anime-movies-4bd2857a33aa?source=rss-3fdaf344b12f------2</link>
            <guid isPermaLink="false">https://medium.com/p/4bd2857a33aa</guid>
            <category><![CDATA[anime]]></category>
            <category><![CDATA[slice-of-life]]></category>
            <category><![CDATA[anime-review]]></category>
            <category><![CDATA[love]]></category>
            <category><![CDATA[movies]]></category>
            <dc:creator><![CDATA[Ashik Poovanna]]></dc:creator>
            <pubDate>Tue, 30 Jan 2018 21:19:44 GMT</pubDate>
            <atom:updated>2018-01-30T21:25:14.946Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article I’ll be sharing my views on what I feel are the best 5 anime movies of all time in the above mentioned genre.</p><p><strong>Spoiler Alert</strong> : I end up giving a snapshot of the movies.</p><h3><strong>1. Wolf children (</strong><em>Ōkami Kodomo no Ame to Yuki</em><strong>)</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*diETz9uRqE3oqzS33Uurvw.jpeg" /></figure><p>I completely loved the anime from the start to the end.I am a big fan of anime movies that allure you with the rustic charms of the countryside.This is one such masterpiece.The story is about how a college student falls in love with a werewolf,live together and then suddenly the husband i.e werewolf dies leaving her to take care of the two baby werewolves without a clue on how to do so.She faces the challenge of raising the half breeds while the latter have a difficult choice to make-Human world or the wild one?</p><p>Trailer- <a href="https://www.youtube.com/watch?v=ed1xwAtF728">https://www.youtube.com/watch?v=ed1xwAtF728</a></p><h3><strong>2. <em>5 Centimeters per Second(</em> <em>Byōsoku Go Senchimētoru)</em></strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s0DJmUGk-uGOAzQmg3o-MA.jpeg" /></figure><p>This is a movie in the form of poetry by Makoto Shinkai. Its the most beautiful thing I have ever seen in the anime movie universe.It’s a romantic drama where a boy and girl meet and fall in love in elementary school,eventually drifting apart,meet momentarily before finally break apart.The movie throws light on ever changing situations in life and how at the end all that remain are those beautiful memories you made on the way.</p><p>Trailer-<a href="https://www.youtube.com/watch?v=wdM7athAem0">https://www.youtube.com/watch?v=wdM7athAem0</a></p><h3>3. Your Name(Kimi no Na Wa)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7cA1uz3PkYrc5F3PxRQ_OQ.png" /></figure><p>This is yet another romantic drama by Makoto Shinkai. I prefer to not reveal the plot of this movie.Its a combination of magic and romance.Here by magic I mean a phenomenon of switching bodies between a boy and a girl whose destinies are tangled by fate.</p><p>Trailer- <a href="https://www.youtube.com/watch?v=hRfHcp2GjVI">https://www.youtube.com/watch?v=hRfHcp2GjVI</a></p><h3>4. Kaze no Matasaburou</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*wyjhJcbnjDkW_C34jama5Q.png" /></figure><p>This movie’s animation is like a moving painting.The plot has a transfer student from a big city moving to a Japanese country side.There is an air of mystery around him with his classmates suspecting him to be Matasaburou-the wind imp from a local legend. Why?How? are the plots of the movie set in a beautiful picturesque.</p><h3>5. The Tale of princess Kaguya (Kaguya-hime no Monogatari)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jK9Q12D0OS08BBOhKh9DuQ.jpeg" /></figure><p>This movie is based on a famous Japanese folktale “The tale of the Bamboo Cutter” where a childless Bamboo cutter magically ends up finding a child in a bamboo shoot and eventually finding gold in the same place.The story shows how princess Kaguya(called so because of her godly birth) grows up quickly like that of a bamboo and how the old couple try to raise her as a princess against her will and eventually causing her to go back to her rightful place the moon to match her glorious beauty.This movie has an enchanting effect that is true in case of the popular fairy tales.</p><p>link- <a href="https://www.youtube.com/watch?v=9lDrkokymLQ">https://www.youtube.com/watch?v=9lDrkokymLQ</a></p><p>The other 2 movies that I debated including in the top 5 list are-</p><p><strong><em>From Up on Poppy Hill(Kokuriko-zaka Kara) &amp;Whisper of the Heart(Mimi o Sumaseba)</em></strong></p><p>This is my list.If you guys/girls feel otherwise please do tag those that you think should make the top-5.I’d love to watch them and debate.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4bd2857a33aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>