<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kruthi M on Medium]]></title>
        <description><![CDATA[Stories by Kruthi M on Medium]]></description>
        <link>https://medium.com/@kruthim?source=rss-73f8e3acfaa4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Ybrf9w-TY24a4XYcW8VGzw.jpeg</url>
            <title>Stories by Kruthi M on Medium</title>
            <link>https://medium.com/@kruthim?source=rss-73f8e3acfaa4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 08:29:51 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kruthim/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[From AlphaGo to Self-Driving Cars: Understanding the Basics of Reinforcement Learning]]></title>
            <link>https://medium.com/data-science-community-srm/from-alphago-to-self-driving-cars-understanding-the-basics-of-reinforcement-learning-4355103b344?source=rss-73f8e3acfaa4------2</link>
            <guid isPermaLink="false">https://medium.com/p/4355103b344</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Kruthi M]]></dc:creator>
            <pubDate>Fri, 31 Mar 2023 15:12:42 GMT</pubDate>
            <atom:updated>2023-03-31T18:28:57.419Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Beginner’s Guide to RL!</h4><p>Are you ready to explore the cutting edge of artificial intelligence? Look no further than the exciting world of <strong>Reinforcement Learning</strong>! From <strong>AlphaGo’s</strong> stunning victory over the world champion to S<strong>elf-Driving Cars</strong> navigating through busy city streets, reinforcement learning is transforming the way we think about intelligent systems.</p><h3><strong>So, What Is Reinforcement Learning?</strong></h3><blockquote>Reinforcement Learning (RL) is a type of machine learning that trains an AI to learn by <strong>trial and error</strong>. An artificial intelligence (AI) agent interacts with its environment, receives feedback in the form of rewards or penalties, and gradually improves its decision-making abilities. <strong>Maximizing</strong> the agent’s overall reward is the goal.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/0*QrhmCxr02pNPeL6B.png" /><figcaption>Fig 1: Illustration of an RL system</figcaption></figure><p>There are four primary components in <strong>Reinforcement Learning:</strong></p><ol><li><strong>Agent:</strong> The entity which has to formulate the decisions.</li><li><strong>Action:</strong> The events performed by an entity.</li><li><strong>Environment:</strong> The scenario in which an agent performs an action. It’s bound by a set of rules where the environment responds for every action.</li><li><strong>Reward: </strong>Feedback mechanism that notifies the agent about the success rate.</li></ol><h3><strong>Exploration Vs Exploitation Trade off:</strong></h3><p>In simple terms, <strong>Exploration</strong> is the process of trying out new actions in order to learn more about the environment and the rewards associated with different actions. <strong>Exploitation</strong>, on the other hand, is the process of choosing actions that have worked well in the past, in order to maximize the reward.</p><h3><strong>Real World Example to Understand Reinforcement Learning Model:</strong></h3><p>Imagine you have a pet dog named Jack who you want to teach to catch a ball.</p><p><strong>Agent</strong> — The pet dog ; Jack</p><p><strong>Environment </strong>— Game of catching the ball</p><p><strong>Exploration Phase — </strong>At first, Jack might not know how to catch a ball, so he will try different things, like jumping or biting, to see what works.</p><p><strong>Reward —</strong> Jack catching a ball ( A dog treat or a pat on the head )</p><p><strong>Exploitation Phase — </strong>He will remember what he did to catch the ball. The next time you throw the ball, Jack will try to do the same thing again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*Qac9GElfrqphYfd9zq19Yg.jpeg" /></figure><p>As a result, RL is a way to teach an animal or machine to learn from its experiences and improve its decision-making abilities. The agent interacts with an environment, receives feedback in the form of rewards or penalties, and learns to make better decisions over time.</p><h3><strong>Model Free vs Model Based :</strong></h3><ul><li><strong>Model-free learning</strong> is a method where the agent just learns from experience rather than creating a model of the outer environment. Without trying to understand the underlying structure of the environment, the agent makes decisions based on the current state of the environment and the anticipated benefits.</li><li><strong>Model-based learning</strong> is a method where the agent creates a model of the surrounding area in order to forecast the results of its actions. With the assistance of this model, the agent can plan and decide in accordance with a long-term perspective of the environment, taking into consideration potential future states and rewards.</li></ul><blockquote><strong>Model-free learning</strong> is often<strong> faster</strong> and <strong>simpler</strong>, but may not generalize well to new situations, while <strong>model-based learning</strong> is more <strong>accurate</strong> and <strong>flexible</strong> but may require more computational resources.</blockquote><h3>Widely used models for Reinforcement Learning :</h3><p><strong>1.Markov Decision Process</strong> (MDP) is a mathematical framework used to model decision-making problems in situations where outcomes are uncertain and may depend on previous decisions. It is defined by state (S), transition function (P), reward (R), and a discount rate (D).</p><blockquote>If D is 0, the agent cares only about the next reward. If D is 1, the agent cares about all future rewards.</blockquote><p>The overall reward accumulated from a given time T is given by a simple mathematical equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/463/1*sxuez2JIzisJk-UjZdrlSQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/941/1*RiOrso5_C92r6kQxEvD3GA.png" /></figure><p>One could imagine an MDP as the interaction between an agent and its surroundings. An environment responds to an agent’s specific set of activities by rewarding it and changing its state. Only the prior state and action have any bearing on the subsequent state and reward.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p><strong>2. Q-learning</strong> is a model-free, off-policy reinforcement learning that will find the best course of action, given the current state of the agent. Depending on where the agent is in the environment, it will decide the next action to be taken.</p><p>The model’s goal is to determine the optimum course of action given the situation as it is. In order to accomplish this, it might devise its own set of rules or might deviate from the prescribed course of action. This indicates that there is no real need for a policy, which is why it is referred to as off-policy.</p><p>It’s given by the equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/0*Y3n3jPXTUS3LQM_K.png" /></figure><blockquote>Basically in <strong><em>Q-Learning</em></strong>, our we create a table with actions and states, called <strong><em>Q-Table</em></strong>. It helps the agent to consider the best action at that instant of time.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*ghEmSHTWFwgAmaeq.png" /></figure><p>After applying the first equation mentioned above, we will receive the maximum reward from the environment and we use the below equation to update the <strong>Q-Table</strong> :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*Hj2Ga9vGvY3WQCx8.png" /></figure><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p>You now have a better knowledge of what Reinforcement Learning includes. Consolidate your understanding by creating a complete RL solution to a challenge.</p><h4><strong>Resources :</strong></h4><ol><li><a href="https://github.com/sudharsan13296/Hands-On-Reinforcement-Learning-With-Python">https://github.com/sudharsan13296/Hands-On-Reinforcement-Learning-With-Python</a> — <strong>Master reinforcement and deep reinforcement learning using OpenAI Gym and TensorFlow.</strong></li><li><a href="https://github.com/aikorea/awesome-rl">https://github.com/aikorea/awesome-rl</a> —<strong> A complete Guide !</strong></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4355103b344" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-community-srm/from-alphago-to-self-driving-cars-understanding-the-basics-of-reinforcement-learning-4355103b344">From AlphaGo to Self-Driving Cars: Understanding the Basics of Reinforcement Learning</a> was originally published in <a href="https://medium.com/data-science-community-srm">DataX Journal</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Handwritten Digit Recognition using Multi Level Perceptron.]]></title>
            <link>https://medium.com/data-science-community-srm/handwritten-digit-recognition-using-multi-level-perceptron-35cf34be3c21?source=rss-73f8e3acfaa4------2</link>
            <guid isPermaLink="false">https://medium.com/p/35cf34be3c21</guid>
            <category><![CDATA[basics-machine-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Kruthi M]]></dc:creator>
            <pubDate>Sun, 14 Nov 2021 08:09:06 GMT</pubDate>
            <atom:updated>2021-11-14T08:09:06.646Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/530/0*w90sxKBxqHGWJ8Ki" /></figure><p>Handwritten digit recognition is the ability of a computer to receive handwritten input from sources such as paper documents, photographs, touch-screens and interpret the correct output with the maximum accuracy. In this project we would be using python and machine learning models to recognize the handwritten digits.</p><h4>First Model</h4><p>In order to build the model we first used the concept of a Convolutional Neural Network which is the best for classification and performing computer vision tasks.</p><p>Convolutional Neural Network is a type of deep neural network which uses a technique known as convolutions. ConvNets process images in time efficient manner without losing any important features that are needed for getting a good prediction. So to start with we need to detect 10 classes of objects that are the digits from 0 to 9 in order to classify the answers. Each image in the dataset has the same size of 28 pixels width,28 pixels height along with the pixel values between 0 and 255.First we added a convolutional layer to our model with 64 filters and a receptive field 3 pixels high and 3 pixels wide. Followed by a pooling layer with a receptive field with a width of 2 inputs and a height of 2 inputs that helps us to generalize feature representations. A MaxPooling function is used which calculates the maximum value in each patch of each feature map. The second convolutional layer is added with 128 filters of the same size as in the 1st layer followed by a pooling layer. In the convolutional layer 1 and layer 2 an activation function “Relu” is used which makes the model easier to train and helps us achieve a better performance. Before adding the fully connected layer the feature maps are flattened using the Flatten function.</p><p>A fully connected layer is defined with 128 hidden neurons and an activation of the “SoftMax” function in the output layer is used for predicting a multinomial probability distribution. The model is compiled using the adam optimizer and the categorical cross-entropy loss function is used which is suitable for multi-class classification.</p><p>We monitored the classification using accuracy metric since we have the same number of examples in each of the 10 classes. The final trained model resulted in a weighted accuracy around 99.71% on the dataset with 70,000 images.</p><p>A rotation augmentation randomly rotates the image clockwise by a given number of degrees from 0 to 360. The rotation will likely rotate pixels out of the image frame and leave areas of the frame with no pixel data that must be filled in. While playing around with our web-app, we noticed that if the digit is tilted at an angle, it tends to be detected incorrectly. We imagined that the end user would face a similar problem to a greater extent than us. We saw a simple solution for this — rotational augmentation.</p><p>We added 65000 new data points to overcome our issue. However, even a minimal rotation created confusion as “0” was being detected as “11”, “2” as “7” and so on. We experimented for a long time before we realized that this fix is costlier than the bug. Hence we decided to drop data augmentation as a whole.</p><h4>Second Model</h4><p>Since we were not able to deploy the CNN model we built a Multi Level Perceptron model or in short an MLP model. An MLP is a feed-forward layered network of artificial neurons. It consists of one input layer, one output layer and a number of hidden or intermediate layers. The output from every neuron in a layer of the MLP is connected to all inputs of each neuron in the immediate next layer of the same. The number of neurons in other layers and the number of layers in the MLP are all determined by a trial and error method at the time of its training.</p><p>The objective is to create a neural network for identifying numbers based on handwritten digits. To train and validate a classifier network, there must be a sufficiently large dataset of handwritten digits. We have used “The Modified National Institute of Standards and Technology dataset, or in short MNIST dataset for our model.</p><p><em>Before we understand the model let’s look at our dataset.</em></p><p>The MNIST dataset has about 70,000 images of handwritten digits ranging from 0 to 9.It has a training set of 60,000 images and 10,000 test images. Each of the images are sized at 28 x 28 pixels in grayscale. We have used the MNIST data from an API that downloads and extracts the images and labels.</p><h4>Tensorboard setup</h4><p>We setup a tensorboard for our model on colab using:</p><blockquote>%load_ext tensorboard</blockquote><blockquote>%tensorboard — logdir=’name of the file’</blockquote><p>We then created a directory for our tensorboard logs.</p><blockquote><em># Folder for Tensorboard</em></blockquote><blockquote>folder_name = f’<strong>{</strong>model_name<strong>}</strong> at <strong>{</strong>strftime(“%H:%M”)<strong>}</strong>’</blockquote><blockquote>directory = os.path.join(LOGGING_PATH, folder_name)</blockquote><blockquote><strong>try</strong>:</blockquote><blockquote>os.makedirs(directory)</blockquote><blockquote><strong>except</strong> <strong>OSError</strong> <strong>as</strong> exception:</blockquote><blockquote>print(exception.strerror)</blockquote><blockquote><strong>else</strong>:</blockquote><blockquote>print(‘Successfully created directories!’)</blockquote><h4>Setting up our Neural network Architecture</h4><p>We have added two hidden layers for this model. The first hidden layer consists of 512 neurons and the second hidden layer consists of 64 neurons. We have used the Dense layer which is a fully connected layer and the most common type of layer used on multi-layer perceptron models. As dense layers require weights and biases and they need to be initialized with a random normal distribution with zero mean and small variance we have used tf.random.truncated_normal.</p><p>The model then multiplied the input for each layer with its respective weights and added the bias term. After weights and biases, we have added an activation function which in the case for hidden layers is ReLu function and SoftMax for the final output layer to get class probability score.</p><p>We defined an optimizer and learning rate for our network to optimize weights and biases on our given loss function. We used the Adam optimizer to achieve this.</p><blockquote>nr_epochs = 50</blockquote><blockquote>learning_rate = 0.001</blockquote><blockquote>n_hidden1 = 512</blockquote><blockquote>n_hidden2 = 64</blockquote><blockquote><strong>def</strong> setup_layer(input, weight_dim, bias_dim, name):</blockquote><blockquote><strong>with</strong> tf.name_scope(name):</blockquote><blockquote>initial_w = tf.random.truncated_normal(shape=weight_dim, stddev=0.1, seed=42)</blockquote><blockquote>w = tf.Variable(initial_value=initial_w, name=’W’)</blockquote><blockquote>initial_b = tf.constant(value=0.0, shape=bias_dim)</blockquote><blockquote>b = tf.Variable(initial_value=initial_b, name=’B’)</blockquote><blockquote>layer_in = tf.matmul(input, w) + b</blockquote><blockquote><strong>if</strong> name==’out’:</blockquote><blockquote>layer_out = tf.nn.softmax(layer_in)</blockquote><blockquote><strong>else</strong>:</blockquote><blockquote>layer_out = tf.nn.relu(layer_in)</blockquote><blockquote>tf.compat.v1.summary.histogram(‘weights’, w)</blockquote><blockquote>tf.compat.v1.summary.histogram(‘biases’, b)</blockquote><blockquote><strong>return</strong> layer_out</blockquote><blockquote>layer_1 = setup_layer(X, weight_dim=[TOTAL_INPUTS, n_hidden1],</blockquote><blockquote>bias_dim=[n_hidden1], name=’layer_1&#39;)</blockquote><blockquote>layer_2 = setup_layer(layer_1, weight_dim=[n_hidden1, n_hidden2],</blockquote><blockquote>bias_dim=[n_hidden2], name=’layer_2&#39;)</blockquote><blockquote>output = setup_layer(layer_2, weight_dim=[n_hidden2, NR_CLASSES],</blockquote><blockquote>bias_dim=[NR_CLASSES], name=’out’)</blockquote><blockquote>model_name = f’<strong>{</strong>n_hidden1<strong>}</strong>-<strong>{</strong>n_hidden2<strong>}</strong> LR<strong>{</strong>learning_rate<strong>}</strong> E<strong>{</strong>nr_epochs<strong>}</strong>’</blockquote><p>In TensorFlow, we define a placeholder for our input variables and output variables and any variables we want to keep track of. We have named each layer using the name_scope function. This was then displayed in our TensorFlow graph. This is the Tensorflow graph of our model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/0*ySsYysnWk8XOg0CA" /></figure><p>We then Saved the model and converted the h5 file to tensorflow.js file. On converting the model we received two files, model.json and group1-shard1of1.bin.</p><blockquote>tf.saved_model.load(sess,[tag_constants.SERVING],export_dir=”name of the file”)</blockquote><blockquote>!pip install tensorflowjs!tensorflowjs_converter — input_format keras ‘/content/model.h5’ ‘/content/model’</blockquote><h4>Website</h4><h4>FRONTEND:</h4><p>Since the project was created for primary school students, we tried to make the website more interactive and fun to work with. Our website has a space theme where an astronaut who is lost in space tries to connect to Houston and asks the user to answer a few questions to help him return home. While the users answer the questions, the astronaut also gives out interesting facts about the space to engage the students.</p><p>The most important component of our frontend was the canvas. This is where the users can write the answers and our model takes the input and predicts the number with maximum accuracy. We have incorporated touch and mouse events in the canvas to make it mobile responsive.</p><h4>BACKEND:</h4><p>To help our model predict the image we have a few steps to follow.</p><ul><li>Load the image</li><li>Convert to B and W</li><li>Calculate Bounding rectangle</li><li>Crop the image</li></ul><p>After the user writes on the canvas we wanted to be able to load the image using OpenCV. We have done this using the Imread method which reads an Image from html canvas.</p><p>We then converted the image color to grayscale using CVT color. We have used thresholding to convert all the pixels above a certain threshold to a different color. This helped us give more defined edges.</p><p>We calculated the bounding rectangle and cropped the image for our model.</p><p>Now since the input images in our model are supposed to be 28 x 28 pixels large we have resized the images. We then have created tensors and made predictions on the new data provided by the user.</p><p>To avoid bleeding our memory every time someone uses the site we used the dispose function to clean up the memory and dispose of the tensors created.</p><h4>Conclusion:</h4><p>You can check out our website here: <a href="https://data-science-community-srm.github.io/Hand-Written-Digit-Classification-Recognition/">https://data-science-community-srm.github.io/Hand-Written-Digit-Classification-Recognition/</a></p><p>In conclusion, a Multilayer Perceptron (MLP) Neural Network was implemented to address the handwritten digit recognition problem. The proposed neural network was trained and tested on a dataset obtained from MNIST. The system performance was observed by varying the number of hidden units and the number of iterations. The final trained model resulted in around 97.67% test accuracy and 99.26% train accuracy on the dataset with 70,000 images.</p><p>Furthermore, in the future we will work on improving the model’s accuracy by using Convolutional Neural Networks (CNN) and fine tune it to get the best outcome possible. We also aim to work on the front end approach of this website by making it even more interactive and responsive.</p><p>Our team has had great coordination and everyone contributed to their parts on time. We started off by deciding the topic which we all agreed on and delegated the workload accordingly. It was our first time working on a Deep learning model. We had our hurdles and problems on the way, but we passed them with flying colors. We had a great time and learnt many new things throughout this process.</p><p>Overall it was a magnificent experience. The teamwork was great and so was the outcome.</p><h4>References:</h4><ol><li><a href="https://machinelearningmastery.com/"><strong>https://machinelearningmastery.com/</strong></a></li><li><a href="https://towardsdatascience.com/"><strong>https://towardsdatascience.com/</strong></a></li><li><a href="https://www.tensorflow.org/js/guide"><strong>https://www.tensorflow.org/js/guide</strong></a></li><li><a href="https://bit.ly/3AVcNtG"><strong>https://bit.ly/3AVcNtG</strong></a></li><li><a href="https://opencv.org/"><strong>https://opencv.org/</strong></a></li><li><a href="https://github.com/tensorflow/tfjs-converter"><strong>https://github.com/tensorflow/tfjs-converter</strong></a></li><li><a href="https://www.tensorflow.org/js/guide/tensors_operations"><strong>https://www.tensorflow.org/js/guide/tensors_operations</strong></a></li></ol><h4>Contributors :</h4><p><a href="https://medium.com/u/73f8e3acfaa4">Kruthi Muralidhara Jety</a></p><p><a href="https://medium.com/u/c4bc137aa0de">Abhaylal</a></p><p><a href="https://medium.com/u/f49b3cff61a4">Jahnavi Darbhamulla</a></p><p><a href="https://medium.com/u/530673b69568">Aryan Karoliwal</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35cf34be3c21" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-community-srm/handwritten-digit-recognition-using-multi-level-perceptron-35cf34be3c21">Handwritten Digit Recognition using Multi Level Perceptron.</a> was originally published in <a href="https://medium.com/data-science-community-srm">DataX Journal</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>