<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Atiar Rahman on Medium]]></title>
        <description><![CDATA[Stories by Atiar Rahman on Medium]]></description>
        <link>https://medium.com/@ar3441?source=rss-fc2d53367165------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*guICSa6xOFyVtTAhdC9gpQ.jpeg</url>
            <title>Stories by Atiar Rahman on Medium</title>
            <link>https://medium.com/@ar3441?source=rss-fc2d53367165------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 13:30:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ar3441/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Central Limit Theorem]]></title>
            <link>https://medium.com/@ar3441/the-central-limit-theorem-9ede4ebfafa5?source=rss-fc2d53367165------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ede4ebfafa5</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Atiar Rahman]]></dc:creator>
            <pubDate>Wed, 13 Nov 2019 03:25:50 GMT</pubDate>
            <atom:updated>2019-11-13T03:25:50.255Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7SEcB3RDck97PJaV.png" /></figure><p>The central limit theorem is one of the most important concepts in all of statistics. Have you ever wondered why we can get information on a population without surveying the entire population? Let’s consider an example. Imagine there are 1,000,000 third graders in a city and we need to estimate the average score on a statewide exam. It would take a long time to count a million test scores, so how do we get an estimation that is as close as possible to the actual population mean.</p><p>So you start thinking and the first thing you do is get a sample of 50 students and get an average of those test scores. Maybe this will give you some idea of what the mean will be. But 50 students isn’t even close to a million. So what do you do? Do you just get as large of a sample as possible and hope that it’s an accurate representation of the population? Clearly this is a tricky problem. But luckily we have the central limit theorem.</p><p>The central limit theorem states:</p><blockquote>If repeated random samples of size n are taken from a population with a mean and standard deviation, the sampling distribution of sample means will have a mean equal to the population mean and a stand error equal to the standard deviation divided by the square root of n.</blockquote><p>Now what does this mean? What is a sampling distribution of sample means and what is the standard error? Let’s go back to the example above. Imagine instead of taking one sample, you took an n amount of samples. Now every time, you took a sample you found the average mean of that sample. You then use these means to create a distribution. This distribution of sample means will in fact form a normal curve and the mean of this curve is equal to the mean of the population! The standard deviation of this distribution of samples is called the standard error, and it is equal to what is mentioned above.</p><h4>More Specifics</h4><p>So looking at this formula, there are some obvious questions one would have? How many samples need to be taken and how large is each sample? You generally want as many samples as possible. Generally, having sample sizes greater than or equal to 30 will suffice.</p><h4>Conclusion</h4><p>The Central Limit Theorem is essential to understanding inferential statistics. It is how statisticians can use samples to understand a population. Some important things to note is that the samples must be independent and random.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/287/0*YMx9gxbBOBWAhapg" /></figure><h4>Resources</h4><p>Here are some resources that might be helpful to get an even better understanding of the central limit theorem.</p><p><a href="https://www.analyticsvidhya.com/blog/2019/05/statistics-101-introduction-central-limit-theorem/">Learn the Central Limit Theorem in R: A Step by Step Guide (Updated 2023)</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ede4ebfafa5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using a CNN to Classify Age Groups]]></title>
            <link>https://pub.towardsai.net/using-a-cnn-to-classify-age-groups-3f2a3c4f4583?source=rss-fc2d53367165------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f2a3c4f4583</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[convolutional-neural-net]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Atiar Rahman]]></dc:creator>
            <pubDate>Tue, 01 Oct 2019 15:34:45 GMT</pubDate>
            <atom:updated>2019-10-03T15:33:02.053Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9udpetF_FdtIE6XP_65VLA.jpeg" /></figure><h4>Diving Into my First Convolutional Neural Network | <a href="https://towardsai.net/">Towards AI</a></h4><p>Having recently learned about convolutional neural networks, I wanted to build my own. Initially, I wanted to be able to get a person’s age just from a picture of their face. However, this comes with its own challenges such as class imbalance and the amount of training time that would be necessary felt daunting. So I decided to make the problem easier by classifying people into age groups instead.</p><p>The biggest challenge with any machine learning project is getting labeled data. Luckily enough a similar project was done recently by the Computer Vision Lab at ETH Zurich. They created a data set by scraping images from IMDB and Wikipedia. The images were also cropped to mostly show faces only! The data set includes photos of actors and when the picture was taken allowing one to deduce their age at the time of the photo. Altogether, the original data set contains 524,230 images. After some exploratory data analysis and cleaning, I settled on using 85,565 images to reduce the computational load of the project and to match my needs in terms of class balance. My goal was to classify images into thirteen possible age groups that range from age 10 to age 80 in increments of five.</p><h4>Building The Network | Transfer Learning</h4><p>For the CNN, transfer learning was used by using VGG-16’s convolutional base. Transfer learning is when you utilize the weights of a previously trained network in a new network. VGG-16 is a famous CNN that was trained on ImageNet which is a data set of over 14 million images belonging to 1000 classes. The model itself achieved 92.5% accuracy on these images. It originally classified images into 1000 different classes, but in this case, the top layers were changed so that it can classify face images into the 13 classes or age groups. This is why transfer learning is so beneficial because you don’t have to retrain your network which can be time-consuming and difficult depending on the resources you have available. Instead of retraining, you can build your network by training on top of a robust network. The full VGG-16 architecture can be seen in the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*1W-8Ki2jBVcqk78k.png" /><figcaption>VGG-16 Network Architecture</figcaption></figure><p>My model uses the same convolutional base as VGG-16 but has different fully connected layers and softmax layers that will classify images into 13 age groups. Only these layers were trained while the previous layers where left untrained. Instead, these layers are using the weights from when VGG-16 was trained on Image-Net.</p><h4>Results</h4><p>The model was trained on 59,110 images with the remaining images that were used for testing and validation. Due to time constraints, the model was trained up to 64% training accuracy. However, the testing and validation accuracy never went beyond 17%. I also created a function to return the three most probable age groups that a person belongs to. This function, the CNN, data cleaning, and exploratory data analysis can be seen on my GitHub.</p><p><a href="https://github.com/AR3441/AgeClassificationFromFace">AR3441/AgeClassificationFromFace</a></p><h4>Resources</h4><p>For those interested in seeing a CNN that classifies faces into ages and not just age groups, I suggest the following <a href="https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/">paper</a> by the Computer Vision Lab at ETH Zurich.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f2a3c4f4583" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/using-a-cnn-to-classify-age-groups-3f2a3c4f4583">Using a CNN to Classify Age Groups</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Optical Character Recognition(OCR)?]]></title>
            <link>https://medium.com/analytics-vidhya/what-is-ocr-f67c9ab218bf?source=rss-fc2d53367165------2</link>
            <guid isPermaLink="false">https://medium.com/p/f67c9ab218bf</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[lstm]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[recurrent-neural-network]]></category>
            <category><![CDATA[ocr]]></category>
            <dc:creator><![CDATA[Atiar Rahman]]></dc:creator>
            <pubDate>Tue, 24 Sep 2019 16:17:39 GMT</pubDate>
            <atom:updated>2019-09-26T05:12:32.219Z</atom:updated>
            <content:encoded><![CDATA[<p>Have you ever tried to deposit a check electronically through your bank’s app? When doing this, you are usually directed to hover your phone over the front and back of your check until your phone decides the optimal angle to take a picture. Using these pictures, the bank recognizes the amount of money you are depositing, what account the money is coming from, who it’s coming from and on top of all this you’ve saved yourself time by not having to go to the bank. But what kind of sorcery is this?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/598/1*ZmhXuAbko_fkROhp6LWnWw.jpeg" /></figure><p>This is done through OCR or Optical Character Recognition. OCR is what allows computers to recognize writing within images and extract that writing. Depositing checks electronically is just one of it’s many uses. Some other uses include checking yourself in at hotels, utility meter readings, and text editors.</p><p>But how exactly does OCR work? There are various OCR models, some more advanced than others, but in principle each character is being compared to a database of characters. The character in the database that most resembles the character being read is selected. The more advanced OCR models will not only look at characters as a whole, but the specific attributes such as curvature allowing them to distinguish characters that may look similar like a “1”(number one) or “l”(lower-case L). At the end, the model may use a spell checker or a dictionary to see if the words that are returned makes sense.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*qOsDRivEvjE0gzfMe-mmBA.png" /></figure><p>Now, let’s get into some of the nuts and bolts of this process. The first thing an OCR model does with an image is pre-processing. This can be a lot of different things depending on the model but essentially changes are made to the image to make it as easy to read as possible. Some examples of pre- processing are rotating the text, straightening the text, removing any background images, making the background white as possible and making the text dark if it isn’t dark already.</p><p>After doing this, the OCR begins the processing of recognizing each character and there are different approaches to doing this. One approach is pattern recognition. This works by first recognizing lines of black pixels with rows of white pixels in between. Individual characters are then looked at the same way by looking at the black pixels that are in between columns of white pixels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KZyABtFgM-2kZ81YOLvPBw.png" /></figure><p>Each character is converted into a binary matrix where the white pixels are 0s and the black pixels are ones. Using the distance formula, the distance between the center of the matrix and the furthest pixel is measured. This is used as a radius to create a circle and divides the circle into different subsections which are each compared to a database of characters to get the character that matches the best.</p><h4>Python Libraries</h4><p>Google has their own open source OCR engine called Tesseract that has the ability to recognize more than 100 languages. It can also be trained to recognize other languages. This engine can be used in Python by using the library python-tesseract which is a wrapper for Google’s OCR engine. More information on this library can be found here:</p><p><a href="https://pypi.org/project/pytesseract/">pytesseract</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*YBdnYbPHxz7UIF6adue_iw.png" /></figure><h4>OCR and Deep Learning</h4><p>The methods above were the traditional ways to perform OCR. However, with the progress of computer science and machine learning, OCR can now be done with deep learning. OCR models can now be trained to recognize characters. However, even with deep learning there are challenges because of the variety of characters caused by different fonts, languages, and of course poor handwriting. There are different deep learning approaches to handle this including EAST and CRNN. These methods can be implemented with tools like Tensorflow and Keras. This picture is a high-level example of OCR being done with deep learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/447/1*dW_1E_edZd62dYzbPBptIg.png" /><figcaption>LSTM is an RNN(recurrent neural network) standing for Long Short-Term Memory</figcaption></figure><p>In the example above LSTM is used which stands for Long Short-Term Memory. LSTM is an artificial recurrent neural network that has feedback connections and can process not only single data points, but sequences of data(speech and video). The following charts show an overview of how it may be used in an OCR model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5Vf4ATFt9-IUEv216X39xQ.png" /><figcaption>OCR Deep Learning Overview</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zLJDPcUHWs-chyZxJtCiIQ.png" /><figcaption>Each bar is a probability vector, with the most probable character being selected</figcaption></figure><h4>Summary</h4><p>In short, OCR is another very cool application of computer science/machine learning that makes daily living easier. It has been around for quite some time but recent advances in machine learning have made it even more accurate.</p><h4>Resources</h4><p>The following resources were very helpful in the making of this post.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FcAkklvGE5io%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DcAkklvGE5io&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FcAkklvGE5io%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c1286446238cbb8dc7c9a09751aa64b/href">https://medium.com/media/4c1286446238cbb8dc7c9a09751aa64b/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FjO-1rztr4O0%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DjO-1rztr4O0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FjO-1rztr4O0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/71fc8595f9fc0b239c135d1f4363a443/href">https://medium.com/media/71fc8595f9fc0b239c135d1f4363a443/href</a></iframe><p><a href="https://towardsdatascience.com/a-gentle-introduction-to-ocr-ee1469a201aa">A gentle introduction to OCR</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f67c9ab218bf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/what-is-ocr-f67c9ab218bf">What is Optical Character Recognition(OCR)?</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spam Filtering Using Naive Bayes]]></title>
            <link>https://medium.com/@ar3441/spam-filtering-using-naive-bayes-98a341224038?source=rss-fc2d53367165------2</link>
            <guid isPermaLink="false">https://medium.com/p/98a341224038</guid>
            <category><![CDATA[email]]></category>
            <category><![CDATA[bayes-theorem]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Atiar Rahman]]></dc:creator>
            <pubDate>Wed, 15 May 2019 16:19:36 GMT</pubDate>
            <atom:updated>2020-11-16T15:43:22.078Z</atom:updated>
            <content:encoded><![CDATA[<h3>Filtering Spam Using Naive Bayes</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/1*byZ8tSrUasgfCvHU0a315Q.png" /></figure><p>You out of the billions of people on Earth have been called upon by the Prince of Nigeria to help move his fortune out of the country. You can’t deny this request and give him the money he needs in return for a piece of his fortune. Unfortunately, you never get any money from this fictional prince and realize you’ve just been scammed. This type of scam has been fooling people for ages and is surprisingly still going strong. Ridiculously enough, Americans have lost $703,000 to these types of scams in 2018 alone according to a report by ADT Security Services. One way to reduce the number of people falling for these types of scams is spam filtering. But how does your email service actually filter out spam emails?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*mzIPUxnxbzdjlJusLnkNRQ.jpeg" /></figure><p>One way spam emails are sorted is by using a Naive Bayes classifier. The Naive Bayes algorithm relies on Bayes Rule. This algorithm will classify each object by looking at all of it’s features individually. Bayes Rule below shows us how to calculate the posterior probability for just one feature. The posterior probability of the object is calculated for each feature and then these probabilities are multiplied together to get a final probability. This probability is calculated for the other class as well. Which ever has the greater probability that ultimately determines what class the object is in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0IPNc3rz6eIpBSiTKNzvFQ.png" /><figcaption>Bayes Rule</figcaption></figure><p>For our purposes, the object is an email and the features are the unique words in the email. Thus, there is a posterior probability calculated for each unique word in them email. Plugging this into Bayes Rule, our formula will look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*a49sFMTNmacaTERHzDVBAg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/1*y3TaQNz_4wKwKIk1yf7FKA.png" /></figure><p>Now that we understand Naive Bayes, we can create our own spam filter.</p><h3>Creating A Spam Filter Using Python/Scikit-Learn</h3><p>Creating your own spam filter is surprisingly very easy. The first step is to get a data set of emails. This can be found on Kaggle and will need to be read into a pandas dataframe. Your dataframe should look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*pl_XCXE4lQCO8IpvoBpvnw.png" /><figcaption>Sample DataFrame containing emails</figcaption></figure><p>In this case, the ‘text’ column contains the message within each email. The ‘label_num’ column has the outcomes for these emails. For this dataset, a 1 represent an email that is ‘spam’ while a 0 represent an email that is not spam or ‘ham’. Besides pandas, you will also need to import the following scikit-learn libraries:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*Ui8IGOYMAyqx1tHQlP9pRA.png" /></figure><p>Now that you have your dataset ready, it’s time to train your classifier with just a few lines of code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/1*YOwFw7aai51CqtwzpeJrMw.png" /></figure><p>In the code above, the first thing I did was create a train-test-split which isn’t necessary to build your classifier. In the next step I use the CountVectorizer() in order to change each email into a vector counting the number of times each word occurs. Now we’re ready to see the classifier being used in action. The following sample emails were created. One is clearly spam while the other is a normal email.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0kp5LJNbXcQyZRXKAMdifA.png" /></figure><p>In order to use your classifier, you must vectorize the example emails. Finally, you can classify the emails. For our examples above, we got the following results with the first email classified as ‘spam’ and the second as ‘not spam’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*PdPj1MR0cBtm1TejNV5giQ.png" /></figure><p>The classifier was successful. In order to get a better understanding of the performance of the model, the accuracy and F1 score was measured. There were also a total of only 22 false positives and false negatives for a testing set with 1293 emails.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*5fOA2b6AGlb470OMD96jZQ.png" /></figure><h3>Conclusion</h3><p>Naive Bayes is powerful yet simple algorithm that is especially useful when filtering out spam emails. To get an even better understanding of Naive Bayes and Bayes rule, this <a href="https://www.analyticsvidhya.com/blog/2017/09/naive-bayes-explained/">article</a> may be very helpful.</p><h3>Resources</h3><ul><li><a href="https://www.analyticsvidhya.com/blog/2017/09/naive-bayes-explained/">https://www.analyticsvidhya.com/blog/2017/09/naive-bayes-explained/</a></li><li><a href="https://www.cnbc.com/2019/04/18/nigerian-prince-scams-still-rake-in-over-700000-dollars-a-year.html">https://www.cnbc.com/2019/04/18/nigerian-prince-scams-still-rake-in-over-700000-dollars-a-year.html</a></li><li><a href="https://www.youtube.com/watch?v=M59h7CFUwPU">https://www.youtube.com/watch?v=M59h7CFUwPU</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98a341224038" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>