<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Obinnamuonanu on Medium]]></title>
        <description><![CDATA[Stories by Obinnamuonanu on Medium]]></description>
        <link>https://medium.com/@obinnamuonanu?source=rss-d5a59b36b048------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*R_6k6BZ5tvy5rXQv</url>
            <title>Stories by Obinnamuonanu on Medium</title>
            <link>https://medium.com/@obinnamuonanu?source=rss-d5a59b36b048------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 08:48:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@obinnamuonanu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How Machine Learning Is Changing the Way We Discover New Materials]]></title>
            <link>https://medium.com/@obinnamuonanu/how-machine-learning-is-changing-the-way-we-discover-new-materials-450bf4bef89b?source=rss-d5a59b36b048------2</link>
            <guid isPermaLink="false">https://medium.com/p/450bf4bef89b</guid>
            <category><![CDATA[renewable-energy]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[energy-storage]]></category>
            <category><![CDATA[material-science]]></category>
            <dc:creator><![CDATA[Obinnamuonanu]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 13:02:19 GMT</pubDate>
            <atom:updated>2026-11-10T13:02:19.627Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*axzREar3MkHSTYgl" /></figure><p><strong>Why do we need new materials?</strong></p><p>Almost every technology we are counting on for a cleaner future is held back, at some point, by a material or combination of materials. Electric cars need battery electrodes and electrolytes that store more energy, charge faster, hold charge longer, and don’t catch fire. Solar panels need materials that absorb sunlight efficiently and survive decades outdoors. Our phones and computers depend on semiconductors with precisely tuned electronic properties. Also, millions of people rely on adsorbents and membranes to pull salt, heavy metals, and other contaminants (which can be micro- or even nano-contaminants) out of their drinking water. During my undergraduate internship at the Standards Organization of Nigeria (SON), I was responsible for measuring heavy metal concentrations in both edible and non-edible finished products. Imagine how dangerous it would be if these products, including packaged drinking water, were left unchecked.</p><p>In each case, this is why researchers and scientists are always searching for better materials to achieve the desired goal.</p><p>One major problem scientists and researchers face during the material discovery process is the search space. The periodic table contains 118 elements. If you try to mix a number of elements from the periodic table and arrange their atoms in different crystal structures, you&#39;ll notice that the number of possible materials quickly runs into millions. You’ll be surprised that out of these possible combinations, only a tiny fraction has ever been made or tested. Testing them one by one in a laboratory is slow, expensive, and often frustrating. So, with the rise of artificial intelligence technology, researchers are increasingly looking into how machine learning can help not only accelerate discovery but also decide which materials are worth making in the first place.</p><p><strong>How have materials been traditionally discovered in the past?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*s1mJ8Kr3HNWXTSgM" /><figcaption>Image generated with AI</figcaption></figure><p>For most of history, material discovery has followed a loop.</p><ol><li>Start with a question—for example, we need a cathode that holds more lithium or an adsorbent that captures more lead.</li><li>Use chemistry intuition and past literature to pick a few promising compositions.</li><li>Synthesize them in the lab.</li><li>Characterize them—for example, measure their structure, conductivity, strength, or adsorption capacity.</li><li>Adjust the composition or the processing conditions.</li><li>Repeat until something works well enough</li></ol><p>This is experimental trial and error, and it is how scientists, researchers, and engineers have discovered most of the materials we rely on today, and it remains the only way to know for certain how a material behaves in the real world. However, the problem with this is speed. A single synthesis-and-test cycle can take weeks or months, and believe me, many attempts do fail, and a failed experiment often says little about why it failed. Consider this analogy: when you run Python code and it fails, you’ll get feedback after running the code, which helps you understand why the code failed. This isn’t the case for traditional science, and trying to figure out what went wrong is a different ball game. When the list of candidates runs into thousands or millions, even a well-equipped lab can only test a fraction of them. The real bottleneck is choosing which few experiments are worth running.</p><p><strong>Machine learning as a new approach to material discovery</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Lw2RhsFTU3-epbMI" /><figcaption>Image generated with AI</figcaption></figure><p>Machine learning helps to solve this problem. It uses what we already know to estimate quickly how materials we haven’t made yet are likely to behave. The idea here is simple. Show an algorithm thousands of examples where you know both the input (a material’s composition and atomic structure) and the output (one of its properties), and the algorithm learns the patterns that connect the two. Then give it a material it has never seen, and it gives you its best guess of that particular output. These examples (materials) used to train the machine learning model come from large materials databases. The best known database is the <strong>Materials Project, </strong>which is an open database that uses supercomputers to calculate the properties of tens of thousands of known and hypothetical inorganic compounds <a href="https://doi.org/10.1063/1.4812323">[1]</a>. Most of these calculated values come from density functional theory (DFT), which is a quantum-mechanical method that is very accurate but slow. A single DFT calculation can take hours, which is why machine learning (a fast approximation for calculating these values) has become valuable.</p><p>Once a machine learning model is trained, it can estimate properties such as</p><ol><li>Formation energy—which is the amount of energy released or absorbed when a compound is created from its basic starting elements.</li><li>Band gap—which decides whether a material behaves as a metal, semiconductor, or insulator.</li><li>Thermal conductivity, mechanical strength, and other electronic properties.</li></ol><p>With machine learning, a prediction that would take hours with DFT can take only seconds. Researchers can now screen millions of materials, rank them, and only send the most promising candidates for further investigation or analysis. Examples of machine learning models used are classical models like XGBoost and Random Forest, artificial neural networks, and graph neural networks. Graph neural networks (GNNs) are quite fascinating for me because they treat material crystal structure as a network. Atoms are represented as nodes, and edges represent the bonds between the atoms. In 2018, Tian Xie and Jeffrey Grossman at MIT showed that this approach could predict eight different crystal properties with accuracy close to DFT itself <a href="https://arxiv.org/abs/1710.10324">[2]</a>. It turns out that representing materials the way scientists actually picture them turned out to be a powerful idea, and GNNs now sit behind many of the latest discoveries today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XHxLRNoLZA1OXv1C" /><figcaption>Image generated with AI</figcaption></figure><p><strong>How is machine learning actually being used?</strong></p><p>The easiest way to understand how machine learning is applied in material science is to look at real projects. Let us look at three cases, each showing a different way in which machine learning is applied in material science.</p><p><strong>A. Finding better battery materials</strong></p><p>Today’s lithium-ion batteries use a liquid electrolyte, the medium that carries lithium ions between the two electrodes. But the problem is that liquids can leak and catch fire, so researchers look towards solid electrolytes instead. Now, there is another problem: only very few solids let ions move through them quickly while staying chemically stable.</p><p>In 2024, Microsoft and the Pacific Northwest National Laboratory (PNNL) team of researchers, led by Chi Chen, Vijayakumar Murugesan, and Matthias Troyer, went looking for a solution to this problem <a href="https://doi.org/10.1021/jacs.4c03849">[3]</a>. They started with more than 32 million candidate materials, generated by swapping different elements into known crystal structures.</p><p>On a high level the entire workflow worked like a funnel:</p><ol><li>Machine learning models estimated which candidates were likely to be stable, cutting the list to around half a million.</li><li>Further models and physics-based simulations checked for battery-relevant properties, such as how easily ions move and how stable the material stays against the electrodes.</li><li>The team then weighed things like cost and the availability of the elements involved.</li></ol><p>At the end, 18 promising candidates with new compositions remained. Battery experts at PNNL chose one family, NaₓLi₃₋ₓYCl₆, a chloride that replaces part of the lithium with sodium. They synthesized it, measured its conductivity, and built a working prototype battery. Two details matter here. First, the final choice was made by humans, using their knowledge of what could realistically be made in a lab. Second, the authors were careful to say that more testing is needed before anyone knows whether the material works over a long service life. The idea I&#39;m trying to pass across here is that, inasmuch as machine learning helps to accelerate discovery, machine learning didn’t deliver a finished battery. It turned a search of tens of millions into a shortlist of viable potential candidates that could actually work.</p><p><strong>B. Materials for water treatment</strong></p><p>Water treatment is another area where machine learning can also be applied. Here, the work is less about discovering new compounds and more about understanding how existing materials behave under real operating conditions.</p><p>Take biochar, a cheap, carbon-rich adsorbent made by heating agricultural waste. How much heavy metal it captures depends on many things at once: the original biomass, the pyrolysis temperature, the surface chemistry, the pH of the water, the starting concentration of the contaminant, and the amount of adsorbent used. Running batch experiments to cover every combination would take years.</p><p>In 2024, Zeeshan Haider Jaffari, Kyung Hwa Cho, and colleagues at UNIST in South Korea gathered 1,518 data points from published adsorption experiments. They trained several machine learning and deep learning models to predict how much heavy metal different biochars would adsorb <a href="https://doi.org/10.1016/j.jhazmat.2023.132773">[4]</a>. Their best model predicted adsorption capacity very accurately on data it hadn’t seen. The most interesting result came from asking the model <em>what mattered in its predictions</em>. Using an interpretability technique called SHAP, which estimates how much each input contributes to a prediction, they found that the adsorption conditions accounted for about 72% of the model’s decisions. Pyrolysis conditions accounted for about 26%. The biochar’s elemental composition and physical properties contributed less than 2%. The useful and slightly humbling insight here is that for this problem, <em>how</em> you use the material can matter more than small differences in <em>what</em> it is made of. Researchers are now applying similar methods to membranes, using machine learning to predict how membrane structure relates to water flux and salt rejection, and to virtually screen new membrane materials before making them in the lab <a href="https://doi.org/10.1021/acs.nanolett.3c05137">[5]</a>.</p><p><strong>C. Designing materials with specific properties</strong></p><p>Everything we have looked at so far works in one direction: propose a material, then predict its properties. But what if we could go backwards? That is, you start with the property you need and ask the model to suggest materials that might have it. This is the concept of <strong>inverse design.</strong></p><p>This is where generative models come in. They are the same family of AI that produces images from text prompts. In 2025, Claudio Zeni, Tian Xie and colleagues at Microsoft Research published <strong>MatterGen</strong>, a model that generates new crystal structures from prompts <a href="https://www.nature.com/articles/s41586-025-08628-5">[6]</a>. Just like how an image generator starts from random noise and gradually refines it into a picture, MatterGen starts from a random arrangement of atoms and gradually adjusts the elements, positions and lattice until it reaches a plausible, stable crystal. It can also be steered towards a target, such as a particular chemistry, symmetry, or mechanical, electronic or magnetic property.</p><p>As a test (proof of concept), the team asked MatterGen for a material with a bulk modulus (a measure of how hard it is to compress) of 200 GPa. They then synthesized one of its suggestions, TaCr₂O₆, and measured a bulk modulus of 169 GPa, within about 20% of the target.</p><p>The most important lesson from the last few years is that every big machine learning claim in the field has needed experimental scientists to check it, and not all predicted materials fully survived the check.</p><p>Now, in the midst of exploring how machine learning is applied in material science, it is important to note that <strong>a model is only as good as its data</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qrPcJg5ddMi14ZVn" /></figure><p>Machine learning models learn from the data they are given. If a database is full of oxides and short on sulfides, the model will be confident about oxides and unreliable about sulfides. A model can perform brilliantly on materials that resemble its training set and poorly on genuinely new chemistry, which is exactly where discovery happens.</p><p>With that said, let’s look at something interesting about machine learning in material science.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*MM6SgN24C131SlWn.gif" /></figure><p>In 2020, Christopher Bartel, Gerbrand Ceder and colleagues at Berkeley tested several machine learning models that predicted formation energy with impressive accuracy <a href="https://www.nature.com/articles/s41524-020-00362-y">[7]</a>. When they used those same predictions to decide whether compounds were actually <em>stable</em>, the models that relied on composition alone performed poorly. Stability depends on whether a compound has lower energy than all the competing compounds it could break down into. A model can score well on a benchmark and still mislead you about the stability.</p><p>Let’s look at an example to understand the message here. Let’s say compound A has a true formation energy of -2.00 eV/atom, and its closest competitor, a chemically similar compound B made from the same elements, sits at -1.98 eV/atom. A is the stable one (compound with a lower energy), but only by 0.02 eV/atom. DFT might calculate A at -1.95 eV/atom and B at -1.92 eV/atom. Each value is off by 0.05–0.06 eV/atom, which is more than twice the gap between them. But because A and B are chemically similar, DFT’s approximations shift both energies in the same direction by roughly the same amount, so most of the error cancels when you compare them. The calculated gap (from DFT calculations) is -0.03 eV/atom instead of -0.02, and DFT still correctly finds A more stable than B.</p><p>Bartel’s team pointed to this systematic cancellation of errors as the reason DFT predicts stability better than its raw error would suggest. A machine learning model trained only on composition doesn’t get this benefit, because its errors don’t line up in the same way from one compound to the next. Suppose it predicts -1.95 eV/atom for A and -1.97 eV/atom for B. Its errors (0.05 and 0.01 eV/atom) are actually smaller than DFT’s, so on a formation-energy benchmark, this result from the model would look better. But because the errors differ, they don’t cancel. Instead, B now appears 0.02 eV/atom more stable than A. The model gets the stability verdict wrong, even though A is the truly stable compound. This is the trap Bartel’s team identified, good formation-energy accuracy does not guarantee good stability predictions.</p><p>Here’s another case scenario.</p><p>In 2023, Google DeepMind’s GNoME project used graph neural networks to predict about 2.2 million new crystal structures, around 380,000 of them classed as the most stable <a href="https://www.nature.com/articles/s41586-023-06735-9">[8]</a>. But when Anthony Cheetham and Ram Seshadri, two experienced solid-state chemists at UC Santa Barbara, examined a sample of the predictions, they found little evidence of compounds that were at once novel, credible and useful <a href="https://doi.org/10.1021/acs.chemmater.4c00643">[9]</a>.</p><p>A recurring problem was <strong>disorder</strong>. Computer models usually treat a crystal as perfectly ordered, with every atom in a fixed position. Many real materials are partly disordered, with different atoms sharing the same sites at random. A model may count an ordered version as new when chemists already know the disordered one. The model claimed to have discovered a novel material but infact the material already exists.</p><p>The same issue appeared in MatterGen’s showcase result. In 2026, Mikkel Juelsholt pointed out that the compound synthesized from MatterGen’s prediction has the same structure as a tantalum chromium oxide first reported decades earlier <a href="https://doi.org/10.1039/d6mh00268d">[10]</a>.</p><p>However, none of this makes machine learning less useful. It just clearly defines its role in the discovery process. The most productive approach combines three tools: machine learning to search widely and cheaply, physics-based simulations such as DFT to check the viable candidates more rigorously, and experiments to establish what is true.</p><p><strong>Challenges facing ML-driven materials discovery</strong></p><p>Machine learning in materials science is still young, and its biggest obstacles are less about algorithms than about data.</p><ol><li><strong>Not enough high-quality data.</strong> Large databases exist for computed properties of inorganic crystals, but far less exists for polymers, membranes, composites or anything measured under real operating conditions.</li><li><strong>Inconsistent experimental data.</strong> Multiple papers may measure the same property using different methods, conditions or units. Failed experiments are rarely published at all, even though they are exactly what a model needs to learn the boundaries.</li><li><strong>Generalization.</strong> Models are most reliable close to what they have seen and least reliable in unexplored chemistry, which is where new discoveries are most likely to be.</li><li><strong>Interpretability.</strong> A prediction is far more useful when researchers understand <em>why</em> the model made it. Tools like SHAP help, but many deep learning models are black-box models and they remain hard to interpret.</li><li><strong>The synthesis gap.</strong> We can now predict materials far faster than we can make them.</li></ol><p><strong>What will the near-future of machine learning in material science look like?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0I9p355xc2wao2aC" /><figcaption>Image generated with AI</figcaption></figure><p><strong>Self-driving laboratories.</strong></p><p>In the near future, we should be able to test materials at the same rate it’s been predicted by the model. This addresses the synthesis gap i mentioned earlier. Nathan Szymanski, Gerbrand Ceder and colleagues at Lawrence Berkeley National Laboratory introduced the A-Lab in 2023, a robotic platform that plans solid-state syntheses, runs them, analyzes the results with machine learning and decides what to try next <a href="https://doi.org/10.1038/s41586-023-06734-w">[11]</a>. Over 17 days of continuous operation, it reported making 41 of 58 target compounds. However, the A-Lab is also a lesson in caution. A group of solid-state chemists led by Leslie Schoop and Robert Palgrave re-examined the results and argued that about two-thirds of the claimed new materials were likely known, compositionally disordered versions of the predicted compounds <a href="https://doi.org/10.1103/PRXEnergy.3.011002">[11]</a>. Despite the automation being impressive, the lesson here is that the step of confirming what had actually been made still needed expert human analysis. The likely future is labs where robots handle the repetitive work and people keep a close eye on the interpretation.</p><p><strong>Generative AI for materials.</strong></p><p>Models like MatterGen are moving the field from screening lists of candidates to proposing them. Their usefulness will depend on teaching them chemistry they currently miss, such as disorder and synthesizability.</p><p><strong>Machine learning plus DFT.</strong></p><p>Rather than replacing quantum-mechanical calculations, machine learning increasingly works as a fast stand-in for them. Machine learning handles the millions of candidates while DFT checks the thousands that are potentially viable. Each verified calculation then becomes new training data.</p><p><strong>Conclusion</strong></p><p>Instead of testing a few dozen candidates chosen by intuition, scientists can now scan millions, rank them, and spend their limited time and lab resources on the handful most likely to succeed. The critiques of GNoME, the A-Lab and MatterGen show why expert judgment and careful experiments remain the final say.</p><p>ML-driven materials discovery is the future and i am really excited to see what the future holds for this field.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=450bf4bef89b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Predictive Model for Graduate School Admissions]]></title>
            <link>https://medium.com/@obinnamuonanu/building-a-predictive-model-for-graduate-school-admissions-8409b254edbf?source=rss-d5a59b36b048------2</link>
            <guid isPermaLink="false">https://medium.com/p/8409b254edbf</guid>
            <dc:creator><![CDATA[Obinnamuonanu]]></dc:creator>
            <pubDate>Wed, 30 Apr 2025 13:34:36 GMT</pubDate>
            <atom:updated>2025-04-30T13:41:20.543Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>“The Road to graduate school according to <strong>DATA”</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3keW1eqoK7b-bPV1" /><figcaption>Photo by <a href="https://unsplash.com/@lsimpson06?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Logan Simpson</a> on <a href="https://unsplash.com/photos/woman-in-white-long-sleeve-shirt-and-black-skirt-walking-on-sidewalk-during-daytime-6KWjKRVwlZE?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Unsplash</a></figcaption></figure><p>Let’s start with a background story</p><p>Simon was a smart lad in his department, had very good grades. Many knew that his heart was set on graduate school after his undergraduate degree. His colleagues were happy for him because he has worked so hard to get to this point. Yet, his grad school applications were rejected in the schools he applied for. On the other hand, Gabriel, a classmate of his, had decent grades too but not as good as Simon’s landed offers from two great universities. It didn’t make any sense. Was it just luck, or was there a pattern to these decisions?</p><p>Graduate admissions can often feel like a black box to applicants. Decisions are influenced by multiple academic and personal factors, yet the criteria for acceptance can seem unclear. With thousands of students applying each year, a data-driven approach to predicting admission outcomes can help applicants better understand the selection process. Simon reached out to Gabriel to understand the secrets behind his success. Gabriel told him that he used <strong>GradScope</strong> to assess how good his academic application package was and to ensure he met the minimum threshold attained by applicants who were successful.</p><h3><strong>What is GradScope?</strong></h3><p><strong>GradScope</strong> is a predictive model that could estimate an applicant’s chances of getting into grad school based on academic features like CGPA, test scores, research experience etc. In this article, I’ll walk you through how <strong>GradScope</strong> was built, the features it considered, and the suprising insights it revealed along the way.</p><p>Let’s get into it….</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*8atpT2KMVcHO2eva" /></figure><h3>📊 <strong>Data Collection/Source</strong></h3><p>I recently participated in a hackathon organised by Hackade which was hosted on Zindi. Hackade founded and run by Victor and Eniola is an initiative dedicated to enhancing data science skills within Nigeria’s ecosystem through competitive programming. The dataset used for this project was provided by Hackade and you can get access to it on 👉<a href="https://zindi.africa/competitions/from-scores-to-seats-the-grad-school-ml-challenge/data">Zindi</a>.</p><p>The dataset included the following features:</p><blockquote>GRE Score</blockquote><blockquote>TOEFL Score</blockquote><blockquote>University Rating</blockquote><blockquote>SOP — Statement of Purpose (rated on the scale of 1–5)</blockquote><blockquote>LOR — Letter of Recommendation (rated on the scale of 1–5)</blockquote><blockquote>CGPA — 10-point scale</blockquote><blockquote>Research — Research Experience</blockquote><blockquote>Admitted — Admission status of the applicant</blockquote><blockquote>Age</blockquote><blockquote>Location</blockquote><blockquote>First Generation</blockquote><h4>🔎<strong>Exploratory Data Analysis</strong></h4><p>The following steps were taken to explore the data and derive insights</p><ul><li><strong>Load the datasets for training and testing</strong></li></ul><pre>data1 = pd.read_csv(r&quot;C:\Users\HP\Desktop\Zindi ML projects\Grad school ML Challenge\train.csv&quot;)<br>data2 = pd.read_csv(r&quot;C:\Users\HP\Desktop\Zindi ML projects\Grad school ML Challenge\test.csv&quot;)<br>sub = pd.read_csv(r&quot;C:\Users\HP\Desktop\Zindi ML projects\Grad school ML Challenge\SampleSubmission.csv&quot;)<br><br>display(&quot;Grad_school train data&quot;, data1.head(), &quot;Grad_school test_data&quot;, data2.head(), &quot;sample_sub&quot;, sub.head())</pre><p>The image below shows how the data used for training looked like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DM8uw9k562yEeoR5DnErLw.png" /><figcaption>Training Data</figcaption></figure><ul><li><strong>Check the descriptive statistics of the data.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*s8yTamer1NaT1RBlQF0IJw.png" /><figcaption>Descriptive Stats of Training Data</figcaption></figure><p>The descriptive statistics tell us that most of the numerical features follow a normal distribution, which means that the values are symmetrically spread out around the average, forming a bell shaped curve when you plot them. Digging deeper revealed some issues with the data.</p><p><strong>SOP and LOR scores</strong> : <strong>SOP</strong> is like a personal essay where a student explains why they want to join the program and their goals while <strong>LOR</strong> is a reference letter from a teacher or mentor vouching for the applicant’s skills and potential. These values should be between 1 (weak) and 5 (excellent) but from the decsriptive stats of <strong>SOP</strong> and <strong>LOR</strong>, you can see the presence of -0.51 and -0.64 as the minimum values and 7.07 and 7.22 as the maximum values respectively.</p><p><strong>GRE and TOEFL scores</strong>: These are standardized exams. The <strong>GRE score</strong> has a maximum score of 340 and <strong>TOEFL</strong> is out of 120. Yet, the data had scores like 341 and 121 respectively which aren’t even valid.</p><p><strong>CGPA</strong>: For this data, the <strong>CGPA</strong> is on a 10-point scale, but you can see that some students had a <strong>CGPA</strong> of more than 10 which doesn’t make sense.</p><ul><li><strong>Check for missing values</strong></li></ul><p>I checked for missing values in the data using the code snippet below:</p><pre>data1.isnull().sum()</pre><p>The result showed that <strong>GRE Score, TOEFL Score, SOP, CGPA </strong>had 360 missing values each.</p><ul><li><strong>Check the distribution of target “Admitted”</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*DegWB9YnD7vmhQF3p3wYCQ.png" /><figcaption>Target distribution</figcaption></figure><p>The pie plot above clearly shows that two-thirds of applicants in this data ends up getting offers from schools. This also indicates that the proportion of admitted and non-admitted applicants is not the same, which means the target admittedis imbalanced.</p><ul><li><strong>Check the distribution of categorical and numerical features</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C3eRT0jQouMvZH-jxBmqdg.png" /><figcaption>Numerical Features distribution</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*T7JzgMGf-0TLXG28FM0ppw.png" /><figcaption>Research experience distribution</figcaption></figure><p>The distribution plots above shows that over 300 applicants had <strong>GRE scores</strong> between 315 and 325, while slightly over 350 applicants scored 110 on their <strong>TOEFL</strong> test. Over 1000 applicants favour mid-tier universities rating between 2–3. This tells us that most applicants prefer to apply to mid-tier university than top-ranked or low-ranked universities. The pie plot shows a clear split in research experience. 54.5% of applicants had no formal research background while 45.5% had been involved in research through projects, publications etc.</p><ul><li><strong>Check for the relationships existing between features and target</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SMtUcy5ULm1bp2lXYtheqg.png" /><figcaption>Numerical features relationship</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lla-fkJrlCtAfzTmDR7i1A.png" /><figcaption>Target-Features relationship</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/966/1*akITECp2CFZ-8i8N-w-lFw.png" /><figcaption>Research experience distribution based on admission status</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*KVVHBpL_7QfTF8fnhQdepg.png" /><figcaption>University rating distribution based on admission status</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*676_2TnhHoHbI0h_64_gJA.png" /><figcaption>Applicants Location distribution based on admission status</figcaption></figure><p>The images above uncovers patterns among grad school applicants.</p><blockquote>There’s a strong connection between <strong>GRE and TOEFL scores, </strong>those who scored high on one often do well on the other.</blockquote><blockquote>Applicants aiming for top-ranking universities tend to have exceptionally high <strong>GRE scores.</strong> This suggests that they recognize the competitve nature of these institutions and they work hard to meet the bar.</blockquote><blockquote>Applicants with high <strong>GRE and TOEFL scores </strong>also tend to submit well-written statements of purpose (<strong>SOP</strong>), receive strong letters of recommendation (<strong>LOR</strong>), and finished with a strong <strong>CGPA.</strong></blockquote><blockquote><strong>Age and location</strong> doesn’t seem to matter much. It shows no meaningful relationship with any other feature. This proves that in this process, it’s not how old you are or where you are from, but how prepared you are that counts.</blockquote><blockquote>Applicants with <strong>research experience </strong>were admitted more often</blockquote><h4>🛠️ <strong>Data Preprocessing and Feature Engineering</strong></h4><p>Based on the insights obtained during EDA, the following steps were taken to preprocess and engineer new features:</p><ul><li><strong>Replace abnormal entries</strong></li></ul><p>Recall that our data contained abnormal values, such as applicants having 12.84 as their CGPA. The code snippet below was used to replace these values</p><pre>#Replace cgpa values&gt;10 with 10, toefl score&gt;120 with 120, sop&gt;5 with 5, lor&gt;5 with 5, and gre score&gt;340 with 340<br><br>train_data = data1.copy()<br>test_data = data2.copy()<br><br>capping_rules = {<br>    &quot;GRE Score&quot;:340,<br>    &quot;TOEFL Score&quot;:120,<br>    &quot;SOP&quot;:5,<br>    &quot;LOR &quot;:5,<br>    &quot;CGPA&quot;:10<br>}<br><br>for col,max_val in capping_rules.items():<br>    train_data[col] = train_data[col].clip(upper=max_val)<br>    test_data[col] = test_data[col].clip(upper=max_val)<br><br><br>#Replace negative and zero values with 1 which should be the minimum<br><br>replace_rules = {<br>    &quot;SOP&quot;:1,<br>    &quot;LOR &quot;:1<br>}<br><br>for column,min_val in replace_rules.items():<br>    train_data[column] = train_data[column].clip(lower=min_val)<br>    test_data[column] = test_data[column].clip(lower=min_val)</pre><ul><li><strong>Handle missing values</strong></li></ul><p>To handle missing values, i used <strong>iterative imputer </strong>instead of just filling the missing values with their average even though most of the features were normally distributed. Filling missing data with mean values can be too simplistic because it treats every applicant the same ignoring patterns and relationships between features. For example, it assumes that someone with a low <strong>GRE score</strong> should always have the same average <strong>CGPA</strong> with someone with a high <strong>GRE score</strong>. Honestly for me, i fill missing values with mean when carrying out EDA to avoid error because of the missing values present in the data.</p><p>Iterative imputer uses regression models to predict each missing value based on the rest of the applicant’s data. It first looks through the complete dataset across all applicants to understand the relationship between features. When it sees a missing value for one applicant (say <strong>CGPA</strong>), it uses the regression model trained on the rest of the data to predict what the missing <strong>CGPA</strong> should likely be based on the other features for that specific applicant and how those features relate to <strong>CGPA</strong> across the whole dataset.</p><p>The code snippet below shows how the missing values were filled using iterative imputer:</p><pre>#Replace missing values using iterative imputer<br>missing_features = [&quot;GRE Score&quot;,&quot;TOEFL Score&quot;,&quot;SOP&quot;,&quot;CGPA&quot;]<br>imputer = IterativeImputer(max_iter=30, random_state=11)<br># Impute missing values<br>train_data[missing_features] = imputer.fit_transform(train_data[missing_features])<br>test_data[missing_features] = imputer.transform(test_data[missing_features])</pre><ul><li><strong>Features engineering</strong></li></ul><p>Before developing new features, it was noticed from EDA that some features were redundant. These features won’t be useful to GradScope during predictions. These features include:</p><blockquote>Age</blockquote><blockquote>Location</blockquote><blockquote>First generation (I honestly had no idea what this feature meant, but it was of no use to GradScope)</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2_VdW1sexrht7Nkswoq2dQ.png" /><figcaption>Heatmap of each feature</figcaption></figure><p>These features were not included during modelling.</p><p>The following features were created and added to further enhance the model’s (<strong>GradScope</strong>) ability to output correct predictions.</p><pre>#weighted average of the standardised test scores<br>train[&#39;academic_strength&#39;] = 0.8 * train[&#39;GRE Score&#39;] + 0.2 * train[&#39;TOEFL Score&#39;] <br>test[&#39;academic_strength&#39;] = 0.8 * test[&#39;GRE Score&#39;] + 0.2 * test[&#39;TOEFL Score&#39;]<br>#=====================================================================================================<br>#Feature interaction between GRE scores and university rating<br>train[&#39;Gre_university&#39;] = train[&#39;GRE Score&#39;] * train[&#39;University Rating&#39;]<br>test[&#39;Gre_university&#39;] = test[&#39;GRE Score&#39;] * test[&#39;University Rating&#39;]<br>#=====================================================================================================<br>#Research strength of applicants<br>train[&#39;Research_strength&#39;] = train[&#39;Research&#39;] + (train[&#39;SOP&#39;] + train[&#39;LOR &#39;] + train[&#39;CGPA&#39;] + train[&#39;GRE Score&#39;])<br>test[&#39;Research_strength&#39;] = test[&#39;Research&#39;] + (test[&#39;SOP&#39;] + test[&#39;LOR &#39;] + test[&#39;CGPA&#39;] + test[&#39;GRE Score&#39;])<br>#=====================================================================================================</pre><p>academic_strength: This feature combines <strong>GRE score</strong> and <strong>TOEFL score</strong> into a single metric with more weight on<strong> GRE score </strong>because <strong>GRE score</strong> had a stronger relationship with the <strong>admitted</strong> class. These two features are strongly correlated and combining both could help improve the model(<strong>GradScope</strong>).</p><p>Gre_university:This feature was designed to capture how an applicant’s GRE score aligns with the competitiveness of the university applied to. For example, an applicant that selected a top-tier university and had a high <strong>GRE score</strong> stands a good chance of getting admitted than an applicant who selected a top-tier university and didn’t really do well in GRE.</p><p>Research_strength:This feature was designed to summarize an applicant’s academic and research profile.</p><p>🤖 <strong>Model(GradScope) Development</strong></p><p>Remember that the target admitted is imbalanced, this needs to be addressed before training <strong>GradScope. <br></strong>I decided to use class weight (cost-sensitive learning) to address imbalance because the difference between the number of admitted and non-admitted applicants was small, so i decided it was best to avoid introducing synthetic data points with SMOTE or duplicated entries with RandomOverSampler.</p><p>The code snippet below was used to calculate what weight to assign each class:</p><pre>def calculate_class_weights(y):<br>    #Function to calculate class weights based on class distribution in the target variable.<br>    unique_classes, class_counts = np.unique(y, return_counts=True)<br>    total_samples = len(y)<br>    class_weights = {}<br><br>    for class_label, class_count in zip(unique_classes, class_counts):<br>        class_weight = total_samples / (2.0 * class_count)<br>        class_weights[class_label] = class_weight<br><br>    return class_weights<br><br># calculate the class weight for each class<br>class_weights = calculate_class_weights(target)<br>print(class_weights)</pre><pre>{0(not_admitted): 1.4937759336099585, 1(admitted): 0.7515657620041754}</pre><p>As you can see from the output, more weight was assigned to the class (not_admitted) with fewer samples. This tells GradScope something like Hey! give extra attention and importance to the not-admitted class because they are fewer, its going to be more costly for you if you get this minority class wrong.</p><p>The dataset was split into training and validation sets and Stratified K-Fold cross validation was used to ensure and check:</p><blockquote>That each training fold gets even distribution of both class</blockquote><blockquote>That <strong>GradScope</strong> was indeed learning and generalizing well across all validation folds.</blockquote><pre>#split the data into training and validation<br>X_train, X_val, y_train, y_val = train_test_split(train, target, test_size=0.1, random_state=23, stratify=target)<br><br>#xgboost stratified kfold cross validation<br><br>xgboost_model = xgb.XGBClassifier(reg_lambda= 10, reg_alpha=1, n_estimators=1443, min_samples_split=6, max_depth=11, learning_rate=0.014,<br>                                  class_weights=class_weights,random_state=20)<br>mean_f1_xgboost = stratified_kfold_cv(xgboost_model, X_train, y_train, n_splits=5,<br>                                           class_weights=class_weights, verbose=True) <br># Print the mean F1-score across folds<br>print(f&quot;Mean F1-Score across folds (XgBoost): {mean_f1_xgboost:.4f}&quot;)</pre><pre>Fold 1/5<br>F1 Score: 0.9046<br>Fold 2/5<br>F1 Score: 0.9085<br>Fold 3/5<br>F1 Score: 0.9068<br>Fold 4/5<br>F1 Score: 0.8902<br>Fold 5/5<br>F1 Score: 0.8974<br>Mean F1 Score: 0.9015<br>Mean F1-Score across folds (XgBoost): 0.9015</pre><p>Xgboostwas not just chosen randomly, i tested other models too like RandomForest, LightGBM, Catboost, RidgeClassifier, AdaBoost, and Logistic Regression.The values of the hyperparameters was obtained during hyper-parameter tuning using RandomizedSearchCV. The F1-Scores indicates that <strong>GradScope</strong> was able to generalize well on the validation data across all 5 folds.</p><pre># Fit the model on the entire training data and <br># evaluate its performance with the validation dataset.<br>xgboost_model.fit(X_train, y_train)<br>xgboost_val_pred = xgboost_model.predict(X_val)<br><br>xgb_f1_score = f1_score(y_val, xgboost_val_pred)<br>xgb_acc_score = accuracy_score(y_val, xgboost_val_pred)<br>xgb_auc_score = roc_auc_score(y_val, xgboost_val_pred)<br><br>print(classification_report(y_val, xgboost_val_pred))<br>print(f&quot;F1 Score: {xgb_f1_score:.4f}&quot;)<br>print(f&quot;AUC Score: {xgb_auc_score:.4f}&quot;)<br>print(f&quot;Accuracy Score: {xgb_acc_score:.4f}&quot;)</pre><pre>                 precision    recall  f1-score   support<br><br>           0       0.79      0.78      0.79       120<br>           1       0.89      0.90      0.89       240<br><br>    accuracy                           0.86       360<br>   macro avg       0.84      0.84      0.84       360<br>weighted avg       0.86      0.86      0.86       360<br><br><br>F1 Score: 0.8940<br>AUC Score: 0.8396<br>Accuracy Score: 0.8583</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*1zN3pPjJE-YZ6uJRmI9pyg.png" /><figcaption>Confusion matrix of GradScope</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*YVqUqe3Y_P6juUA7icwafw.png" /><figcaption>ROC-AUC Curve</figcaption></figure><p>What do these results tell us?</p><blockquote><strong>GradScope</strong> achieved strong performance with an 89.4% F1-score, demonstrating excellent balance between precision and recall in classifying admission status for applicants. 85.83% accuracy score indicates robust overall prediction correctness, while the 83.96% AUC-ROC score confirms good (though not perfect) ability to distinguish between admitted and non-admitted applicants. Together, these metrics suggest that <strong>GradScope</strong> is highly reliable for identifying successful applicants while maintaining reasonable discrimination across probability thresholds.</blockquote><p>In other words, <strong>GradScope</strong> is doing well!!</p><p>Let’s look at what features were important to <strong>GradScope</strong> when predicting…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*FI623KKcJuC6gi9Xn-bnUQ.png" /><figcaption>GradScope(Xgboost) feature importance</figcaption></figure><p>The plot above tells us while all the features were important, some were more important to GradScope.</p><p>When it comes to grad school admissions, the data tells a clear story. It’s not just about having a high <strong>CGPA or standardized test scores, </strong>it’s about where you aim and what you bring beyond the numbers.</p><p><strong>GradScope</strong> found that three features stood out the most in predicting who gets admitted.</p><p>Gre_university:A powerful combination of <strong>GRE Scores</strong> and <strong>University Rating. </strong>This reveals that applicants who selected top-tier school with very high rating and scored very high in their GRE exams often gets admitted.</p><p>Research:Research experience also ranked among the top 3 contributors. Applicants who had publications, projects or any kind of prior research clearly held an advantage over those without them.</p><p>Research_strength:One of the features engineered during the feature engineering process is a blend of research experience, academic grades/scores, quality of SOP and recommendation letters. This emphasizes that admission committees value applicants who cannot only perform academically but also show clear research skills.</p><p>Now that you’ve understood how <strong>GradScope</strong> was developed, let’s get back to Simon… You remember Simon right? Great.</p><p>Simon took Gabriel’s advice and used GradScope via the app.</p><p>Note: Admission into graduate school does not solely depend on the factors discussed above, other factors like your academic CV or interview session with the admission board also come in play. However GradScope is going to help you check if your academic profile is strong enough to get you admitted.</p><p>Simon using GradScope: <a href="https://github.com/obinna-Muonanu/GradScope/blob/main/streamlit-GradScope-2025-04-30-13-04-65.webm">Here</a></p><p>Simon saw that his <strong>SOP</strong> and <strong>LOR </strong>wasn’t so good, there and then he knew he had to improve them if he must stand a chance of getting admitted.</p><p>That’s it guys!!!</p><p>Thank you for staying till the end.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*_16qkFUWhD-TJlRe" /></figure><p>If you’re interested in the codes both for modelling and app development, check it out here on my github page <a href="https://github.com/obinna-Muonanu/GradScope">Github</a></p><p>You can also check out the app here: <a href="https://gradscope-cnw9tu8jqmmukyaz7dfprr.streamlit.app/">GradScope App</a></p><p>See you soon!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8409b254edbf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Computers See: Understanding Convolutional Neural Networks (CNN)]]></title>
            <link>https://medium.com/@obinnamuonanu/how-computers-see-understanding-convolutional-neural-networks-cnn-6bca9c68f69c?source=rss-d5a59b36b048------2</link>
            <guid isPermaLink="false">https://medium.com/p/6bca9c68f69c</guid>
            <dc:creator><![CDATA[Obinnamuonanu]]></dc:creator>
            <pubDate>Thu, 27 Mar 2025 23:38:45 GMT</pubDate>
            <atom:updated>2025-03-27T23:38:45.374Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mWv5BWxpvD1a38xeoHo6uA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@opticonor?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Conor Luddy</a></figcaption></figure><p>Imagine trying to recognize your friend in a crowded environment like the one in the picture below, do you know what actually happens behind the scenes? Obviously you’ve known your friend for a long time, but how do you think you were able to recognize him/her from a crowd of people? Your brain scans the crowd, focusing on familiar features like hair, skin colour, height, face, nose shape, walking posture and the likes and finally spots them. You were able to recognize your friend based on these features.</p><p>Here’s another case scenario, let’s say your mom offers you an apple, but she gives you a bowl containing apples and oranges instead. How did your brain correctly identify the apple? Its simple, your brain focuses on features that are peculiar to the apple and of course different from that of an orange, features like colour, texture, shape etc. This is similar to how CNN’s see and recognize images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yY_d8soo3McxTZEgYFF2_Q.jpeg" /><figcaption>Image by <a href="https://unsplash.com/@niceyem?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash">Opeyemi Adisa</a></figcaption></figure><h4>What exactly is Convolutional Neural Networks (CNN)?</h4><p>Convolutional neural networks also known as ConvNets are a special and powerful type of deep learning models that helps computers see and recognize visual data just like humans do. They are specifically designed to analyze visual data like images and videos. It works by breaking down images into small parts, looking for patterns (like edges, shapes, colours etc…), and then learning to identify objects based on those features just like we described in the examples above.</p><p>Before CNNs, engineers and developers had to manually define image features (such as detecting shapes using mathematical filters), you could imagine how daunting and time consuming the process will be. CNNs now automate this process making them powerful and efficient. They’ve achieved state-of-the-arts results in numerous applications like <em>f</em><strong><em>acial recognition, image classification, medical scans, self-driving cars</em></strong><em> </em>and more.</p><p>Now that you have a solid grasp of what CNNs are, let’s understand how they work. <strong>I recommend having a solid understanding of deep learning and neural networks.</strong> Don’t be scared if you don’t, i’ll break down CNN’s in a way that’s easy to understand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*FPFn2_MISojeNpp2hwv8yg.gif" /></figure><h4>How does a CNN work?</h4><p>To understand how CNNs works, we need to first of all know and understand its architecture. CNNs consists of <strong>four main layers</strong></p><blockquote>Convolutional layer</blockquote><blockquote>Activation function (ReLu)</blockquote><blockquote>Pooling layers</blockquote><blockquote>Fully-connected layers</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*G1Lq98vX1dUYGH6f.jpg" /></figure><p>Let’s break each layer down and go deeper into each part of the CNN architecture to fully understand how it works.</p><h4>The Convolutional Layer</h4><p>The convolutional layer is the core building block of CNNs and where the actual magic happens. It is responsible for extracting features like shapes, edges, texture, colour and other features in an image. It does this from the input image by applying a small filter also known as kernel that slides over the image in steps to detect these features. Think of it like using a magnifying glass moving slowly across a map to see the hidden locations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*KvdJTv3wk6apKCN-.jpg" /><figcaption>Image source: Pinterest</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*N68as4k0VV4aKK6yhxkgkA.png" /><figcaption>Image source: ResearchGate</figcaption></figure><p>Images are represented as a matrix of pixel values. A grayscale image also known as white and black image is a 2D matrix where each value represents brightness (0=black, 255=white). A colour image (RGB) is a 3D matrix with three channels (red, green, and blue).</p><p>From the illustration above, the image is represented as a grid of pixel values, the filter/kernel is a 3 x 3 matrix represented by the matrix in the middle. There are different kinds of filters and each serves a specific purpose, some detect horizontal edges, some detect loops, textures, blur etc. The filter slides over the image (this movement is known as <strong>stride</strong>), and performs an element-wise multiplication between the filter values (this values are learned by the network during training) and the corresponding pixel values of the image. The resulting matrix after multiplication is known as a <strong>feature map (convolved feature)</strong> and this process is known as a <strong>convolution operation.</strong></p><pre>Let&#39;s see how the feature map came about<br><br>perform element-wise multiplication<br>[(1.2 x 1.0) + (1.5 x 0) + (2.1 x 1.0) + (0 x 0) + (1.0 x 1.0) + (1.0 x 0) +<br>(0 x 1.0) + (0 x 0) + (1.0 x 1.0)] = 5.3<br><br>Taking a stride of 1, the 3x3 filter values include:<br>[[1.5, 2.1, 0],<br>[1.0, 1.0, 1.0],<br>[0, 1.0, 1.0]]<br><br>perform element-wise multiplication<br>[(1.5 x 1.0) + (2.1 x 0) + (0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (1.0 x 0) +<br>(0 x 1.0) + (1.0 x 0) + (1.0 x 1.0)] = 3.5<br><br>Taking a stride of 1, the 3x3 filter values include:<br>[[2.1, 0, 0],<br>[1.0, 1.0, 0],<br>[1.0, 1.0, 1.0]]<br><br>perform element-wise multiplication<br>[(2.1 x 1.0) + (0 x 0) + (0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (0 x 0) +<br>(1.0 x 1.0) + (1.0 x 0) + (1.0 x 1.0)] = 5.1<br>=========================================================================<br>Taking a stride of 1, the 3x3 filter values include:<br>[[0, 1.0, 1.0],<br>[0, 0, 1.0],<br>[0, 0, 1.0]]<br><br>perform element-wise multiplication<br>[(0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (0 x 0) + (0 x 1.0) + (1.0 x 0) +<br>(0 x 1.0) + (0 x 0) + (1.0 x 1.0)] = 2.0<br><br>Taking a stride of 1, the 3x3 filter values include:<br>[[1.0, 1.0, 1.0],<br>[0, 1.0, 1.0],<br>[0, 1.0, 1.0]]<br><br>perform element-wise multiplication<br>[(1.0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (0 x 0) + (1.0 x 1.0) + (1.0 x 0) +<br>(0 x 1.0) + (1.0 x 0) + (1.0 x 1.0)] = 4.0<br><br>Taking a stride of 1, the 3x3 filter values include:<br>[[1.0, 1.0, 0],<br>[1.0, 1.0, 1.0],<br>[1.0, 1.0, 0]]<br><br>perform element-wise multiplication<br>[(1.0 x 1.0) + (1.0 x 0) + (0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (1.0 x 0) +<br>(1.0 x 1.0) + (1.0 x 0) + (0 x 1.0)] = 3.0<br>============================================================================<br>Taking a stride of 1, the 3x3 filter values include:<br>[[0, 0, 1.0],<br>[0, 0, 1.0],<br>[0, 1.0, 1.0]]<br><br>perform element-wise multiplication<br>[(0 x 1.0) + (0 x 0) + (1.0 x 1.0) + (0 x 0) + (0 x 1.0) + (1.0 x 0) +<br>(0 x 1.0) + (1.0 x 0) + (1.0 x 1.0)] = 2.0<br><br>Taking a stride of 1, the 3x3 filter values include:<br>[[0, 1.0, 1.0],<br>[0, 1.0, 1.0],<br>[1.0, 1.0, 0]]<br><br>perform element-wise multiplication<br>[(0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (0 x 0) + (1.0 x 1.0) + (1.0 x 0) +<br>(1.0 x 1.0) + (1.0 x 0) + (0 x 1.0)] = 3.0<br><br>Taking a stride of 1, the 3x3 filter values include:<br>[[1.0, 1.0, 1.0],<br>[1.0, 1.0, 0],<br>[1.0, 0, 0]]<br><br>perform element-wise multiplication<br>[(1.0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (0 x 0) +<br>(1.0 x 1.0) + (0 x 0) + (0 x 1.0)] = 4.0</pre><pre>You can also take a stride of 2<br>From the image, let&#39;s take a stride of 2 instead of 1<br><br>perform element-wise multiplication<br>[(1.2 x 1.0) + (1.5 x 0) + (2.1 x 1.0) + (0 x 0) + (1.0 x 1.0) + (1.0 x 0) +<br>(0 x 1.0) + (0 x 0) + (1.0 x 1.0)] = 5.3<br><br>Taking a stride of 2, the 3x3 filter values include:<br>[[2.1, 0, 0],<br>[1.0, 1.0, 0],<br>[1.0, 1.0, 1.0]]<br><br>perform element-wise multiplication<br>[(2.1 x 1.0) + (0 x 0) + (0 x 1.0) + (1.0 x 0) + (1.0 x 1.0) + (0 x 0) +<br>(1.0 x 1.0) + (1.0 x 0) + (1.0 x 1.0)] = 5.1<br>...<br>...<br>...</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/562/0*mMtwDXwftn9yAPMo.PNG" /><figcaption>convolution operation with stride=1</figcaption></figure><p>You might wonder how the size of the feature map matrix is determined. For example, why is the feature map matrix above a 4x4 and not a 3x3 or 5x5 matrix?</p><p>The output feature map size is calculated with the formula below:</p><pre>Without Padding<br>O = [(I - F)/S] + 1<br><br>where <br>O = Output feature map size<br>I = Input image size<br>S = stride</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h7yyzFrpnToltB6y.png" /><figcaption>convolution operation with stride=2</figcaption></figure><p>Padding</p><p>Padding is a technique used in convolutional neural network to control or maintain the size of an output feature map after applying convolution operation. It involves adding extra pixels usually zeros around the input image before applying convolution.</p><p>Why do we need padding?</p><ol><li>To preserve the original dimensions of the image after convolution. This helps the network learn better by not shrinking the image too fast. It also makes sure that important information especially at the edges are not lost.</li><li>To ensure that edge pixels are equally processed as center pixels.</li><li>To improve feature extraction especially at the boundary of the image.</li><li>To allow deeper networks without rapid size reduction.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XWS3jXwXwMf4SRWj.png" /><figcaption>Input Image with zero Padding. Image source: <a href="https://images.app.goo.gl/D4ErJZG6AuqCn5zg6">https://images.app.goo.gl/D4ErJZG6AuqCn5zg6</a></figcaption></figure><p>If you consider the image above without padding and stride=1, the size of the feature map would be a 4x4 matrix after convolution operation. Notice how the numbers at the edges would appear just once as the filter moves across the matrix. If these were important features in the image the network would probably miss it ultimately leading to a less accurate model.</p><p>With padding, however, the numbers at the edges get to appear more than once as the filter moves across the matrix, giving the network enough time to learn the important features at the edges.</p><p>There are 2 kinds of padding:</p><ol><li>Valid padding — No padding</li><li>Same padding — commonly known as zero padding, although its not limited to just adding zeros. Ones and negative values may be used.</li></ol><p>What then happens after convolution?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*8mCWQ4sX991dR5x-.gif" /></figure><p><strong>Activation Function (ReLU)</strong></p><p>After extracting the features using convolution, the output is passed through ReLU (Rectified Linear Unit), the most commonly used activation function in CNN. Asides introducing non-linearity into the network to allow it learn complex patterns (images are not purely linear), the output from the convolutional layer may contain negative values especially when padded with zeros, which may not be useful. ReLU removes the negative values by converting them to zeros while keeping the positive values as they are.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/241/1*_aJDEoaa0ze0VDxNIqwxTg.png" /><figcaption>Image by ResearchGate: ReLU activation function.</figcaption></figure><p><strong>Pooling Layers</strong></p><p>Once the important features have been extracted after passing the input through the convolution layer and activation function, it is typical that the dimensions of the output from the convolutional layer is reduced periodically using pooling layers. You can think of pooling layers as a way to summarize the features from a feature map (after activation). Pooling layers reduces the number of parameters in the network. This is a desirable effect as this reduces the computations required for training. It also helps reduce the effect of overfitting (memorizing noise instead of general patterns), making CNNs faster and more efficient while keeping critical features intact.</p><p>Similar to the convolution layer, pooling operations also involve moving filters across the image in strides just like we discussed earlier. Unlike the filters in the convolution layer, the filters here do not have weights (recall that these weights in the convolution layer are learned during training), instead the filters applies an aggregation function to the values in the feature map. This brings us to the 2 kinds of pooling layers:</p><ol><li>Max Pooling : selects the maximum value from a region of the feature map. It helps highlight dominant features.</li><li>Average Pooling : selects the average value from a region of the feature map. It helps retain more background information and smoothen feature maps.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*bdWwI8NHozsgcBJsPTipzA.jpeg" /><figcaption>Max Pooling and Average Poolin</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*3opDG8RKiyaE96Gc" /><figcaption>Max Pooling vs Average pooling</figcaption></figure><p>We’ve come a long way! Let’s talk about the last layer which is the Dense layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*vrTq7jspnXBr638Z" /></figure><p><strong>Fully-connected Layer (Dense Layer)</strong></p><p>After several rounds of convolution and pooling, the final layer of the convolutional neural network is the fully-connected layer which takes in the extracted features flattened into a single long vector. The flattened vector is sent to a traditional neural network which consists of neurons and weights. The fully connected layer literally works like a regular deep learning model but the difference is that it processes CNN extracted features. It connects every neuron from the previous layer to every neuron in the next layer and makes predictions (final output). While the convolution layer tends to use the ReLU activation function, the fully-connected layers usually leverage functions like sigmoid (for binary classification), softmax (for multi-class classification), and ReLU (for regression).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/0*1aAslPwan7mneaIf" /><figcaption>Fully-connected Layer</figcaption></figure><p>Putting everything together…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*moRSIIlydB7uwFKA.png" /><figcaption>CNN Architecture</figcaption></figure><p>Having explored the four main layers of CNNs, we can generally say from the image shown above, that the network takes in an image and passes it through a series of convolution (plus activation function), pooling layers where feature extraction takes place. The result is a feature map which is then flattened and fed into a fully connected layer where the output is predicted.</p><p>Congratulations!! You now have a solid understanding of how CNNs works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/0*yGK8gFBqtOcKfSrx.gif" /></figure><p><strong>Thank you for reading and keep learning!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6bca9c68f69c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>