<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by A. Jeremy Mahoney on Medium]]></title>
        <description><![CDATA[Stories by A. Jeremy Mahoney on Medium]]></description>
        <link>https://medium.com/@a.jeremymah?source=rss-17b3955c7601------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*9HKVzfdtC4OI4_Rchlihqw.jpeg</url>
            <title>Stories by A. Jeremy Mahoney on Medium</title>
            <link>https://medium.com/@a.jeremymah?source=rss-17b3955c7601------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 21:38:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@a.jeremymah/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Learning as a Sacrament: School without Grades?]]></title>
            <link>https://medium.com/@a.jeremymah/learning-as-a-sacrament-school-without-grades-a6e78a10bf6a?source=rss-17b3955c7601------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6e78a10bf6a</guid>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[spirituality]]></category>
            <category><![CDATA[education]]></category>
            <dc:creator><![CDATA[A. Jeremy Mahoney]]></dc:creator>
            <pubDate>Wed, 27 Sep 2023 21:36:39 GMT</pubDate>
            <atom:updated>2023-09-27T21:36:39.972Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aql8JIjhah55vMjwaYTAtg.jpeg" /></figure><p>I’ve always ranked learning amongst the purest forms of joy. There are no goosebumps quite like those that spread after visualizing (and fully understanding) a mathematical object that Da Vinci couldn’t have painted in a hundred years. Or when you’re just starting to get mathematically mature enough to read a textbook in the same way an orchestra conductor reads a 16-part score: experiencing not the dots on the page, but only the soaring mental music that they outline. A flavor of this goosebumpy feeling exists in every topic worth thinking about. In history, it’s found in the creeping fractal of connections between random-seeming events. In literature, it arises from the dizzying layers of meaning within the best of works. In music, you don’t even need to think to get those goosebumps, but knowing a little harmonic theory can electrify even gas station muzak. This is the stuff of passion, poetry, intellectual lust.</p><p>Considering that this joy exists in the pages of every half-decent book that lines the shelves of the university library, it seems odd that it’s rarely explicitly discussed amongst students. Young adults crave peak experiences–concerts, travel, drugs, love. And no doubt, in their best forms, such things are wondrous. But how many folks realize just how far they can go by sitting down and thinking really hard in the right ways?</p><p>It’d be lovely to have institutions whose main purpose is to help people discover the vast landscapes through which their minds can pilot them. Not that people need institutions to do this for them, as watching any book-nosed fifth grader or ham radio tinkerer makes clear. But just as the healthiest of religious groups have helped nurture the human spiritual instinct throughout history, there’s a place for communities of learning as well.</p><p>Our closest large-scale approximation to this ideal is the formal education system. However, the dominant emotion within this network of institutions is not the exhilaration of discovery, but rather heart-pounding stress. What has gone wrong?</p><p>Grades are far from the only problem, but they contribute. For one, they’re the only thing propping up the hoops all too many “classes” consist almost entirely of jumping through. They allow a “teacher” to declare victory upon the receipt from students of a set of papers with specific pencil marks on them, regardless of what the brains that produced these marks are up to. If grades (and thus my eligibility for my master’s degree, during which I’ll get to study great topics like operating systems and real analysis and get paid more money in the future for having done so) weren’t at play, I sure wouldn’t be paying too much attention to accurately regurgitating Monsanto-funded “nutrition” advice on my health quizzes. Without grades, dumb coursework would wither. But I suspect the real stuff would remain alive and well; the math majors who sat next to me in Combinatorial Analysis were hard and hungry men, eager to kill for cognitive fodder.</p><p>I admit that grades can, at times, feel marginally useful for helping to delay gratification to work on something difficult and worthwhile. I tend to be in the mood to go climbing for the fourth time in a week over working probability questions a little more often than the most Apollonian version of myself would prefer. The threat of grades can make those problem sets a little more enticing. But even here, I hesitate to declare that grades are truly useful. After all, such a scenario presents an ideal opportunity to practice observing the transience of one’s preferences. Getting that math done requires generating motivation from fatigue, turning the proverbial water into wine at will–a sagacious skill far beyond the scope of any syllabus. In a gradeless system, every successful pupil would stand a chance at being as fluent in this sorcery as today’s well-coached SAT taker is at “choosing the best answer” to a multiple choice question.</p><p>There’s no doubt that dispensing with grades would present a few administrative difficulties. The question of objectively evaluating students arises, along with the question of what to do with students who completely slack off (to which “almost nothing” may in fact be a reasonable answer–an experienced friend of mine assures me that there’s nothing like a couple years of sooty factory work to whet one’s appetite for the intellectual). As for the first question, any internship-hunting CS major will assure you that software companies are already a step or two ahead of universities. And you’d be amazed at just how good the prompt “provide one example of an invertible matrix” is for filtering out data science applicants.</p><p>If I could teach a class with total freedom, my grading philosophy would be as follows: forget about the damn letters, pour your heart and mind into doing a first-rate job, and never lose sight of the beauty, the drama, the humor inherent in everything we’re doing. Seems to me about as good a way to teach a class as to live a life. And maybe that’s what we’ve really been trying to teach all along.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6e78a10bf6a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An Overlooked Superpower: How to Explain Complex Concepts]]></title>
            <link>https://medium.com/@a.jeremymah/an-overlooked-superpower-how-to-explain-complex-concepts-2dd14573ac13?source=rss-17b3955c7601------2</link>
            <guid isPermaLink="false">https://medium.com/p/2dd14573ac13</guid>
            <category><![CDATA[writing-tips]]></category>
            <dc:creator><![CDATA[A. Jeremy Mahoney]]></dc:creator>
            <pubDate>Fri, 11 Sep 2020 07:30:35 GMT</pubDate>
            <atom:updated>2022-10-24T23:33:52.916Z</atom:updated>
            <content:encoded><![CDATA[<p>The ability to effectively explain complex concepts isn’t just awesome. It’s a service to the world.</p><p>We live in a time with a lot of cool things to learn about. Here are just a few:</p><ul><li>Brain computer interfaces</li><li>Space travel</li><li>Artificial intelligence</li><li>Cryptocurrency</li><li>Language teaching with machine learning</li></ul><p>Besides being interesting, all these topics have one key thing in common.</p><p>They’re all complicated.</p><p>Or at least they all sound complicated. The fact that they sound complicated prevents most of us from ever delving into them.</p><p>But the thing is, once we delve into these ideas and start understanding them, we start realizing how amazing they are and can’t imagine not knowing anything about them. When a concept really <em>clicks</em>, the entire world suddenly seems a little more exciting. If you know what I’m talking about, you know it’s an awesome feeling. It’s the kind of thing that can totally make your day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-18mFIkLABeeuPFTNKBA5g.jpeg" /></figure><p>And at least for me, usually when something <em>clicks</em> in my mind like that, it’s because someone explained it really well. Sometimes it’s in person, sometimes it’s in a textbook, and sometimes it’s because someone made a solid infographic and posted it on Medium. But somewhere behind things, there’s always a person who’s a good explainer.</p><p>Good explainers are epic.</p><p>The thing is, it’s not like people are just born being able to explain stuff well. It takes a lot of practice, and anyone can learn how to do it.</p><p>If you learn to explain things well, you can give tons of people all over the world their own amazing <em>click</em> moments about anything you want them to understand. <strong>That is a seriously powerful ability.</strong></p><p>I don’t claim to be any kind of expert explainer, but I always seek to get better at explaining stuff. As I’ve started writing on Medium, I’ve discovered 5 really useful things to keep in mind that I think have made me a little better at explaining things, and I want to share these 5 things with you here.</p><h3>1. Convince your audience that they’re capable of understanding your topic.</h3><p>If you can do this successfully, you’re halfway there. This is also part of the reason that people who think that they can learn anything somehow seem to have a way of learning anything they put them mind to.</p><p>The key is to get your audience in a state of mind in which they can easily imagine themselves understanding a topic that they don’t yet understand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*IG8yMMiJE6lO9Zu8bei41g.jpeg" /></figure><p>To achieve this, there are several specific things that you can do.</p><p><strong>If you’re explaining a concept that’s often associated with math, such as the inner workings of AI, then make it clear the depth of understanding the reader can achieve with their current math knowledge. </strong>Even if this level of understanding doesn’t seem very deep to you (as someone who knows the topic), being honest with your audience will help them develop realistic expectations for themselves. Such expectations will make learning about your topic feel more realistic to your audience, which makes it more likely that your audience will absorb what you’re saying.</p><p><strong>Acknowledge that that topic is challenging. </strong>This is super important. If you’re explaining a complicated concept that’s challenging for your audience to understand, and you tell your audience that the topic is simple, then your audience will feel stupid. Even if you accidentally <em>imply</em> that the concept is simple, your audience will still feel stupid for not catching on quickly. Make it abundantly clear that the things you’re explaining take some time and mental effort to absorb.</p><p><strong>Periodically summarize things to remind your audience how much they’ve learned by consuming even part of your explanation.</strong> If your audience is halfway through consuming your explanation and they realize that they already know way more than they did when they started, they’re very likely to want to hear the rest of what you have to say. To help spur these moments of realization, take mental “breathers” every once in a while where you comment on how far you and your audience have come.</p><p>The next four tips will also help convince your audience that they’re capable of understanding the concept you’re writing about. If your writing is decently clear, then instilling confidence in your readers may be the most important thing you can do.</p><h3>2. Build a strong foundation before moving up.</h3><p>Tech entrepreneur Elon Musk has learned a staggering number of things throughout his life. Musk taught himself rocket science from textbooks and founded a space company. He built the world’s most popular electric car without any auto industry experience. He’s become an expert on solar power by mindfully interrogating engineers.</p><p>When people ask Musk how he did it, he often attributes his success to his strong grasp of “first principles” in each industry. Comparing learning to a tree, he always stresses the importance of developing a strong foundation of basic understanding before moving onto complicated concepts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/849/1*z1_xD4EUHDtMOu5OCayo4g.jpeg" /></figure><p>This is excellent general advice when it comes to explaining complex topics. The more complex the end of your explanation is, the simpler the beginning should be.</p><p>If you try to explain something without first giving your audience a proper foundation of understanding, the information has almost no chance in sticking in your audience’s mind, just like leaves have almost no chance of floating if you release them in midair. But if you gradually build up your audience’s understanding from the simplest level, then your audience will have the foundation they need to retain the information you give them, just as leaves easily stick on a sturdy tree.</p><p><em>This </em>is how Elon Musk says he remembers so much. He has a strong enough foundational understanding of enough fields that he’s able to connect nearly anything new he learns to a whole network of things he already knows. This allows new information to stick in his brain easily.</p><p>We want to replicate this feeling in both ourselves as we research and in our readers’ minds.</p><ul><li><strong>When researching,</strong> focus on giving <em>yourself</em> that foundational knowledge before even trying to explain anything. Don’t be afraid to go down rabbit holes online to reach the bedrock of understanding. It’s often deeper down that you expect.</li><li><strong>When explaining,</strong> start at bedrock understanding and work your way up. Even if your explanation is longer than it would have been if you had started closer to your final goal, your audience is more likely to stay engaged if they feel like they understand what you’re saying.</li></ul><p>Finally, as you work your way up in your explanation, do your best to move in chunks. This lets your audience focus on one section at a time, which increases their confidence that they can understand the section they’re reading. It also gives them those periodic opportunities to realize that they’ve learned a lot halfway through your explanation, which increases their engagement.</p><h3>3. Include drawings for technical concepts.</h3><p>I have yet to meet somebody who doesn’t appreciate diagrams/drawings when they’re trying to learn something hard.</p><p>There are a few guidelines I’d suggest following to make your drawings as effective as possible.</p><ol><li><strong>Simple does the job.</strong> Extra lines are confusing. Your audience should be able to make some sense of a chart or drawing within 5 seconds, and near-instantaneously is ideal.</li><li><strong>Stay unpretentious.</strong> It’s remarkable how far stick figures can get you. If you want to see well-placed stick figures in action, visit <a href="https://waitbutwhy.com/">Tim Urban’s legendary blog, Wait But Why</a>.</li><li><strong>Add some color! </strong>For maximum clarity, once you’ve used a color to represent one concept, use only that color for only that concept for the rest of your explanation. Deliberately bringing back a color from earlier in your explanation when you’re bringing back the same concept can be a great way to remind your audience what you were talking about.</li></ol><p>Simple, colorful illustrations that are clear and sometimes border on childish can contrast very nicely with technical subject matter. They give your audience something to psychologically hang on to as you take them out to unknown worlds.</p><h3>4. Build a narrative, if appropriate.</h3><p>Humans respond well to stories that involve people. We evolved to be good at keeping track of people, and some linguists estimate that over 80% of the words that come out of our mouths each day can be considered gossip.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/0*Bha5SG4yOXuhCLoO.jpeg" /></figure><p>Building a narrative with characters–even a simple one–can have several powerful effects.</p><ol><li><strong>Narratives give your story chronological structure.</strong> This structure makes it much easier for your readers to remember the flow of ideas that you’re seeking to communicate.</li><li><strong>Narratives make abstract ideas concrete.</strong> This concreteness is a great way to give your readers a bedrock of understanding, because bedrock understanding often involves simple interactions between people. This holds true for countless complex topics, from cybersecurity to international geopolitics.</li><li><strong>Narratives prompt emotions, which aid in memory. </strong>Competitive memory athletes succeed by making visualizations that are so emotionally vivid that they couldn’t possibly forget them. Many studies show that linking concepts to even simple emotions can embed these concepts deeper in our memories, and all narratives prompt some kind of emotion.</li></ol><p>Don’t feel pressure to turn every single explanation you produce into a story. A narrative isn’t always the best format to use for a complex explanation, but it’s always something you should consider.</p><h3>5. Use easily understandable language.</h3><p>It’s important to tailor your writing style to be as easy to understand as possible. There’s not a single tone or register that you have to use, but you shouldn’t necessarily use the writing style that comes most naturally to you. Your tone and register should be a deliberate choice.</p><p>Personally, I try to use an informal register and a lighthearted, interested tone for my more technical articles. I think this combination of tone and register mimics an in-person explanation most effectively. If implemented well, reading good informal writing with stick figure drawings can feel a lot like listening to a great teacher explain something directly to you.</p><p>For pieces that involve data analysis, I deliberately switch to a slightly more formal register, because it’s important to be precise when discussing statistics. However, I try to keep my writing reasonably easy to understand for someone without much of a background in stats.</p><p>No matter what tone and register you select, there are several language tips that apply universally.</p><ol><li><strong>Always explain technical terms. </strong>You don’t need to completely avoid technical terms; they can be super useful if you and your audience are on the same page about things. But <em>always </em>make sure to clearly introduce and explain them. Jargon can make an explanation incomprehensible very quickly.</li><li><strong>Avoid long sentences. </strong>If you’re explaining something complex, the ideas you’re communicating will be plenty to keep your audience’s mind occupied. Don’t make them decipher long sentences too.</li><li><strong>Don’t be afraid of short paragraphs. </strong>This is especially true on platforms like Medium, where short paragraphs are the norm. Short paragraphs do a good job at mimicking the pauses we naturally take when speaking, which make an explanation feel more conversational.</li></ol><p>Just like anything else in life, explaining complex topics is something anyone can get good at with practice. The best way to get better at writing clear explanations is to practice a lot and constantly feedback!</p><p>Just remember that inducing <em>click</em> moments in others’ minds is truly one of the most powerful skills you can learn.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2dd14573ac13" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Credit Risk Modeling with Machine Learning]]></title>
            <link>https://medium.com/data-science/credit-risk-modeling-with-machine-learning-8c8a2657b4c4?source=rss-17b3955c7601------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c8a2657b4c4</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[credit-risk-analysis]]></category>
            <dc:creator><![CDATA[A. Jeremy Mahoney]]></dc:creator>
            <pubDate>Wed, 09 Sep 2020 02:29:18 GMT</pubDate>
            <atom:updated>2020-09-30T18:02:02.171Z</atom:updated>
            <content:encoded><![CDATA[<h4>What a real-world machine learning solution looks like — no background knowledge required</h4><p><strong><em>Note from Towards Data Science’s editors:</em></strong><em> While we allow independent authors to publish articles in accordance with our </em><a href="https://towardsdatascience.com/questions-96667b06af5"><em>rules and guidelines</em></a><em>, we do not endorse each author’s contribution. You should not rely on an author’s works without seeking professional advice. See our </em><a href="https://towardsdatascience.com/readers-terms-b5d780a700a4"><em>Reader Terms</em></a><em> for details.</em></p><p><strong>Credit risk modeling</strong>–the process of estimating the probability someone will pay back a loan–is one of the most important mathematical problems of the modern world. In this article, we’ll explore from the ground up how <strong>machine learning</strong> is applied to credit risk modeling. You don’t need to know anything about machine learning to understand this article!</p><p>To explain credit risk modeling with machine learning, we’ll first develop domain knowledge about credit risk modeling. Then, we’ll introduce four fundamental machine learning systems that can be used for credit risk modeling:</p><ul><li>K-Nearest Neighbors</li><li>Logistic Regression</li><li>Decision Trees</li><li>Neural Networks</li></ul><p>By the end of this article, you’ll understand how each of these algorithms can be applied to the real-world problem of credit risk modeling, and you’ll be well on your way to understanding the field of machine learning in general!</p><p>Let’s begin learning about what credit risk modeling is by looking at a simple situation.</p><h3>The Situation</h3><p>Say your buddy Ted needs ten bucks. You’ll want those bucks back, so he promises he’ll repay you tomorrow when you see him again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OdbkyVQK8e7JWzjGFgBkfw.jpeg" /><figcaption>Image created by author</figcaption></figure><p>But you’ve heard that Ted has a bit of a history with being difficult when it comes to repaying people. As a result, you know there’s a chance you’ll never see your ten dollars again.</p><p>Would you lend him the money?</p><p>To make this decision, you need another piece of information.</p><p>Just <em>how</em> difficult has Ted been in the past?</p><p>If he just forgot to pay back the wrong dude once, then you’re probably good. On the other hand, if he has a habit of getting deep into debt and fleeing to other countries, then you should probably keep your money to yourself.</p><p>Ted’s history tells us something about the <strong>risk </strong>of lending money to him.</p><p>If he’s been trustworthy in the past, then the <strong>risk</strong> of lending money to him is relatively low. If he’s been untrustworthy in the past, then the <strong>risk</strong> of lending money to him is relatively high.</p><p>The concept of risk gives you a logical way to make the decision about whether or not to lend Ted the money. First, determine how much risk you’re okay with. Then, figure out how risky Ted’s history makes lending money to him. If the risk of lending money to Ted isn’t greater than your maximum tolerated risk, then you can go ahead and lend him the money.</p><p>If it’s just Ted asking you for money, you’ll probably rely on your intuition and feelings to determine the risk associated with loaning money to Ted. This works well enough for just one person.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dszmcKTrX5LE8tI7Vg5XyA.jpeg" /><figcaption>Image created by author</figcaption></figure><p>But what if you had hundreds of Teds asking you for money?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iajnDizuqs0WistQRLq5Rw.jpeg" /><figcaption>Image created by author</figcaption></figure><p>You can’t get to know every one of these people so that you can decide whether loaning to them feels like a good idea. But if you just loaned to them randomly, you might loan money to some people who are never going to pay you back. You also might reject some perfectly responsible people who would pay you back quickly. Instead, you’d want to use a more scientific process.</p><p>This scientific process is called <strong>credit risk modeling</strong>, and it’s what we’ll be exploring in this article.</p><p>Formally speaking, <strong>credit risk modeling</strong> is the process of using data about a person to determine how likely it is that the person will pay back a loan. Based on the name of the process, it’s no surprise that credit card companies do credit risk modeling all the time. But credit risk modeling doesn’t necessarily have anything to do with credit cards, even though “credit” is in the name. Credit risk modeling applies to <em>any</em> loans, not just loans associated with credit cards.</p><p>Countless organizations use credit risk modeling, including insurance companies, banks, investment firms, and government treasuries. Sometimes, individual people make a <em>living</em> using credit risk modeling to strategically loan away their own money. Credit risk modeling is ultra important anywhere people are borrowing money.</p><p>Even more importantly, understanding the process of credit risk modeling will make it easier to understand <em>any</em> type of modeling that involves probabilities.</p><p>But for now, let’s come back to Ted so that we can get a solid grasp of the fundamentals of loans.</p><h3><strong>What are interest rates?</strong></h3><p>If you’re lending money to Ted just to be nice, you’ll probably just ask him to give you back the same amount of money you gave him. But you could also take this opportunity to make a bit of extra money.</p><p>You could tell Ted that he has to give you back the same amount of money you gave him, <em>plus</em> ten percent.</p><p>This means if you give Ted $10, he’ll have to give you $11 back. The extra money he gives you back is called the <strong>interest</strong>. In this case, Ted is paying you $1 in interest, and the <strong>interest rate</strong> is ten percent. If you raised the interest rate to 20%, Ted would have to pay you $2 in interest, which means he’d have to pay you $12 in total.</p><p>If Ted really needs the money, then he’ll probably be okay with paying some interest. <em>Paying interest on a loan is basically buying the ability to spend money that you’ll have in the future.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wMdTQazCbryOSx_BOnWYYQ.jpeg" /><figcaption>Image created by author</figcaption></figure><p>Spending future money in the present is a convenient thing to do, and like all convenient things in the business world, it comes with a price.</p><p>With interest rates, you can see how it could be pretty useful to have 500 Teds asking you for money at the same time. If you charged all of them 20% interest, then loaning $10 to each of them could make you a thousand dollars.</p><p>But we get to that figure of $1000 by making a pretty big–and invalid–assumption: that everyone will pay you back.</p><p>If somebody doesn’t pay you back, then you permanently lose the $10 you gave them. You also miss out on the $2 they would have paid you in interest. This means that for each person who doesn’t pay you back, you miss out on $12.</p><p>If just 84 out of the 500 people don’t pay you back, then you’ll actually <em>lose money</em> from this whole lending process. It’s amazing that it takes less than 17% of the people to screw this whole system up.</p><p>To make a living off of this, you need a better system.</p><h3><strong>What is default risk?</strong></h3><p>Fortunately, credit risk modeling lets you estimate the probability that each person will fail to pay you back. <strong>Defaulting</strong> on a loan means failing to pay it back, so each person’s probability that they’ll fail to pay back their loan is called their <strong>default risk</strong>.</p><p>Determining somebody’s default risk is important. Once you know somebody’s default risk, there’s a way to calibrate their interest rate to mitigate the risk of lending money to them.</p><p>But before we can understand how to use a default risk to calibrate somebody’s interest rate, it’s important to understand a fundamental statistics concept: the Law of Large Numbers.</p><h3><strong>The Law of Large Numbers</strong></h3><p>Let’s start with a simple situation: flipping a coin.</p><p>We’ve all heard that there’s a 50% chance of getting a heads and a 50% chance of getting a tails. This is called a <strong>theoretical probability</strong>.</p><p>If you flip the coin once, then you’re going to get either heads or tails. If you flip the coin 10 times, you could get 5 heads and 5 tails (50% heads). But it’s also totally possible that you get 6 heads and 4 tails (60% heads), or even 9 heads and 1 tails (90% heads).</p><p>These observed results are called the <strong>experimental probability</strong>. The experimental probability doesn’t necessarily equal the theoretical probability.</p><p>So what does a 50% chance of getting heads actually mean?</p><p>It means that if you flip a coin a lots and lots and lots of times, the overall result will be that about 50% of the flips come of up heads.</p><p>Let’s look at this step by step.</p><p>Let’s flip a coin 5 times. The number of heads we got is the number of red dots, and the number of tails we got is the number of blue dots.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-NiENY_BNSGrm4ENcueWsA.jpeg" /><figcaption>Image created by author</figcaption></figure><p>This looks pretty imbalanced so far, with 80% of our flips coming up heads. That’s an 80% experimental probability, which is pretty far from our 50% theoretical probability. But let’s keep flipping. Here’s what we’ve gotten after flipping the coin 15 times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x37ce7FFXEKUzncKRelXlw.jpeg" /><figcaption>Image created by author</figcaption></figure><p>Now we’ve had a total of 60% of our results come up as heads, which is a lot closer to 50%. If we keep flipping until we’ve flipped the coin 50 times, here’s what we get.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jkqFRKI_wP1LbU8_lCamFg.jpeg" /><figcaption>Image created by author</figcaption></figure><p>That’s 46% of our total flips now coming up heads, which is even closer to 50%. If we keep flipping the coin over and over again, then we’re going to see this percentage of heads get closer and closer to the theoretical probability of 50%. It might not get closer to 50% with every single flip, but it will generally tend to get closer to 50%.</p><p>If we keep flipping, until we get to 100 flips, the blue line represents the cumulative percentage of heads over time. The red line is 50%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*mapHld6TViWDl0apKy1FtA.png" /><figcaption>Image created by author</figcaption></figure><p>The blue line starts off really high because our first four flips were heads, which means that 100% of our cumulative tosses came out heads for a little bit. But as we get some tails in the mix, the blue line tends to get closer to the red line.</p><p>As you can see, as we flip the coin more and more times, the blue line, which represents our experimental probability, approaches the red line, which is our theoretical probability.</p><p>This is how we define a theoretical probability, like a 50% chance of getting a heads or a 16.7% chance of getting a 1 when rolling a 6-sided die.</p><p><strong>The Law of Large Numbers is that the experimental probability tends to approach the theoretical probability as we do a large number of trials. </strong>The more trials we do, the closer the experimental probability tends to get to the theoretical probability.</p><p>To illustrate this concept, here’s a coin flip simulation graph with fifty thousand flips:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*KzIilPa07Fe9i8Ey-bJNlA.png" /><figcaption>Image created by author</figcaption></figure><p>By the end, the experimental probability gets so close to the theoretical probability that the blue and red line are almost on top of each other.</p><p>This means that when we’re dealing with a ton of trials, we can assume that the experimental probability will basically equal the theoretical probability. In this case, if we flip a coin fifty thousand times, the experimental probability will be super close to 50%.</p><p>It’s important to highlight that each time you<em> </em>flip the coin, <em>the probability of getting a heads is always 50%, no matter what came before it</em>. Even if you’ve gotten 10 heads in a row, the probability of getting another heads on the next flip is <em>still</em> 50%.</p><p>This helps explain why the blue line in the graph above doesn’t always go straight to the red line. Even near flip 30,000, you can see that the blue line still wiggles just a little. But over time, the blue line generally tends to approach the red line.</p><p>The important thing to take away from the Law of Large Numbers is that a large number of trials gets the experimental probability really close to the theoretical probability.</p><p>If you think this idea is making a bit of sense, you’re ready to understand the process of calibrating interest rates based on default risk.</p><h3><strong>Using default risk to calibrate interest rates</strong></h3><p>Let’s say you’re thinking about making loans of $10 to a large number of people, such as 10,000 people. Using credit risk modeling, you discover that on average, each person’s theoretical default risk is 15%. The Law of Large Numbers tells you that the experimental probability approaches the theoretical probability for a large number of trials. Given that you have a large number of people here (10,000 people), it’s safe to assume that about 15% of the people you loan to really will default, and 85% will pay you back.</p><p>If you know that about 15% of people probably won’t pay you back, then you know that you’re going to permanently lose 15% of the $10 loans you make. Because 15% of 10,000 people is 1500 people, and you’re loaning each person $10, you know you’ll probably lose $15,000.</p><p>But you don’t need to just be okay with losing $15,000. There are a few things you can do here.</p><h4><strong>1. Increase everyone’s interest rate to make up for that $15,000.</strong></h4><p>This way, the 85% of people who actually do pay back their loan will end up paying a total of $15,000 extra. Because 85% of 10,000 people is 8,500 people, that’s about an extra $1.76 of interest per person. In this case, that’s equal to raising the interest rate by 17.6 percentage points.</p><p>Here’s the issue: if the interest rate is already set at 20% so that you can make some profit, then raising the interest rate another 17.6 percentage points makes the total interest rate 37.6%. That’s a big increase, and increasing the interest rate like that might deter some potential borrowers.</p><h4><strong>2. Only accept the safest borrowers.</strong></h4><p>Credit risk modeling lets you see each individual’s default risk separately. Some people might have a 40% default risk, while others might have just a 1% default risk. If you only give loans to people whose default risk is below a certain threshold, say 2%, then the percentage of people who will pay you back will be much higher. This means that fewer people will default, and you’ll lose less money.</p><p>As a result, you can charge a lower interest rate, because you won’t have to make up for as much lost money due to defaulting. The problem with this strategy is that you might have to turn away a big portion of people who want a loan, which would decrease your profits.</p><h4><strong>3. Charge people interest rates that are proportional to their default risk.</strong></h4><p>This is the most important strategy in the real world. As we go through all the borrowers, we’ll call each individual borrower “borrower x”. <strong>For each borrower x, you charge borrower x an interest rate such that you wouldn’t lose any money if every borrower had borrower x’s default risk.</strong></p><p>That’s a dense idea, so let’s unpack it.</p><p>When analyzing the first strategy, we discovered that we should add 17.6 percentage points to each borrower’s interest rate if each borrower has a 15% default risk. (This is the same as 15% being the average default risk.) This way, the extra interest paid by the 85% who don’t default will make up for the 15% of people who default.</p><p>If everybody you’re loaning money to has the same default risk, then there’s always a certain amount that should be added to each borrower’s interest rate to offset the people who default.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hmig2f9ZPNwjT6k2Qk1itw.jpeg" /><figcaption>Image created by author</figcaption></figure><p>In fact, the amount that should be added to each borrower’s interest rate can be calculated with the following formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ogawNqsb4EYEtw3c_PlZdg.jpeg" /><figcaption>Image created by author</figcaption></figure><p>There’s no need to delve into this formula if you don’t want to, but if you want to learn how it works, this paragraph will show you how. The risk term on top represents the share of people who are going to default. Defaulting is a binary outcome; this means that every person is either going to default or not default. There’s no in-between. The number 1 represents 100% of the people, so the term 1 — risk (one minus risk) is the share of people who don’t default. You divide the defaulters by the non-defaulters to determine what percentage of each defaulter’s loan the non-defaulters need to pay off.</p><p>Even if each borrower’s default risk is different, you can use this formula to calculate exactly how much you should add to each borrower’s interest rate to counterbalance their risk. If you add this amount to every borrower’s interest rate, and you have a large number of borrowers, the Law of Large Numbers shows us that you almost certainly won’t lose money in the long run.</p><p>Just to recap, here’s a breakdown of the money somebody might pay you back after you loan money to them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mgttMx6JlZkpLahOLbUBog.jpeg" /><figcaption>Image created by author</figcaption></figure><p>In the real world, these strategies are combined. In accordance with strategy 1, lenders often increase everyone’s interest rate by small fixed rate, regardless of default risk, to make up for uncertainty in their models. In accordance with strategy 2, lenders often have a risk threshold, so they won’t accept everyone. And finally, in accordance with strategy 3, lenders use each borrower’s default risk to add a certain number of percentage points to each individual’s interest rate.</p><p>If you’re getting the hang of these three strategies, then you have a solid grasp of the fundamentals of loaning money.</p><p>At this point, we’ve explored why finding somebody’s default risk is so useful. Now, we’re ready to dive into the machine learning models used in credit risk modeling to calculate these default risks.</p><h3><strong>What is machine learning?</strong></h3><p>People toss around the term “machine learning” a lot nowadays, and sometimes it’s hard to know what it means.</p><p>Simply put, a machine learning model is a mathematical predictive model that gets better as it sees more data.</p><p>To do machine learning, you need two things: a model, and data.</p><p>There are tons of different types of machine learning models. A machine learning model is the name for the set of steps that are used to make predictions based off the data. Below, we’ll explore four fundamental machine learning models that are important in credit risk modeling.</p><p>You also need historical data, and lots of it. This data needs to contain both the thing you’re trying to predict (called the <strong>target</strong>) and other characteristics that are related to the thing you’re trying to predict (called the <strong>features</strong>). In the case of credit risk modeling, you need historical data for lots of individual people. The target data here is whether or not each person defaulted. The feature data here consists of statistics about each person, such as their income, age, and employment status.</p><p>Once you have your data, you need to <strong>train</strong> the model by feeding it historical data. Machine learning models are built to connect the feature data to the target data. After training, the model can look at cases where the feature data is known and the target data isn’t, and the model can make predictions.</p><p>This situation is common in real life. When you’re trying to decide whether or not to give someone a loan, you usually have all their feature data, such as income and age. You just don’t know the target data: the probability that they’ll default on their loan.</p><p>Calculating this probability is where machine learning models shine.</p><p>Let’s start with a simple, but surprisingly powerful model.</p><h3><strong>The K-Nearest Neighbors Model</strong></h3><p>Let’s say we have a bunch of historical lending data with two features: each person’s income and their age. We also know whether each person defaulted on their loan. If they defaulted, then their dot is colored red on the graph. If they paid back their loan, their dot is colored blue on the graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rFPyJpGfGq2Mn6HN0wmuCg.jpeg" /><figcaption>Image created by author</figcaption></figure><p>Now let’s say Ted is asking again for a loan. We want to know how likely it is that he’ll pay back his loan, and we now have a much more powerful tool than our feelings alone. If we know Ted’s income and age, then we can plot him in green on the same graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nn6mFvHmufup5aYjWDRcfQ.jpeg" /><figcaption>Image created by author</figcaption></figure><p>We then select a number of “nearest neighbors” that we’ll look at. This number is called K. If K = 5, then we’ll select the 5 dots that are closest to Ted’s dot and figure out what percentage of them defaulted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y_bWWqp8OlzsgoxwwQwWiw.jpeg" /><figcaption>Image created by author</figcaption></figure><p>In this case, 80% of people nearest to Ted paid back their loans (4 people), and 20% of them defaulted (1 person). This means that we tentatively estimate Ted’s default risk as 20%.</p><p>Of course, you’re probably thinking that setting K equal to 5 seems pretty arbitrary. There is a way to optimize our K value, and it lies in <strong>testing</strong> the model.</p><p>To test the model, we split our historical data into two parts: the <strong>training data </strong>and the <strong>testing data</strong>. We then set our K value to an arbitrary value, say 10, and we feed the model our training data to generate a graph like the one above. Then, for each person in the testing data, we make our trained model guess whether or not the person defaulted on their loan. The model doesn’t get to see the target column of the testing data; it only sees the features. During testing, the model has to make predictions based on the features alone, just like it would in real life.</p><p>Each time the model makes a prediction, we record whether the model predicted correctly. We then calculate the model’s <strong>accuracy score </strong>by finding the percentage of cases it predicted correctly.</p><p>To optimize our K value, we test the model over and over again with different K values until we find one that works best. Sometimes the ideal K value is 1, while other times the ideal K value is huge. (Of course, for credit risk modeling, a K value of 1 would be silly, because this would either give us a 0% or 100% default risk.)</p><p>In the 2D graph above, we only had 2 feature variables: income and age. But the k-nearest neighbors algorithm works great with more than 2 feature variables as well. If we had 3 feature variables, we could plot the points in a 3D box instead of a 2D square.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kqsmOvegT3aQorRdBGGk5g.jpeg" /><figcaption>Image created by author</figcaption></figure><p>With 3 variables, the same set of steps applies. Training the model gives us a 3D graph with a bunch of blue and red dots on it. When we have a person whose default risk we’re trying to predict, we plot them on the graph. To calculate their default risk, we find this person’s K nearest neighbors, and we calculate the percentage of these neighbors who defaulted. We say that this percentage is the new person’s default risk.</p><p>Once we get to 4 variables, the graph becomes hard to visualize. But the computer can deal with a nearly infinite number of variables. In fact, it’s not too difficult to algebraically calculate nearest neighbors by hand with 4 or more variables using the Pythagorean Theorem. (Google it if you’re interested!)</p><p>k-nearest neighbors might seem like quite a simplistic model, but with thousands of data points and many features, it can be quite powerful.</p><p>In some cases, however, other models are more effective. Let’s discuss another classic machine learning model: logistic regression.</p><h3><strong>The Logistic Regression Model</strong></h3><p>The simplest form of logistic regression involves a dataset with a target column and a single feature column. For instance, a historical dataset that contains borrowers’ incomes and whether or not they defaulted on their loan would be very simple to use for logistic regression.</p><p>For logistic regression, the target data must be <strong>binary</strong>, which means that it can only have two outcomes. In credit risk modeling, the target data is indeed binary: a person can either default or not default on a loan.</p><p>In order to make a graph, we can represent this binary outcome as the probability that <em>a person in the past </em>paid back their loan. If somebody defaulted on a loan in the past, then we can say that there was a 0 probability that they paid back their loan. If somebody actually did pay back their loan, then we can say that the probability that they paid back their loan was 1. It might feel weird to assign probabilities to past events, but probabilities of 0% or 100% are just another way of saying that we know something either definitely didn’t happen or definitely did happen.</p><p>Here’s what a graph of income vs historical default probability might look like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5lzoWnfYRh611isUoeC3Kg.jpeg" /><figcaption>Image created by author</figcaption></figure><p>Because we’re dealing with definite past data, all the dots are either on the top, or bottom. A blue dot on the top represents a person who did pay back their loan, while a red dot on the bottom represents a person who defaulted.</p><p>This is where things get cool. Using some interesting math that we won’t dive into here, the logistic regression algorithm calculates a line that looks something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SqukfcIC-_Pogj24EwN6Qw.jpeg" /><figcaption>Image created by author</figcaption></figure><p>This line tells us the probability that a new person with a given income level will pay back their loan.</p><p>Predicting a new person’s default risk with logistic regression is pretty simple. You just find the point on the income axis that corresponds to the new person’s income, and the logistic regression line’s y-value that corresponds to this point gives you the probability that they’ll pay back the loan. Subtracting this value from 1 gives you their default risk.</p><p>Let’s say that Ted’s income is right here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_G47TgoFT485ftE4sFgpuw.jpeg" /><figcaption>Image created by author</figcaption></figure><p>To find the probability that Ted will pay back his loan, we go up and meet the logistic regression trend line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vGuPzXEcHipbIDLQDbjKjQ.jpeg" /><figcaption>Image created by author</figcaption></figure><p>Based on this model, it looks like he has about a 85% chance of paying back his loan. This means that his default risk is 15%.</p><p>It’s worth noting that this version of logistic regression put Ted’s default risk at 15%, while our k-nearest neighbors model put Ted’s default risk at 20%. Getting different predictions from different models is totally normal in the world of machine learning. To decide which model to use, we would have to do a bunch of train/test cycles with historical data to see which model consistently performs best in this scenario.</p><p>A 2D version of the logistic regression model is pretty simplistic, just like 2D k-nearest neighbors. But just like K-Nearest-Neighbors, a computer (or a person with a pencil!) can extend logistic regression to an infinite number of dimensions without any problem.</p><p>K-nearest neighbors and logistic regression are both great models for making approximations, especially if the data follows a linear pattern. But sometimes, it’s nice to have a model that picks up on subtle, non-linear patterns in the data. The decision tree model is a simple model that’s excellent at finding such patterns.</p><h3><strong>Decision Trees</strong></h3><p>To understand decision trees, let’s go back to the graph we used to illustrate how k-nearest neighbors works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nn6mFvHmufup5aYjWDRcfQ.jpeg" /><figcaption>Image created by author</figcaption></figure><p>As we’ve seen, one way to make sense of this graph would be to use the k-nearest neighbors algorithm. However, we could also make some simple rules for how we classify points based on their location. For instance, we could draw a line that splits the graph in half, like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mVN6JHXyz8KkKVHgeXuO6A.jpeg" /><figcaption>Image created by author</figcaption></figure><p>Drawing this line splits the data into two halves. One way to represent this is using a tree diagram, as we can see below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g_v3eEAoXDnPPsOL5VAJbw.jpeg" /><figcaption>Image created by author</figcaption></figure><p>Then, we can split each of these splits again, so that they end up looking like this on the graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FVuk01xzokAIzy1z-_v9UA.jpeg" /><figcaption>Image created by author</figcaption></figure><p>And here’s what this second split looks like in tree form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rdF6fkrTjKsoAlEwpEfmfA.jpeg" /><figcaption>Image created by author</figcaption></figure><p>We can keep making smaller and smaller splits like these until we’re left with just one outcome–default or no default–on the bottom of each branch. Another way to look at this is that we keep making the boxes on the graph smaller and smaller until each box only contains one color of dots.</p><p>For each split, the computer finds the optimal value to split on by using an interesting mathematical process that we won’t delve into here.</p><p>To test a new case using the decision tree algorithm, you simply start at the beginning and follow the branches down.</p><p>Let’s say Ted makes $50,000 a year and is 30 years old. We start at the top of the tree.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gbGuNUEUWIs1Tw9_IWF7Fg.jpeg" /><figcaption>Image created by author</figcaption></figure><p>Then, we move down and right, because Ted makes more than $20,000 per year.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fJpdVXh17AYxJL2ayLOu2w.jpeg" /><figcaption>Image created by author</figcaption></figure><p>From here, we move to the right because Ted is older than 28.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8yyOhiX5nBp6BwljrSU7jQ.jpeg" /><figcaption>Image created by author</figcaption></figure><p>This process continues, until we get all the way to the bottom of the tree, where there is only one possible outcome on a leaf. In a real decision tree, there would be many more branches than the two layers shown here.</p><p>The issue with this algorithm is that it returns a binary result: either Ted will default, or Ted won’t default. Given that any model is inherently imperfect, this binary result isn’t that useful for credit modeling. We can’t use a binary result to set Ted’s interest rate.</p><p>The solution is to use <strong>random forests</strong>. A random forest is simply a collection of decision trees, all structured slightly differently. To make a prediction, each tree in the forest “votes” for whether or not it thinks Ted will default on the loan. Ted’s credit risk is the percentage of the trees that say he will default.</p><p>Like the other algorithms we discussed, the decision tree algorithm can be extended to as many dimensions as we want. One advantage of the decision tree algorithm is that it’s very good at capturing non-linear data. However, decision trees sometimes look too far into noisy data, thinking that they see patterns where statistically significant patterns don’t exist.</p><p>There’s one more type of machine learning algorithm that we’ll briefly touch on here.</p><h3><strong>Neural Networks</strong></h3><p>Neural networks are considerably more complex than the three algorithms we’ve already discussed. They’re a fascinating algorithm that’s modeled off of the human brain, and they’re at the core of most systems that can be considered “AI”.</p><p>Neural networks are usually overkill for simple credit risk modeling problems, but in certain cases, they can be tremendously useful.</p><p>If you want to learn more about neural networks, you can read a detailed explanation of them <a href="https://medium.com/@a.jeremymah/behind-the-eyeballs-of-ai-how-neural-networks-work-207c31e0b1ff">here</a>.</p><h3><strong>Summary of the risk modeling process</strong></h3><p>These four classes of algorithms (k-nearest neighbors, logistic regression, decision tress, and neural networks) are just the beginning of the machine learning used in credit risk modeling. But understanding the basics of these algorithms gives you a bit of insight into the whole credit risk modeling process, and this understanding gives you the perfect springboard to learn more about machine learning.</p><p>First, we developed a bit of domain knowledge about how loans and interest rates are used in the real world. Then, we looked at splitting the data into train and test sets to analyze various models. Finally, we analyzed several models that we could test out, and we know that we’d use the most effective models’ outputs to make our predictions.</p><p>A similar process is applicable to almost any problem that involves machine learning.</p><p>First, it’s important to develop some domain knowledge about the problem you’re dealing with so that you know how to ask the right questions. Then, you have to clean and prepare your data (a topic we didn’t delve into here). Next, you train and test a variety of models, honing the models to maximize prediction accuracy. Finally, you present your results to others and apply them to real world problems, repeating this process continuously.</p><p>If you have a better understanding of this general process, then you’re well on your way to better understanding credit risk modeling and machine learning as a whole!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c8a2657b4c4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/credit-risk-modeling-with-machine-learning-8c8a2657b4c4">Credit Risk Modeling with Machine Learning</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Behind the Eyeballs of AI: How Neural Networks Work]]></title>
            <link>https://medium.com/swlh/behind-the-eyeballs-of-ai-how-neural-networks-work-207c31e0b1ff?source=rss-17b3955c7601------2</link>
            <guid isPermaLink="false">https://medium.com/p/207c31e0b1ff</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[A. Jeremy Mahoney]]></dc:creator>
            <pubDate>Wed, 02 Sep 2020 07:42:04 GMT</pubDate>
            <atom:updated>2020-09-30T01:54:24.448Z</atom:updated>
            <content:encoded><![CDATA[<p>Sometimes it seems like practically every time I ask how a cool piece of new technology works, I get a two letter response: AI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*glZLpGmeqtROeg9rWpa1nw.jpeg" /></figure><p>But then, if I try to ask how AI (artificial intelligence) works, I’m told that it’s complicated and takes a lot of linear algebra and multivariable calculus to understand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*riGszVIbcNtqXyQunVVQcw.jpeg" /></figure><p>For any curious person, this is annoying! It’s fun to look into the inner workings of complex systems that affect our lives. Being told you need to read two math textbooks to understand the basics of something is discouraging.</p><p>Fortunately, in the case of neural networks, it’s not true!</p><p>You definitely don’t need a deep understanding of linear algebra and multivariable calculus to understand the basics of neural networks. Now don’t get me wrong: complex math <em>is </em>essential for understanding neural networks at a deep level and creating your own neural networks from scratch.</p><p>But that’s not why most of us ask how AI works. If you just want to get a better feel for what’s going on behind the scenes of things people call “AI”, you don’t even need to know what linear algebra or multivariable calculus <em>is</em>.</p><p>Reading and understanding this article will make you more knowledgeable about neural networks than 99% of people. If you want to <em>create</em> <em>your own</em> neural networks and become more knowledgeable on the topic than 99.99% of people, then go learn some math.</p><p>After reading this article, you’ll be able to explain to anyone the structure of a neural network and how neural networks “learn” as they’re exposed to new data!</p><p>A neural network is the name for the computer program that’s the “brain” of an AI system. But before talking about neural networks, we’ll first look at what AI is, in a broad sense.</p><h3><strong>What even is AI?</strong></h3><p>It’s weird that it’s so hard to define a technology that’s supposed to be all around us.</p><p>Self-driving cars use AI to not run into stuff. Speech-to-text systems use AI to transcribe words. My phone’s handwriting keyboard uses AI to make sense of my sloppy pointer finger calligraphy.</p><p>While these might all seem like super different tasks, all these systems have a key thing in common.</p><p>Broadly speaking, an AI program is a computer program that <strong>finds patterns</strong> and <strong>does cool things with these patterns</strong>.</p><p>This doesn’t sound that impressive until you realize that humans pretty much just do the same thing.</p><p>Think about any complex, human-y thing that humans like to do. Let’s take having a conversation with someone. At a fundamental level, conversations work because each person finds patterns coming from the other person, then does cool things with these patterns. Each person’s ears and brain work together to find patterns of vibrations in the air. The brain then converts these vibration patterns into words with meanings, which is a pretty cool thing to do. Each person’s eyes and brain work together to find patterns of movements in the other person’s face. The brain then converts these tiny movements into emotional signals, which is also a pretty cool thing to do.</p><p>No matter how complicated a task is, any task can be reduced to <strong>finding patterns<em> </em></strong>and <strong>using these patterns to do cool things</strong>. Of course, most tasks involve insanely large numbers of patterns, but they’re ultimately still based on patterns.</p><p>A self-driving car AI system learns to <strong>recognize objects on the road</strong>, like signs and road markings. The system then <strong>uses these patterns</strong> to construct an internal model of the road and steer safely.</p><p>Speech-to-text systems learn to <strong>recognize common speech sounds</strong>, and they combine <strong>patterns of these sounds</strong> into words. In other words, these systems are built to recognize <strong>patterns of patterns</strong>.</p><p>At its core, the challenge of making really good AI is trying to make systems that can accurately recognize patterns of patterns (of patterns of patterns of patterns…) in the real world.</p><p>If that idea makes sense to you, then you understand the essence of AI.</p><h3><strong>Types of AI</strong></h3><p>Now is also a good time to clear up the difference between two commonly discussed forms of AI: artificial narrow intelligence and artificial general intelligence.</p><p><strong>Artificial narrow intelligence (ANI)</strong> is a pattern recognition based AI system that is designed to do a specific task. This is the kind of AI that’s all around us today. Examples of ANI include fingerprint scanners, automatic credit card fraud detection systems, and every other application of AI discussed above. These systems are built to recognize patterns in specific situations, and the state-of-the-art ones are damn good at it.</p><p><strong>Artificial general intelligence (AGI) </strong>is where things start getting really crazy. AGI is designed to match or exceed the intelligence of a human. This means that an AGI system should be able to learn to recognize any pattern that a human could learn to recognize. <strong>As of 2020, AGI does not exist yet</strong>. Most experts agree that we’ll probably have AGI sometime between decades and centuries from now, but at the moment, we’re still working on it.</p><p>Here, we’ll be talking specifically about <strong>ANI</strong>. The general principles we’ll explore here would totally be relevant to an AGI system as well, but they’ve only ever been implemented in ANI.</p><p>Now that we understand the basics of AI, we can start getting into neural networks.</p><h3><strong>How neural networks work — <em>broad </em>overview</strong></h3><p>A neural network is the name for the computer program that’s the “brain” of an AI system. Neural networks are designed so that they get smarter as they see more and more data.</p><p>To understand this process of a neural network “getting smarter,” we’ll look at a classic application of neural networks: handwritten digit recognition.</p><p>The University of California, Irvine, assembled a famous dataset of handwritten numbers from 0–9. The dataset contains images of numbers volunteers wrote by hand, and these images are paired with the actual number each one is supposed to be.</p><p>To make it easier for normal people’s computers to work with, each image was scaled down to 8x8 pixel resolution. Here’s what the actual scaled-down images look like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/617/1*7fBbjXVhGbGbaQaqlecfkA.png" /></figure><p>Not too hard for a person to figure out. Just stand about 20 feet back so the images get all blurry. It’s pretty clear that the first one is a 0, the second one is a 3, and the third one is a 7 written by some nerd who crosses their 7s.</p><p>But if I asked you to write a computer program to identity which is which, then that’d be a good challenge. That’s a perfect thing to use a neural network for.</p><p>Just like a human baby, a neural network can’t just recognize these images right away. First, we have to <strong>train</strong> the neural network.</p><p>As we talk about training, just pretend the neural network is a person. This makes the verbs in the rest of this section make a lot more sense. In the next section, we’ll get into how a neural network does all this neat stuff without actually being a person.</p><p>To <strong>train</strong> the neural network, we start by showing it an image and telling it what that image is supposed to be. So basically, “neural network, this thing is a 0.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/1*KXM8X5JB1EmQonXvkdDwLw.png" /></figure><p>Then we do it again. “Neural network, this thing is a 3.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/406/1*VGGuSFYHF6ASqbttVtBO1Q.png" /></figure><p>And we keep doing that over and over again until we’ve shown it lots of images of every number. But don’t show it <em>every </em>image we have. The images and solutions that we show it are called the <strong>training data</strong>.</p><p>After it’s seen a bunch of images, we say that the neural network has been <strong>trained</strong>. The next step is to <strong>test</strong> the trained neural network.</p><p>To test the neural network, we show it one of the images that <em>it hasn’t seen yet </em>and ask the neural network what number it thinks it is. “Neural network, what do you think this is?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/444/1*GPyEcKBq0fG0QcYHAdZcdg.png" /></figure><p>If it identifies the image correctly, as a 5 in this case, we keep track of that. If it identifies the image incorrectly, we keep track of that too. We keep showing the neural network images that it hasn’t seen yet and making it identify them until we’ve run out of images. The images that we show the neural network during the <strong>test</strong> phase are called the <strong>testing data</strong>.</p><p>The neural network’s <strong>accuracy score</strong> is the percentage of images it correctly identifies. We want to get this neural network’s <strong>accuracy score</strong> as close to 100% as possible.</p><p>There are several parts of the neural network that we can adjust to improve the accuracy score, which we’ll explore later on. After we tweak the neural network a bit, we <strong>train </strong>it once again, then <strong>test</strong> it again, and calculate a new <strong>accuracy score</strong>. We keep repeating this process, changing the neural network a little each time, until we’re satisfied with how it performs!</p><p>For challenges like this one that involve classifying things, this is how basically all neural networks get smarter over time.</p><p>But there’s still one thing that you’re probably confused about, for good reason.</p><p>The neural network isn’t a person. How is it looking at stuff and identifying patterns like a person would?</p><p>Before we get into how the neural network identifies patterns, let’s first figure out how it “looks at” stuff at all.</p><h3><strong>How neural networks look at stuff</strong></h3><p>The pixelated numbers in this dataset are really just a bunch of light and dark colored boxes. <em>We</em> immediately see them as numbers, but that’s only because we have a really well trained neural network living inside our heads. For a neural network that doesn’t know anything yet, we need an easier way of representing these images.</p><p>Let’s say we have some random image like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*9yp5oMC4dtI7vyTvXqn4LA.png" /></figure><p>If we wanted to, we could represent the shades of the boxes as numbers instead of colors. 0 could stand for white, 1 could stand for black, and numbers in between 0 and 1 could stand for shades between white and black. Here’s what that would look like:</p><pre>[[0.5, 0.75, 0.5],<br>[0, 1, 0],<br>[0.5, 0.75, 0.5]]</pre><p>If we know that this is supposed to be a 3x3 square image, we wouldn’t even need to put the numbers on separate lines. We could just make one list of numbers, like this:</p><pre>[0.5, 0.75, 0.5, 0, 1, 0, 0.5, 0,75, 0.5]</pre><p>This list is a lot easier for a neural network to handle! We could follow this same set of steps for any image, including the number images we already have. This means that the image of the number 5 above would look like this to a neural network:</p><pre>[0.0, 0.125, 1.0, 1.0, 1.0, 0.875, 0.3125, 0.0, 0.0, 0.5625, 1.0, 0.6875, 0.375, 0.5, 0.1875, 0.0, 0.0, 0.5625, 1.0, 0.125, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1875, 1.0, 0.8125, 0.0625, 0.0, 0.0, 0.0, 0.0, 0.0, 0.5, 1.0, 0.75, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.75, 1.0, 0.25, 0.0, 0.0, 0.0, 0.0, 0.0625, 0.6875, 1.0, 0.375, 0.0, 0.0, 0.0, 0.25, 1.0, 1.0, 0.5, 0.0, 0.0, 0.0]</pre><p>Neural networks see images as a <strong>list of numbers, </strong>just like this.</p><p>Now that we understand how neural networks see images, we’ll look at the structure of a very simple neural network.</p><h3><strong>Simple neural network structure</strong></h3><p>A neural network is simply made up of <strong>neurons connected to other neurons</strong>. If every circle is a neuron, and every line is a connection, here’s what that looks like for a simple neural network:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NF0UbDsUvrf0HWGCqz_95Q.jpeg" /></figure><p>This is called a <strong>feedforward neural network, </strong>and it’s the type of neural network we’ll explore here. There are tons of different types of neural networks, but feedforward neural networks are the simplest to understand.</p><p>Each of those neurons on the far left is called an <strong>input neuron</strong>. That neuron on the right is called the <strong>output neuron.</strong></p><p>The word “neuron” sounds really fancy and sciencey, but a neuron is really just a <strong>box that holds a number</strong>.</p><p>This neural network might look pretty useless, but it’s actually very simple to input any one of our images into it.</p><p>First, increase the number of input neurons to the number of pixels in the image. Because it’s an 8x8 image, the number of pixels in the image equals 8 X 8, which equals 64. So we need 64 input neurons.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E-BrCt9B4GGDEb8RqpqxNg.jpeg" /></figure><p>Remember that “list representation” of an image that we made above? Because each number in the list representation we made above represents a pixel, the list is also 64 numbers long.</p><p>From here, we can put each number in the list inside its own input neuron.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9S2bjG8mw8Q9geWaSodxGw.jpeg" /></figure><p>Each input neuron represents a single pixel of the image. And just like that, we’ve fed the neural network an entire image!</p><p>Of course, the neural networks needs to actually do something with the image to be useful.</p><h3><strong>Passing information between neurons</strong></h3><p>This is where the action starts. Don’t worry about the purpose of passing information between neurons just yet. We’ll talk about that in just a moment! For now, focus on understanding <em>how</em> information is passed between neurons.</p><p>To figure out what number should be in the output neuron, we <strong>add up all the input neurons connected to it</strong>.</p><p>As an example, let’s say a neural network with 5 input neurons looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tsBNzUmvWPRQPg8X_cPSEQ.jpeg" /></figure><p>To figure out what value should be in the output neuron, we add up all the input neurons that are connected to the output neuron. The result would look like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rahf8cxuY9b-7FotcCD7gA.jpeg" /></figure><p>But addition alone isn’t quite enough for a working neural network. We also need to give each connection a certain <strong>weight</strong>.</p><p>When a connection between an input neuron and the output neuron is <strong>weighted, </strong>the <strong>input neuron’s number</strong> is multiplied by the connection’s <strong>weight</strong> before being added to the next layer. Let’s see that in action. Below, the numbers in green are the weights of each neuron’s connection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p5Frwx-D5hXDWwMdsYTb4w.jpeg" /></figure><p>Below, we can see the results of the multiplication in green as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dokf-o4LlWHyzbzw6em6gw.jpeg" /></figure><p>The sum of the weighted input neurons is stored in the output layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aWX3v2iNtlbypPUbyfak1g.jpeg" /></figure><p>So now we have two different parameters we in this system that we can mess with: the value of the input neurons, and the weights of each connection.</p><p>There’s just one more thing we need to do to this output neuron. All the input neurons have a value between 0 and 1, but the output neuron’s value right now is way above 1. To keep things consistent, we’ll put the output neuron’s number into a mathematical function called the <strong>sigmoid function </strong>that spits out a number between 0 and 1, no matter what the input is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*iOvHpfiCCtiq9PzaK3ojKA.png" /></figure><p>This way, the output neuron’s range always matches the input neurons’ range. This will become important later. Putting the output neuron’s value into the <strong>sigmoid function </strong>and replacing the output neuron’s original number with the result is called <strong>transforming</strong> the output neuron. In the context of neural networks, a function like this that transforms values is called an <strong>activation function</strong>.</p><p>This whole weighting process still probably seems pretty arbitrary. But weights are very useful, and you’re ready to understand why!</p><h3><strong>Super simple classification neural network</strong></h3><p>It’s actually possible to make a simple classification system with just the concepts we’ve already covered! Let’s say you want to make a system that can distinguish between the number 1 and the number 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P5yZPkEOo7kdiZw8fuKGTQ.png" /></figure><p>To make seeing the boundaries of each image a little easier, I’ve added in border lines.</p><p>Looking at the 1 and the 0 next to each other makes a very important detail clear: 1s usually have very dark pixels in the middle of the image, while 0s usually have very light pixels in the middle of the image.</p><p>If we can get our neural network to recognize this, then we have a pretty good shot at classifying 1s and 0s correctly.</p><p>Fortunately, we have the perfect tool for this: <strong>weighting</strong>.</p><p>Each input neuron represents a pixel. The darker a pixel is, the closer its neuron’s value is to 1. The lighter a pixel is, the closer its neuron’s value is to 0. By weighting each input neuron’s connection to the output neuron, <strong>we can effectively give every pixel a weight</strong>. This is an important concept, so spend a second thinking about this to make sure it makes sense.</p><p>With weighting, we can make some pixels more important in calculating the output neuron’s value, and we can make some pixels less important.</p><p>Say we gave all the neurons that correspond to pixels in the middle of the image a <strong>positive weight</strong>. Let’s say we also gave all the input neurons that correspond to pixels around the edges of image a <strong>negative weight</strong>. This means that dark pixels in the middle of the image make the output neuron’s value go up, while dark pixels on the edges of the image make the output neuron’s value go down.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*hhPdTSpzegLkXc3ImaQbWw.jpeg" /></figure><p>Now let’s imagine what would happen if we put an image of the number 1 into our weighted neural network. Images of the number 1 (like the one above) tend to have a ton of dark pixels in the middle of the image. Because the pixels are really dark, the numbers in the neurons that represent these pixels will be close to 1. We have these middle neurons’ connections weighted <strong>positively</strong>, which means that the value of the output neuron will be large and positive. Once the output neuron’s number has been <strong>transformed </strong>with the <strong>sigmoid activation function</strong>, the output neuron’s number will be super close to 1. (If the transformation aspect is confusing, take another look at the graph below.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*iOvHpfiCCtiq9PzaK3ojKA.png" /></figure><p>In contrast, let’s imagine what would happen if we put an image of the number 0 into our weighted neural network. Images of the number 0 tend to have more dark pixels at the edges of the image. Dark pixels on the edges of the image mean that the input neurons that represent the edge pixels will have values that are close to 1. Since these edge neurons’ connections are <strong>negatively</strong> weighted, the value of the output neuron will also be negative. Once the output neuron’s number has been <strong>transformed </strong>with the <strong>sigmoid activation function</strong>, the output neuron’s number will be close to 0.</p><p>With this system, images of the number 1 tend to make the output neuron’s value very close to 1, while images of the number 0 tend to make the output neuron’s value close to zero.</p><p>This gives us a simple classification rule.</p><p>If the output neuron’s value is quite close to 1, then the image is a 1. If the output neuron’s value is quite close to 0, then the image is a 0.</p><p>You may be wondering how the network defines “quite close” when it comes to the output neuron’s value. This is where training comes in! If we ran a bunch of pictures of 1s and 0s through this neural network, it would become clear that second layer neuron values that are larger than a certain number (not necessarily 0.5) mean that the image is probably a 1.</p><p>However, this super simple neural network structure has a major limitation. If we tried to test it with a new image, we’d get the number inside the output neuron, but the neural network wouldn’t actually make a guess. From here, another program or a person would have to look at the output neuron’s value and see whether it seems high enough to be a 1 or low enough to be a 0.</p><p>Fortunately, there’s a pretty simple solution to this issue.</p><h3><strong>Neural network with multiple output neurons</strong></h3><p>The neural network we just imagined weighted the middle pixel neurons’ connections positively and the outer pixel neurons’ connections negatively.</p><p>But there’s no reason we had to weight the middle neurons’ connections positively and the outer neurons negatively. If we wanted to, we could have weighted the outer neurons positively and the middle neurons negatively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8DmueDunFFzModN2-O-y3A.jpeg" /></figure><p>This inverse weighting would mean we get inverse results. An image of a 0 would cause the output neuron’s value to be close to 1, while an image of a 1 would cause the output neuron’s value to be close to 0.</p><p>If we wanted the neural network to automatically make a guess during testing without having to actually look at the output neuron’s value, we could put a neural network with our original weighting next to a neural network with inverse weighting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qRc-flWzZBNFjLQahoHdDw.jpeg" /></figure><p>We would put the same image into both neural networks. If the original-weighted neural network’s output neuron had a higher value, then the image would be classified as a 1. If the reverse-weighted neural network’s output neuron had a higher value, then the image would be classified as a 0.</p><p>Another way of expressing this idea is a neural network with two output neurons! This is simply putting the two neural networks above on top of each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KABQcmvogDzs3ZTfNQ3bqA.jpeg" /></figure><p>If the top neuron’s value is higher, then the image is classified as a 1. If the bottom output neuron’s value is higher, then the image is classified as a 0.</p><p>It’s important to highlight that each neuron’s connection to each output neuron has its own weight.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EfxQjhPGtIuJLxukDPdf7w.jpeg" /></figure><p>It’s totally allowed for neuron A’s connection to neuron B to be weighted super heavily, even if neuron A’s connection to neuron C is weighted super lightly. Or A–B could be positively weighted, and A–C could be negatively weighted. Connections between two neurons are weighted, but neurons themselves are not.</p><p><em>This</em> is closer to how real neural networks classify images. Each classification possibility has its own output neuron, and whichever output neuron has the highest number is the classification that the neural network guesses.</p><p>In this case, 1 and 0 each have their own output neuron. If the 1 output neuron’s value is higher, then the neural network guesses 1. If the 0 output neuron’s value is higher, then the neural network guesses 0.</p><p>If things are making sense so far, then you now understand the structure of a very simple <strong>binary classification neural network!</strong></p><p>But there is an aspect of this neural network that we haven’t addressed in much detail.</p><p>How exactly do you set the weights of each connection?</p><p>We have the basic idea of how it’s done: depending on which output neuron we’re talking about, the connections between the <em>middle</em> input neurons and the output neuron we’re talking about are weighted either positively or negatively to identify the hole in the middle of the 0 or the line in the middle of the 1.</p><p>But really, that’s pretty vague. I mean, it’s not like somebody goes in and manually sets the weights for each connection. This would be tedious for even an 8x8 image, and it would be insanely hard for a 100x100 image!</p><p>The answer lies in a concept called the <strong>loss function</strong>.</p><h3><strong>The loss function</strong></h3><p>If our neural network were perfect, we know what we’d want it to do. If we showed it a 0, we’d want the 0 output neuron’s value to be exactly 1, and we’d want the 1 output neuron’s value to be exactly 0.</p><p>This would mean the neural network is telling us that it is <em>sure</em> it’s seeing a 0 when it actually is seeing a 0.</p><p>Of course, it’s basically impossible to reach this level of perfection. But it is possible to quantify how far away we are!</p><p>For each image, we do this by finding the difference between the <strong>ideal value </strong>and the <strong>observed value </strong>of each output neuron, then squaring this result. We add these squared values together to get our result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O4mVLxYY8kqQWRwx8GRFWA.jpeg" /></figure><p>The function that produces this result is called <strong>the loss function</strong>. People always want to minimize losses, and we want to minimize the result of our loss function to make the neural network as accurate as possible.</p><p>But how do we know how to minimize the loss function?</p><p>Let’s start by understanding the function that produces the <strong>observed value</strong> of one output neuron. Remember, this value is generated by multiplying each input neuron by the weight of its connection to the output neuron we’re working with, and adding these products all together. Using a bit of math, we could write that process like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GNERBHVx9ZoBH16PBB-Xxw.jpeg" /></figure><p>And the <strong>ideal value</strong> is simple to find for each output neuron. If we want the test image to be classified as the output neuron’s image, the ideal value is 1. If we don’t want the test image to be classified as the output neuron’s image, the ideal value is 0. One reason we use the <strong>sigmoid function</strong> is so that we can make the ideal value a nice 0 or 1, instead of trying to take a wild guess.</p><p>If the ideal value is 1, we just subtract 1 from the equation above. The expanded form of the difference between the observed value and the ideal value in this case looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sTcPPEboQUYsVHaqx2LtQw.jpeg" /></figure><p>The <strong>loss function</strong> for a <em>single output neuron</em> is the square of this big thing. In this case, this is the loss function of the output neuron that indicates if the image represents a 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mUKnhS88WpTgeVh-dEiQHg.jpeg" /></figure><p>To get the loss function for an <em>entire image</em>, you repeat this process for each output neuron. If the output neuron you’re working with corresponds to the actual image shown, then the ideal value is 1. Otherwise, the ideal value is 0.</p><p>At the end, you add together the loss function for each output neuron.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P3l6t4g4tPNfmIytUgg0QA.jpeg" /></figure><p>This gives you the<em> loss function for the entire image</em>. The lower this value, the better the neural network did for the image.</p><p>But we’re not quite done. This is just the total loss function for <em>one</em> image.</p><p>This is where things start getting crazy.</p><p>To find the total loss for <em>all </em>images, we repeat this entire process for every image in the entire set of training data. At the end, we add together the loss functions for each image to get a single mind-bogglingly massive loss function for the <em>whole neural network on the entire training set</em>. The lower this gargantuan loss function is, the better the neural network is doing!</p><p>The reason we don’t just minimize the loss function for a single image is because we need to make sure the neural network can classify different kinds of images. If we just tried to minimize the loss function for a single image of the number 1, the neural network would find this easy. The best way for it to minimize this image’s loss function would be to simply <em>classify every single image as a 1,</em> no matter what. What a smartass.</p><p>We want to force the neural network to classify images as either 1s or 0s based on their appearance, so we add together the loss functions for each image to make the neural network take all images into consideration.</p><p>Now that we’ve explored what the loss function is, we can take a look at how the computer minimizes it.</p><h3><strong>Minimizing the loss function</strong></h3><p>The loss function that we just unpacked has three inputs:</p><ol><li>The value of each input neuron</li><li>The ideal value of each output neuron (1 or 0, depending on the neuron)</li><li>The weights of each neuron’s connection with each output neuron</li></ol><p>The value of each input neuron depends on the image the neural network is being fed, so we can’t change that. The ideal value of each output neuron also depends on the image, so we can’t change that either.</p><p>But we can do whatever we want to with the weights!</p><p>The weights are the component that the computer adjusts to minimize the loss function.</p><p>To understand a bit about how this works, let’s look at a 2D function.</p><p>Let’s say we start at a random point on this curve, and we want to find our way to the bottom.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OX8g_3vzPCPlYkIGjPCQyA.jpeg" /></figure><p>But here’s the catch: we can’t actually see the rest of the function. We can see <em>just</em> enough to figure out whether the function is going up or down where we are, but no more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/136/1*ByAU5-JG2JY54rI0-58vug.jpeg" /></figure><p>That’s not a big problem, though. If we’re trying to get to the bottom, we can just hop down a little.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*pXteC7yJ3Eq0XE1fxUMy3A.jpeg" /></figure><p>And if we’re not down yet, we just hop down again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*YjhfUjWFPYDwlqYUY5t8pQ.jpeg" /></figure><p>And again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*flxeyWNovFQ_bQkz82zGiQ.jpeg" /></figure><p>If we realized we hopped too far and found the curve pointing the other way like that, we know we hopped over the bottom, which is awesome! Then we take a smaller hop the other way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/336/1*oeOhZbWX3uk1eHicEUKceg.jpeg" /></figure><p>At this point, we know we’re pretty close to the bottom.</p><p>The computer does something similar to minimize the loss function!</p><p>The computer adjusts one weight at a time, while leaving the others constant. On the graphs above, the vertical axis is the value of the loss function, and the horizontal axis is the value of the weight that’s being adjusted.</p><p>But just like the example above, the computer only calculates enough of the loss function to see the slope of the function.</p><p>That’s okay for the computer though! Just like we did, the computer uses the slope of this graph to decide which direction to adjust the weight. If the slope is really steep, the computer adjusts the weight a lot, and if the slope isn’t that steep, the computer adjusts the weight just a little.</p><p>After the computer adjusts one weight, it moves onto the next weight and repeats the same process. It keeps doing this for the weight of every single connection in the entire network!</p><p>After all the weights are adjusted, the computer looks at all of the training data again to figure out where the new weights put it on the loss function graph. The computer then minimizes adjusts the weights more to further minimize the loss function, creating <em>another</em> new set of weights.</p><p>This process continues, potentially thousands of times, until the programmers tell the computer to stop or the computer is satisfied that it’s minimized the loss function.</p><p>And this all happens automatically during a <em>single training session</em>!</p><p>In short, training a neural network involves adjusting the weights of each connection between input and output neurons to minimize the loss function.</p><p>If that sentence makes sense to you, then you have a solid grasp on the basics of neural network training!</p><p>There’s just one more piece of the puzzle we’ll add now.</p><h3><strong>Structure of a bigger neural network</strong></h3><p>So far, we’ve been working with a neural network that only has two output neurons: one that corresponds to an image of a 0, and another that corresponds to an image of a 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KABQcmvogDzs3ZTfNQ3bqA.jpeg" /></figure><p>But by now, you may have guessed that it’s possible to add an output neuron for every possible number from 0–9.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oSWiM_o9F33UO-hdfDskOw.jpeg" /></figure><p>This neural network operates just like the one we’ve been discussing! The only difference is that it can be trained on the entire dataset, and it can identify any number from 0–9.</p><p>When training begins, the neural network sets random values for the weights of each connection. Over multiple cycles of optimization, the neural network gradually adjusts the weights to minimize the loss function and maximize accuracy.</p><p>But there’s another, even cooler aspect of neural networks that we haven’t even discussed yet.</p><p>So far, we’ve been linking input neurons directly to neurons that try to identify an entire number.</p><p>But what would you guess happens if we put a whole line of neurons in the middle of the network? This line of neurons is called a <strong>hidden layer</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*NzvvOv3k3GNht_aaidSZyQ.jpeg" /></figure><p>For each input neuron, the <strong>hidden layer</strong> acts like an output neuron. For each output neuron, the hidden layer acts like an input neuron.</p><p>A beautiful thing occurs here. Following the optimization process we discuss above, the neurons in the hidden layer<strong> </strong>naturally learn to identify <em>subsections of numbers</em>! These small subsections can be things like lines, curves, and dashes that work together to form a full number, just as pixels work together to form lines, curves, and dashes. Often, the subsections of numbers that hidden layers identify are shapes that look nothing like part of a number to the human eye, but rather have only subtle mathematical relevance to the shape of a complete number.</p><p>In other words, the neural network uses pixels to identify small parts of numbers. It then uses these small parts of numbers to identify entire numbers.</p><p>Let’s just stop and take a moment to realize how <em>epic </em>that is.</p><p>Unleashing the equations and optimization procedures we’ve discussed above <em>naturally</em> causes hidden layers of neurons to identify subsections of numbers. The laws of mathematics alone identify deep patterns, without any human intervention.</p><p>And you don’t have to stop at just one hidden layer. Two, three, or hundreds of hidden layers can be added to models for different results. Each hidden layer can also contain different numbers of neurons. The possibilities for messing around with hidden layers are endless.</p><p>If you’re wondering how a programmer can improve a model’s accuracy once it’s been trained, <em>adjusting hidden layers</em> is a common thing to try. (It’s also worth mentioning that there are dozens of completely different neural network structures, each with their own fascinating and unique characteristics.)</p><p>Hidden layers are the way in which neural networks learn to recognize patterns of patterns (of patterns of patterns of patterns…), as we discussed in the beginning of the article. In the digit classification problem, hidden layers let the neural network identify patterns of numbers from patterns of number subcomponents from patterns of pixels.</p><p>The possibilities for utilizing neural networks extend far beyond handwriting recognition. Neural networks can be relevant to any activity that involves recognizing patterns. They can be relevant to cooking, flying planes, building houses, optimizing supply chains, writing AI-generated papers, growing peas, teaching languages, and almost any other activity that you could possibly think of.</p><p>That’s why it seems like AI is in every single new technology you ask about.</p><p>Patterns are at the core of everything.</p><p><em>Note: If you want to learn more about neural networks and AI, there are tons of high-quality resources online. Andrew Ng’s coursera courses on machine learning and deep learning are great places to start.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=207c31e0b1ff" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/behind-the-eyeballs-of-ai-how-neural-networks-work-207c31e0b1ff">Behind the Eyeballs of AI: How Neural Networks Work</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Cryptocurrency Works: A Deep Explanation of the Fundamentals]]></title>
            <link>https://medium.com/@a.jeremymah/how-cryptocurrency-works-a-deep-explanation-of-the-fundamentals-ddc5f8aa53e3?source=rss-17b3955c7601------2</link>
            <guid isPermaLink="false">https://medium.com/p/ddc5f8aa53e3</guid>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[A. Jeremy Mahoney]]></dc:creator>
            <pubDate>Thu, 27 Aug 2020 06:26:26 GMT</pubDate>
            <atom:updated>2020-08-27T19:07:47.095Z</atom:updated>
            <content:encoded><![CDATA[<p>“Cryptocurrency is going to displace the US dollar any day now!”</p><p>“Watch out for cryptocurrency, bro. It’s basically just a big network of organized crime.”</p><p>“My grandpa is investing his retirement savings in cryptocurrency. A brilliant man, he is.”</p><p>I don’t know about you, but for years, I’ve heard all sorts of stuff about this thing called cryptocurrency. Most of what I’ve heard about cryptocurrency is in the form of interesting-sounding statements like these. These are statements I’ve really wanted to have an opinion on.</p><p>But all these years, I’ve had a problem.</p><p>I had no idea what cryptocurrency actually is.</p><p>Cryptocurrency, to me, was one of the most nebulous concepts of the modern world. Googling its definition wasn’t really that helpful.</p><p>“A <strong>cryptocurrency</strong> (or <strong>crypto currency</strong>) is a digital asset designed to work as a <a href="https://en.wikipedia.org/wiki/Medium_of_exchange">medium of exchange</a> wherein individual coin ownership records are stored in a <a href="https://en.wikipedia.org/wiki/Ledger">ledger</a> existing in a form of computerized <a href="https://en.wikipedia.org/wiki/Database">database</a> using <a href="https://en.wikipedia.org/wiki/Strong_cryptography">strong cryptography</a> to secure transaction records, to control the creation of additional coins, and to verify the transfer of coin ownership.” (<a href="https://en.wikipedia.org/wiki/Cryptocurrency">https://en.wikipedia.org/wiki/Cryptocurrency</a>)</p><p>This is one of those definitions that only makes sense <em>after</em> you understand the thing it’s trying to define. But until you understand the thing, you’re out of luck.</p><p>I finally had some time on my hands and decided that I was tired of not understanding anything about cryptocurrency, so I spent the past few days reading and watching everything I can find about the topic. The goal of this article is to get you from where I was a week ago (barely knowing what cryptocurrency is) to being able to give a decent explanation to your friends about the mechanics of cryptocurrency.</p><p>To learn about cryptocurrency, we’ll start with a simple situation and gradually build up to the inner workings of Bitcoin, the world’s most popular form of cryptocurrency.</p><h3><strong>The ledgers</strong></h3><p>Jim, Debbie, Katie, and Ted are pals.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*NETCD-_rG6AiROQ3ktl_jQ.jpeg" /></figure><p>Like any good pals, they’re cool with lending each other money when they do things together like go out to eat. Of course, they do like being repaid.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-28wED3whkr0XtB0mLL5AQ.jpeg" /></figure><p>But since everyone lends money to everyone all the time, remembering who owes money to whom starts getting pretty confusing. In a moment of inspiration, they come up with the idea of just writing down the money people give each other on a piece of paper, which they call the ledger.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eDh7smAJDpI6Wwy1cGImoQ.jpeg" /></figure><p>At the end of the month, everyone just pays each other in real dollars. This is way more convenient than trying to remember everything! Ted just brings the ledger every time they hang out.</p><p>But one day, Debbie notices a transaction on the ledger that she doesn’t remember.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eMlV0RXX-t31BUbXm_6wNQ.jpeg" /></figure><p>Debbie says Ted just wrote it down so he could get some extra cash, and Ted says Debbie just forgot about it. To prevent this from happening in the future, the group decides that the person who is <em>paying</em> must sign each transaction. If each transaction is written in pen, then signatures should eliminate this kind of confusion. In the future, the ledger should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P8WE4Izr_4SqwTixkLdU4g.jpeg" /></figure><p>For several months, everything goes smoothly, with everyone signing their own name every time they paid someone. But during one end-of-month pay day, Ted realizes that he doesn’t have the $150 necessary to pay everyone back.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mKuKvwyDn3LO7gw3kE2bIw.jpeg" /></figure><p>Fortunately for Ted, the group is in a forgiving mood, but they come up with another idea to prevent this from happening again. Everyone puts as many dollars as they want into an envelope that they keep at home and promise not to spend, and they automatically receive that number of ledger dollars (LD). Nobody is allowed to go into ledger dollar debt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/895/1*csMeOPWvvir7Il5RzoxwMQ.jpeg" /></figure><p>Now that Ted’s shown that he’s kind of a jerk with money, Ted isn’t allowed to take the ledger home. When they bring this up to Ted, he gets annoyed. After some compromise they decide that software developer Jim will make a simple ledger website that they can all look at whenever they want to make sure nobody is messing with it. (Of course, Jim knows he could still game the system if he wants because he has access to all the code, but he doesn’t tell anyone. Nobody else realizes this, and Jim seems honest enough, so it’s okay.)</p><p>The website is pretty simple. There are no accounts or anything. If someone wants to make a transaction, they just put it into the ledger website, just like writing on the paper ledger. Once a transaction has been made, it can’t be removed. It’s just like writing with pen on a virtual piece of paper.</p><p>But there’s one issue. With the paper ledger, people signed their name whenever they paid someone else. But it wouldn’t work for someone to just type their name next to the transaction, because everyone’s typing looks the same. How is it possible to make a signature on a computer?</p><p>To understand digital signatures, there’s one key concept that’s important to understand at a basic level: the <strong>hash algorithm</strong>.</p><h3><strong>The hash algorithm</strong></h3><p>An <strong>algorithm<em> </em></strong>is a set of steps that a computer performs an an input, resulting in an output. For instance, I could create an algorithm called add(firstNumber, secondNumber) that outputs the sum of firstNumber and secondNumber.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0k3MB0JCj6u8jhXoGqy04w.jpeg" /></figure><p>As long as firstNumber and secondNumber are both numbers, the set of steps inside the algorithm stays the same, and the result will always be firstNumber plus secondNumber.</p><pre>add(9, 2) = 11<br>add(1, 4) = 5</pre><p>Different kinds of algorithms can do <em>tons </em>of different things.</p><p>One type of hash algorithm is the <strong>256 bit Secure</strong> <strong>Hash Algorithm</strong>, commonly known as <strong>SHA-256. </strong>SHA-256 takes just one input: a message of any length, which can contain both numbers and letters.</p><p>Using some complex math that we won’t delve into here, SHA-256 turns the input message into a 256 bit output. (A bit is a 1 or 0 in a computer’s memory, so “256 bit” just means 256 ones and zeros long.) This 256 bit output is called a<strong> hash</strong>. So, the function of a hash algorithm is to turn an <strong>input message</strong> into a <strong>hash</strong>.</p><p>For example, SHA256(“hello”) outputs the following <strong>hash</strong>:</p><pre>SHA256(&quot;hello&quot;) = 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824</pre><p>The long string of letters and numbers is the <strong>hash.</strong></p><p>Finding the hash value that corresponds to an input message is often called <strong>hashing</strong> the input message.</p><h3><strong>Why hashes are so important</strong></h3><p>There are two reasons that hashes are super important in cryptocurrency.</p><p><strong>Reason 1</strong></p><p>If the input message is changed even a <em>tiny</em> bit, the hash output will change <em>completely</em>.</p><pre>SHA256(&quot;hello&quot;) = 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824</pre><pre>SHA256(&quot;helllo&quot;) = 8848f587ba9e5cc326947054eb394a4fbe0cae5036a837ceb93b3b8b224d6b9d</pre><p>If you want to experiment and find the hashes of different messages, <a href="https://cryptii.com/pipes/hash-function">this</a> site lets you try out SHA-256 for real.</p><p><strong>Reason 2</strong></p><p>It’s very easy to hash a message, but it’s virtually impossible to generate a message from a hash. So it’s super easy to get from “hello” to “2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824” using the hash algorithm, but there’s no way you could get from “2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824” to “hello”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9OEK6gziSGrd2QFkMQh98w.jpeg" /></figure><p>So it’s not at all like language translation. Language translation is two-way, which means that you can translate from French to Japanese or from Japanese to French. The hash algorithm is one-way, which would be like being able to translate French into Japanese, but not Japanese into French.</p><p>This is a really funky thing to think about at first, so it helps to use prime numbers as an analogy.</p><p>Prime numbers are numbers that can’t be evenly divided by any whole number. 17 is a prime number, because you can’t divide 17 by any whole number and be left with a whole number.</p><p>Now let’s say I ask you to multiply two really massive prime numbers. Let’s say</p><pre>7438045953401190140561498842842418462037335114546168161692065073499519441019501961530711285174006849<br>*<br>8289882780788090274389516624364752519998653672494424589035748665004515220123413652619657316439217539</pre><p>If you and your friends had a few hours (days?), you could bust this one out on a long sheet of paper. It would be awfully tedious, and you’d make some mistakes, but the set of steps is simple enough that a bunch of bright fourth graders could probably pull it off.</p><p>Alternatively, you could just google “big number calculator,” and it wouldn’t take you long to find out that the answer is</p><pre>61660529071811060254084326509709240105406225710293317404699562919158609570113733192337799392972742964983266265470974751072977341757518028447138276653970978577596489771776529799129908610611758986924611.</pre><p>But now let’s say I asked you to do the opposite. Let’s say I gave you that huge number above and asked you to find the two prime numbers that multiply to form it. Now THAT would be hard. Where would you even start? Guess and check isn’t gonna work too well when your guesses are 30 digits long. In fact, it would take even a powerful computer and our best algorithms quite a long time to figure that one out.</p><p>SHA-256 doesn’t use prime numbers like that, but the end result is similar. The message is like the two pretty big prime numbers, and the hash is like the massive product number. Just like it’s pretty easy to use the two prime numbers to find the massive product number, it’s pretty easy to use the message to find the hash. And just like it’s super difficult to find the two prime numbers from the massive product number, it’s almost impossible to find the message from the hash.</p><p>Now that we understand hashes, we can move on to how digital signatures work.</p><h3><strong>Digital signatures</strong></h3><p>In the world of digital signatures, every person has a number that they never share with anyone. This number is a user’s digital soul. This number is called a private key. Here’s what a private key looks like:</p><pre>83f4a51c2649dfc8d291cc44cc05caeeca87166c5532d8a61f4b95e00cddc428</pre><p>You might be thinking that this looks like a hash value, and you would be correct. Private keys and hash values are stored in the same format.</p><p>Every user also has a public key. A user’s public key is generated by hashing their private key.</p><pre>SHA256(&quot;83f4a51c2649dfc8d291cc44cc05caeeca87166c5532d8a61f4b95e00cddc428&quot;) = 38e96f324a9b1fb902cb30a4543e80ae60296451c4ed4279bc74376f2faf1119</pre><p>The public key is shared, but the private key never is. The beauty of the hash algorithm is that it’s easy for a user to determine their public key based on their private key, but it’s impossible for anyone else to determine a user’s private key based on their public key!</p><p>Public and private keys, combined with the hash algorithm, make it possible to digitally sign a document. To see how, let’s watch Debbie securely pay Ted with Jim’s new online system. For our reference, Debbie’s private key is fd67583d9674023d1a1e1ae2aa4af8bbd5c916a81c1ab729e597188a1448744d. Don’t tell her we know it.</p><p>First, Debbie types out her transaction on the website. Debbie also sends her public key to Ted.</p><pre>Debbie pays 50 LD to Ted.</pre><p>Then, Debbie hashes this message.</p><pre>SHA256(“Debbie pays 50 LD to Ted.”) = 38e96f324a9b1fb902cb30a4543e80ae60296451c4ed4279bc74376f2faf1119</pre><p>Debbie then adds her private key to the end of this hash.</p><pre>38e96f324a9b1fb902cb30a4543e80ae60296451c4ed4279bc74376f2faf1119fd67583d9674023d1a1e1ae2aa4af8bbd5c916a81c1ab729e597188a1448744d</pre><p>Finally, Debbie hashes this entire thing to get her digital signature for this transaction.</p><pre>SHA256(38e96f324a9b1fb902cb30a4543e80ae60296451c4ed4279bc74376f2faf1119fd67583d9674023d1a1e1ae2aa4af8bbd5c916a81c1ab729e597188a1448744d) = 209ac80dda3f65dd82987b4b271372d39b66a07f9416285dee791ee568ec4d52</pre><p>209ac80dda3f65dd82987b4b271372d39b66a07f9416285dee791ee568ec4d52 is Debbie’s digital signature that goes with this transaction!</p><p>To “sign” the transaction, Debbie sticks her digital signature next to the transaction, just like this.</p><pre>Debbie pays 50 LD to Ted.<br>209ac80dda3f65dd82987b4b271372d39b66a07f9416285dee791ee568ec4d52</pre><p>By itself, the digital signature doesn’t look that useful. But there is an algorithm that can verify that the signature actually came from Debbie! The <strong>validation algorithm</strong> has three inputs:</p><ol><li>The message (in this case, “Debbie pays 50 LD to Ted.”)</li><li>Debbie’s <strong>public</strong> key</li><li>Debbie’s signature that corresponds to the message</li></ol><p>The algorithm can mathematically determine whether it is possible to create the inputted signature by combining the hash of the message and the private key that <em>corresponds</em> to the inputted public key. This algorithm can do its job without ever knowing the private key. As a result, Ted, or anyone else, can determine whether it was actually Debbie who signed on to any transaction Debbie makes!</p><p>Because Debbie needed her private key to produce the signature, nobody else could have produced it unless they know Debbie’s private key. Assuming Debbie hasn’t given anyone else her private key, only Debbie could have produced this signature, just like a physical signature.</p><p>Because the hash algorithm is one-way, nobody could figure out Debbie’s private key by looking at her digital signature.</p><p>In fact, a digital signature is even better than a physical signature. A digital signature is message-specific. If somebody changed the wording of the original message after Debbie signed it, even a <em>tiny </em>bit, the hash of the message would be different. This would cause the validation algorithm to say the signature was invalid, which would cancel the transaction. This way, if Ted wants to modify the message to “Debbie pays 60 LD to Ted.” <em>after</em> Debbie signs it, Debbie’s signature would no longer apply to the modified message!</p><h3><strong>Decentralization</strong></h3><p>So far, Ted and friends have created a system that’s a good deal more secure than pen and paper. Use of internal ledger dollars prevents people from spending more than they actually have, and use of digital signatures prevents people from modifying past transactions or spending each other’s money.</p><p>But one massive question remains.</p><p><strong>Do we trust Jim?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1_Cksr0GCRNs0ROt91dLlg.jpeg" /></figure><p>And if this system is expanded to include everyone in the world, does everyone trust Jim?</p><p>Because Jim and Jim alone ultimately controls the ledger website, Jim has a whole lot of power. It is true that Jim can’t spend other people’s money, thanks to digital signatures. But if he wants, Jim can delete transactions entirely. If he felt like it, Jim could delete every transaction where he pays someone and leave every transaction where someone pays him. Jim would get every ledger dollar in existence, and nobody would be able to prove that Jim shouldn’t really have them, because Jim deleted all the records. Basically, Jim is King. (If this sounds scary to you, now is a good time to mention that this is pretty much what banks could do too.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i2N0v6MOgbVW7d3oUCil-w.jpeg" /></figure><p>To make this more democratic, let’s have every user notify everyone else whenever they pay anyone.</p><p>Then, every user can automatically record each transaction on their computer. Every user now has a complete record of all transactions on their own computer. This way, if Jim tries to screw around with the records, all the other users will be able to call him out and keep going with business as usual.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bBWkomOWgfMA6BbU_HO7Gw.jpeg" /></figure><p>This process of broadcasting every transaction to every user is known as <strong>decentralization. </strong>Decentralization takes the documentation power out of the hands of a single entity (Jim), giving every person a share of the documentation power instead.</p><p>But there’s one major loophole left in this system.</p><p>Let’s say Ted is tired of hanging out with Jim and Debbie and Katie and wants to get everything he can out of them before he moves to Uzbekistan. Ted only has 100 LD left, but he knows how he can make the most of it. First, Ted uses all 100 of his ledger dollars to buy Debbie’s old flute from her. But Ted decides to be a little twit and <em>only </em>broadcasts the transaction to Debbie. He doesn’t broadcast the transaction to Katie or Jim, so they don’t know that he spent the money.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y9deN64OIAaYWttwChdkEA.jpeg" /></figure><p>After receiving Debbie’s flute, Ted immediately uses <em>the same 100 LD </em>to buy Jim’s merino wool sweater. He only broadcasts this transaction to Jim, not Debbie or Katie. Even though Ted spent all his ledger dollars on Debbie’s flute, neither Katie nor Jim <em>know </em>about this, because he didn’t broadcast his transaction to them. According to Jim and Katie’s records, Ted hasn’t spent any money at all.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zaiRqLLKklkVRx9au3hOsw.jpeg" /></figure><p>Finally, Ted uses all his ledger dollars to buy Katie’s old suitcase. Once again, he only broadcasts this transaction to Katie, leaving Jim and Debbie in the dark.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p40OqfKBk2SMeHFYzdw6AQ.jpeg" /></figure><p>After collecting his loot, Ted makes his getaway and is never seen again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*km-xqRGW4rSeltYwxWQ_hQ.jpeg" /></figure><p>Jim, Debbie, and Katie eventually figure out about Ted’s antics by talking to each other, but not before Ted leaves the continent.</p><p>Pulling a Ted would be even <em>easier</em> if there were more users. We need to find a way to close this loophole.</p><p>You may not realize it, but after we close this loophole, Jim’s ledger website will be a fully fledged cryptocurrency. In fact, it will closely resemble the most popular form of cryptocurrency in the world: Bitcoin.</p><p>From here on out, we’ll no longer just look at the imaginary ledger website that Jim created. We’ll be exploring the mechanics of Bitcoin itself.</p><h3><strong>Blockchain</strong></h3><p>For this system to work, we need lots of users. Let’s leave the world of Ted, Jim, Debbie, Katie and and the ledger and move to a bigger world where there are enough people to get this thing running.</p><p>So far, we’ve just been using one huge ledger for everyone. But with thousands of users, this ledger is going to get real long real fast. Opening it up to verify transactions is going to be a mess. So let’s break it up into segments called <strong>blocks</strong>. Each block has a limited number of transactions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6SkMFYEfnXMtUgibXf8pXQ.jpeg" /></figure><p>To form the big ledger from the blocks, we’ll just put the blocks in chronological order, or in a <strong>chain.</strong> This chain of blocks is called the <strong>blockchain.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IvuqWPbcRxoSVTL2afAo1A.jpeg" /></figure><p>Each time somebody wants to pay someone else, they follow the steps that we’re familiar with. They identify the recipient of their money, they send the recipient their public key, they digitally sign their transaction, and they record it on the ledger. For the sake of privacy, the larger system identifies people by an address only, not their name. So a couple of transactions might look a bit like this.</p><pre>1BvBMSEYstWetqTFn5Au4m4GFg7xJaNVN2 pays 2 LD to 3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy</pre><pre>3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy pays 31 LD to bc1qar0srrr7xfkvy5l643lydnw9re59gtzzwf5mdq</pre><pre>1BvBMSEYstWetqTFn5Au4m4GFg7xJaNVN2 pays 14 LD to bc1qar0srrr7xfkvy5l643lydnw9re59gtzzwf5mdq</pre><p>Pretty familiar so far. But here’s where blockchain is a little different from what we’re used to. The reason for this might not make sense at first, but keep reading, and you’ll get it.</p><h3><strong>How mining works</strong></h3><p>To be considered legitimate, each block must be <strong>mined </strong>once it fills up with transactions. Don’t worry about the purpose of mining just yet. We’ll cover that in the next section. For now, just focus on what mining is and how it’s done.</p><p>Because every transaction is broadcast to every user immediately, every user can see every block. As a result, anybody involved in the blockchain network can mine a block, whether or not a transaction of theirs is in the block that they’re mining. Mining a block is no easy task. Mining is attempting to solve what initially seems to be a <em>stupidly</em> difficult math/luck problem:</p><p>“Find a sequence of digits to add to the end of a block such that the block’s hash begins with a specified number of zeros.”</p><p>Let’s break this down. It’s possible to take the hash of any block’s raw text. For instance, the hash of the sample block above looks like this:</p><pre>SHA256(“1BvBMSEYstWetqTFn5Au4m4GFg7xJaNVN2 pays 2 LD to 3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy</pre><pre>3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy pays 31 LD to bc1qar0srrr7xfkvy5l643lydnw9re59gtzzwf5mdq</pre><pre>1BvBMSEYstWetqTFn5Au4m4GFg7xJaNVN2 pays 14 LD to bc1qar0srrr7xfkvy5l643lydnw9re59gtzzwf5mdq”)=</pre><pre>2571ed3daa513daeaf920592baf9c514845ed11ed1bb3e69780e240c75bddf33</pre><p>Remember, adding even a single character to the the block’s text would change the block’s hash completely. The goal of mining is to find a number to add to the block’s text to make the hash start with a specified number of zeros. For this example, we’ll say that the hash needs to start with 17 zeros. In other words, it the hash would need to look something like this:</p><pre>00000000000000000101cfe6d29df69535dbaf11bc4e10867e7b570a20bacee4</pre><p>(Note: Each block contains more than just text, but for the sake of explanation, we’ll ignore the other parts of the block.) The number that is added to the block is called a <strong>nonce</strong>.</p><p>There’s no way to predict what nonce you need to add to the block so that the block’s hash will start with 17 zeros. The nonce could be 23909670187093840, it could be 0987983423847928374, or it could be totally different. To add a nonce, you basically just stick the nonce in front of the block’s text. So if we were trying out the nonce 0987983423847928374 with the transaction above, this resembles the thing that we’d hash:</p><pre>SHA256(&quot;09879834238479283741BvBMSEYstWetqTFn5Au4m4GFg7xJaNVN2 pays 2 LD to 3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy</pre><pre>3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy pays 31 LD to bc1qar0srrr7xfkvy5l643lydnw9re59gtzzwf5mdq</pre><pre>1BvBMSEYstWetqTFn5Au4m4GFg7xJaNVN2 pays 14 LD to bc1qar0srrr7xfkvy5l643lydnw9re59gtzzwf5mdq&quot;) = c385b44997249f7a1c24ac5e0dc430008cd786432c30d395cd52b1b6d7bedad0</pre><p>Looks like that one didn’t give us a hash with 17 zeros. Actually, it didn’t even give us a hash with 1 zero.</p><p>To find the correct nonce, a computer has to randomly guess a nonce, add this nonce to a block, and calculate the hash of the block with the nonce added. The computer keeps doing this over and over and over again until it finally finds a nonce that produces a hash with 17 zeros at the beginning. This process is called mining because its tediousness resembles real mining. The miner’s computer has to keep trying nonces over and over again, just as a real world miner has to tunnel through miles of stone.</p><p>The person who discovers the nonce that produces a hash with 17 zeros is said to have <strong>solved</strong> the block. Once somebody solves a block, that block cannot be mined again. Everyone switches to mining the next block, and the process continues.</p><h3><strong>Why does anyone mine?</strong></h3><p>Mining is at least as difficult as it sounds. As computers all over the world compete to mine a block, billions of nonces are tested. According to the New York Times, the energy that it takes to mine a single block could <a href="https://www.nytimes.com/2018/01/21/technology/bitcoin-mining-energy-consumption.html?action=click&amp;contentCollection=Times%20Insider&amp;module=RelatedCoverage&amp;region=Marginalia&amp;pgtype=article">power a house for 2 years</a>.</p><p>You’re probably wondering why on earth anyone would put their poor computer through this toil. The answer is that any person who successfully mines a block is rewarded handsomely, in two ways.</p><ol><li>New bitcoins are created that go straight to the miner’s account. In August of 2020, a miner who solves a block receive 6.25 bitcoins, which is equal to about $70,000 US Dollars.</li><li>Whenever a user makes a payment, they must set aside a small amount of money as a transaction fee. The miner of a block receives the transaction fee for every transaction in the block they mine.</li></ol><p>Mining is the only way new bitcoins are ever generated. Bitcoins aren’t “printed,” like traditional currency. Every four years, the number of bitcoins a miner receives for mining a block is halved. When Bitcoin began, a miner received 50 bitcoins for solving a block. This reward was eventually halved to 25 bitcoins, then 12.5 bitcoins, and finally 6.25 bitcoins today. When a miner receives this reward, we say that they “mined bitcoins”.</p><p>When Bitcoin was founded, the creators decided that there will only ever be 21 million bitcoins in circulation. As of August 2020, there are about 18.5 million bitcoins in circulation. This means that 88% of bitcoins that will ever exist have already been mined.</p><p>To control the rate at which new bitcoins are created, an algorithm associated with the Bitcoin network periodically determines how hard mining should be. The algorithm does this simply by changing the number of zeros that a solution hash has to contain! I used 17 zeros as the example when explaining how mining works, but the required number of zeros could be above 30 or less than 10.</p><p>The goal of this algorithm is to keep the average time it takes for a block to be solved at around 10 minutes. When lots of computers are mining, the algorithm makes solving blocks more difficult by requiring the solution hash to contain more zeros. On the other hand, when there aren’t a ton of people mining, the algorithm makes solving blocks easier by reducing the number of zeros that the solution hash must contain.</p><p>Because the reward for solving a block halves every four years, you may be wondering why people would still mine in the future. Forty years from now, the bitcoin reward for solving a block will have been cut in half 10 times, to 0.006 bitcoins. The answer is that miners will still receive users’ transaction fees, which will still provide a significant economic incentive.</p><h3><strong>Why is mining even a thing?</strong></h3><p>We’ve explored how mining works, and the incentives that individuals have to mine in the first place. But this whole mining thing probably still seems pointlessly hard and wasteful of electricity. Here’s where it all comes together.</p><p>After someone solves a block, everyone starts mining the next block. But that solved block doesn’t just disappear. It stays in the chain, playing an important role.</p><p>Let’s imagine Block A, Block B, and Block C, in order.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xtzb_aOFxXhMRIB_MN1ErQ.jpeg" /></figure><p>Remember, the hash of the solved block’s text+nonce combo is unique. This unique hash is then put at the <em>beginning </em>of the next block. So Block A’s hash is put at the beginning of Block B. Once Block B is solved, then Block B’s hash is put at the beginning of Block C, and the cycle continues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tO9vwG9FGkcedNK90MNTFA.jpeg" /></figure><p>This is critical, because the fact that each block’s hash is in the next block means that the blocks can never be taken out of order! To understand why, let’s see what would happen if you swapped blocks B and C.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CufrMLpsTp_kONqMt3n7ZA.jpeg" /></figure><p>The hashes wouldn’t line up, and it would be obvious that someone messed with the ordering.</p><p>Even more importantly, the connections between the blocks prevent anyone from secretly changing the contents of any block. Remember, if even one character is changed in a block’s text, the hash of this text changes completely. But the nonce for each solved block only works with <em>precisely</em> the original text. If a block were altered, then the nonce that corresponds to that block wouldn’t work. That would unsolve the block.</p><p>In fact, changing the contents of even one block would unsolve <em>every single block in the entire chain</em>.</p><p>Let’s say somebody messed with the text of Block A to remove a transaction. This alteration would change Block A’s hash, which would unsolve block A. But since Block A’s hash is at the beginning of Block B, changing Block A’s hash would also change Block B. Changing Block B would unsolve Block B, and changing Block B also changes Block B’s hash. Because Block B’s hash is at the beginning of Block C, changing Block B’s hash would also change and unsolve Block C…see where this is going? These changes would progress down the <em>entire</em> chain, unsolving every block and making it obvious to every user that something had been altered.</p><p>This is the digital version of withdrawing a penny from someone else’s safe and having the bank collapse on top of you.</p><p>You might think that a fraudulent user could just change Block A, then mine Block A again to find a new nonce that works with the altered Block A. However, finding a new nonce for Block A would change Block A’s hash, which would then make it necessary to mine Block B again. Finding a new nonce for block B would change Block B’s hash, which would make it necessary to mine Block C again. This cycle would continue, and a record-altering con artist would have to re-mine <em>every single block in the entire blockchain.</em></p><p>And <em>this </em>is why they make mining so damn difficult.</p><p>Because a single con artist would have no chance of pulling this off. It took countless watt-hours of electrical energy–contributed by computers all over the world–to mine every block in the blockchain. One malicious person acting alone, or even a huge organization, would never be able to re-mine the entire blockchain after altering records.</p><p><a href="https://andersbrownworth.com/blockchain">This</a> is a website with an excellent mini-simulation of the blockchain. You can mess around with changing the contents of each block to see how it disrupts the whole chain.</p><p>The blockchain also makes Ted’s heist impossible. Let’s explore why.</p><h3><strong>Foiling Ted</strong></h3><p>High in the mountains of Uzbekistan, Ted hears about bitcoin and decides to join. After conducting a few legitimate transactions to get a feel for the system, Ted decides to try his hand at fraud. He does, after all, have some experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xP4_1TiH5R-ALuP6fl0d0w.jpeg" /></figure><p>After a bit of googling, Ted finds a pair of beautiful Uzbek earmuffs for sale on Reddit. The seller’s bitcoin address is attached to their Reddit post, so Ted opens Bitcoin and sends 0.0035 BTC (around $40) to the seller. However, Ted is running a modified version of Bitcoin software that allows him to broadcast his transaction <em>only</em> to the seller, rather than the entire bitcoin community. This is exactly what he did to Debbie, Jim, and Katie. Ted thinks that once he’s sent the seller the money in exchange for the earmuffs, he can simply resend these bitcoins to other people without anyone else in the community realizing.</p><p>Thanks to the design of the blockchain, this isn’t quite what’s happening.</p><p>Ted doesn’t realize it, but by only broadcasting his purchase to one person, he’s actually started a new block in the blockchain that diverges from the rest of the chain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DfJLhiZXyCs7S5sBYGPrrw.jpeg" /></figure><p>Ted’s block includes his payment to the sock seller, while the main block doesn’t. As a result, Ted’s block and the main block will have different hashes. This means that they will have different nonce solutions.</p><p>People can only mine blocks that they know about. Everyone knows about the main block, so powerful computers all over the world are working on mining the main block. But Ted and the seller are the only ones who know about Ted’s block. Ted is literally competing against the whole world to mine. The mining race basically looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Af3mAMEH7JE9vX3SZGOmpQ.jpeg" /></figure><p>Bitcoin is calibrated so that a new block is solved every 10 minutes. This means that after an hour, the seller sees this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QKGA1EEup4XN7jsuJ5zPDw.jpeg" /></figure><p>This looks awfully sketchy to the seller. Remember, the seller is getting told about <em>both</em> Ted’s block and the main block. Standard blockchain procedure is that <em>the longest chain is always the trusted chain</em>. The longest chain always has the most computing power mining it. It’s clear that practically nobody knows about the single block in which the seller was paid because nobody is trying to mine it.</p><p>After one glance at this, the seller is going to decide not to ship the socks after all, and Ted’s fraudulent block will be forgotten forever.</p><p>In the real world, people usually don’t trust a transaction until an hour or two after it was made, when around 6-12 blocks have been mined after it. This time is called <em>verification time</em> because it prevents jerks like Ted from screwing up the system.</p><p>For Ted–or some organization with Ted’s intentions–to actually pull off this kind of scheme, they would need to control over 50% of the computing power in <em>the entire Bitcoin blockchain</em>. The only way to verify their fraudulent chain would be for them to mine their fraudulent chain faster than <em>everyone else combined</em>.</p><p>This would be virtually impossible for an individual to accomplish. Even if a tremendous number of organizations with access to high mining power cooperated, it’s unlikely that they would want to change every block in the exact same way, which would be required to accomplish this manipulation.</p><p>What I find beautiful about Bitcoin is that it relies on the robustness of truth. It’s pretty hard for networks of humans to agree on anything. But when it comes down to a choice between truth and one person’s lie, the majority tend to choose the truth. At least, in the world of Bitcoin they do.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ddc5f8aa53e3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>