<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kiara Cuter on Medium]]></title>
        <description><![CDATA[Stories by Kiara Cuter on Medium]]></description>
        <link>https://medium.com/@KiaraCuter?source=rss-a5c1faedece------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Gy4Kf6uWWh8YCykP91e7VA@2x.jpeg</url>
            <title>Stories by Kiara Cuter on Medium</title>
            <link>https://medium.com/@KiaraCuter?source=rss-a5c1faedece------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 19:05:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@KiaraCuter/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Mental Models for the Modern Maverick]]></title>
            <link>https://medium.com/@KiaraCuter/mental-models-for-the-modern-maverick-834d6893c6b?source=rss-a5c1faedece------2</link>
            <guid isPermaLink="false">https://medium.com/p/834d6893c6b</guid>
            <category><![CDATA[self-improvement]]></category>
            <category><![CDATA[inspiration]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <dc:creator><![CDATA[Kiara Cuter]]></dc:creator>
            <pubDate>Mon, 04 Feb 2019 13:30:03 GMT</pubDate>
            <atom:updated>2019-02-05T15:43:15.794Z</atom:updated>
            <content:encoded><![CDATA[<h4>Chamath Palihapitiya On Change, Power, And People</h4><blockquote>[Businesses that can stand the test of time] are by definition hard and non-obvious. And so they compound in scale very slowly. That’s really counterintuitive to the overriding logic of Silicon Valley which says, “Quick, fast, go.” That kind of premature ejaculation doesn’t work if you’re trying to be around forever.</blockquote><p>With razor-sharp wit and an unmatched candor — If you haven’t heard of Chamath Palihapitiya up until this point you’ve been seriously missing out. The pride of Waterloo University and former refugee from Sri Lanka; Palihapitiya is one of the Valley’s most prominent venture capital investors and entrepreneurs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZyR4pI6PrcN6CCXK.jpg" /></figure><p>Palihapitiya is the founder &amp; CEO of Social Capital, a venture capital firm dedicated to transforming society with technology. In addition to this, he was also an early senior executive at Facebook!</p><p>It doesn’t take a genius to have an inkling that Chamath is an extremely knowledgeable man. You can take it from this 17-year-old.</p><p>In this article, I’m going to be breaking down the wisdom and musings of one of the world’s most innovative minds. Trust me, its stuff that can be so <strong><em>profoundly simple</em></strong> that you’ll double back, but so <strong>fundamentally life-changing</strong> it might just steer you to where you need to be.</p><h3><em>Business and Leadership</em></h3><blockquote>“To be an entrepreneur right now is <em>the </em>most powerful thing that you can do. It is our version of the Renaissance. So people that are creating companies now, especially if they create them correctly, have the ability to do two things:</blockquote><blockquote>#1 is massively impact the world in really important ways.</blockquote><blockquote>#2 is capture exceptional amounts of wealth.”</blockquote><p>One of the beliefs that echo throughout the work and speeches delivered by Palihapitiya is that being an entrepreneur is one of the most powerful things to be in our current day.</p><h4><em>Climate Change, Cancer, and Security, Oh My!</em></h4><p>It would be an understatement to say the world has no lack of problems that need solving. This puts entrepreneurs in a rather demanding, but duly rewarding pinch designed by the world’s demands.</p><p>Chamath once said there is always scarcity somewhere. Deciding to organize around safety, inventing languages, creating money, building a machine, creating a smartphone all have something in common.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CWm5pRqWj-GLw28d" /><figcaption>Photo by <a href="https://unsplash.com/@olav_ahrens?utm_source=medium&amp;utm_medium=referral">Olav Ahrens Røtne</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>They all stem from one thing. Someone was asking the question, what is the next great problem that needs to be solved?</p><blockquote>“Where I stand today, what is the scarce resource that exists? What limits progress?</blockquote><blockquote><em>That</em> is the thing that you need to focus on. And what’s interesting in technology is that answering that question has always been true, and has always been the path to build immense success, wealth, and value.”</blockquote><blockquote><em>How does this tie even remotely to material wealth? Is there even a correlation?</em></blockquote><h4>Capital and the Driving of Change</h4><p>The reality is, not every entrepreneur can change the world. Few people will be able to lay upon their deathbeds and say they truly played a part in changing the world. That their impact on this planet and its denizens had greatly surpassed that of their shadow on the ground.</p><p>I believe any college student rushing unto the subway in the bitter cold, nose bit by the virulence of a Canadian winter and anyone who’s ever struggled to make ends meet would be able to tell you that this world, runs on money.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DeEfKt7h6q4pWZs1" /><figcaption>Photo by <a href="https://unsplash.com/@visuals_by_fred?utm_source=medium&amp;utm_medium=referral">Freddie Collins</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Money may be the way for you to get that fancy car you’ve always wanted, the big home, some semblance of ‘respect’. But to Chamath, money is powerful for a different reason. A reason that delights all those looking to earnestly solve the world’s biggest issues. To Chamath, Capitalism more than an economic system is a philosophy.</p><figure><img alt="Image result for chamath" src="https://cdn-images-1.medium.com/proxy/0*eTEFXohSPqGAYaWy." /></figure><p>It is a philosophy that states: <strong>We will direct the change we wish to see with our ‘votes’ and our investments.</strong> You have a unique viewpoint that should be shared. Without capital that’s irrelevant.</p><p>The most successful (and non-coincidentally influential) people, he says, have built a network of influence based on capital. Their worldview is propagated into the world in an unbelievably aggressive rate that has been compounding for decades. Their opinions on change respective to them and around them direct our reality.</p><blockquote>It doesn’t matter whether you agree with their worldview, the question is: What’s the counterweight to their worldview? Can you name one? Can you name a group of people that operate in the exact same way, methodically, meticulously, assembling capital, influence, assets, who believe the opposite?</blockquote><blockquote>Get the money. Get the money, and then let’s get around a table, and let’s create new rules. Literally that simple: get the fucking money. I’m serious.</blockquote><blockquote>It is going to be made, it is going to be allocated, and you have a moral imperative to make sure that if you have a point of view that matters, and you want to reflect it, you get it.</blockquote><p>At Stanford, Palihapitiya pressed into his seat, his expression a mixture of strain, urgency, and excitement. The future, he asserted, would be a competition of views. If you control the capital, <em>the podium is yours.</em></p><p>Aside from the limelight, he said, a better distribution of capital globally would mean a greater diversity of viewpoints present among change-makers.</p><blockquote>The more rational actors we have, and the more of a balanced, fair system. That’s what diversity means <em>to me</em>.</blockquote><h4>TLDR:</h4><p>To have an impact or assert your worldview you must first obtain capital. Without capital your grand visions for the world, and your <em>unique </em>and <em>valuable </em>opinions on how society progresses and creates provisions are irrelevant. Oh and — if we fail to push this vision into mainstream we actively encourage a monopolization on society by the richest people alive.</p><h3>Life and Success</h3><p>We all enter the world with some fundamental deficiency. Be it confidence, support, mental fortitude, wealth, or health, every human is hindered by an aspect of the self. These hindrances prevent us from reaching our full potential, and cripple us if left unresolved.</p><p><strong><em>That is to say the turnstile of life turns round and round. Waiting for us to pick up our baggage before we move on to where we ought to be.</em></strong></p><p>Chamath is worthy of praise because of his indomitable work ethic, and clarity of judgement. But, above all else, I believe he is revered because he is a champion of the people. He is a living case of a man who fought the age-old battle against his own baggage and came out not only living, but <em>thriving. </em>In his own words:</p><blockquote>I’ve been extraordinarily lucky to fight through a lot of that to get to the other side. What I feel every day, is I have a responsibility to be authentically myself in a way that energizes other people</blockquote><p>In his commemorative birthday Medium article entitled, <a href="https://medium.com/@chamath/life-eb8d2efd9746">“Life”</a>, Chamath reflected upon his time on earth until this point and his goals for the next chapter of his life. His goals for the next 13,383 days ( 36.6 years) of his life. Put another way, his goals to complete before the end of his life.</p><blockquote>1) Family — I want to be around them, love them, support them, hug and kiss them.</blockquote><blockquote>2) Friends — I want to support them, embellish them, promote them, confide in them, count on them.</blockquote><blockquote>3) Memories — I want to think back on events and have an emotional reaction. Capital is a lubricant to my and other’s ideas no matter how big, small, indulgent or sacrificial as long as I’ll have some awesome reaction after it’s done.</blockquote><blockquote>4) Legacy — I must do something that lasts well after these next 13,383 days.</blockquote><p>He concluded with a simple action plan. One so simple I think everyone reading this should implement it into their weekly tasks. Palihapitiya decided that he would be asking himself the following questions at the end of each week in hopes of reaching his goal.</p><blockquote>1) How many days did I do the things above?</blockquote><blockquote>2) How many days did I just give into the norm vs push back? Fight back…</blockquote><blockquote>3) How many days was I happy? Like, really happy. Not fake, cosmetic happy.</blockquote><blockquote>4) How many days did I give up what I really wanted because someone else’s marginal validation tricked me from focusing on the bigger picture?</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1T4FiHWAvVk4T-Fc" /><figcaption>Photo by <a href="https://unsplash.com/@ashkned?utm_source=medium&amp;utm_medium=referral">Ashley Knedler</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Dear Reader, I do hope you implement some form of this in your life, and let it guide you. Find your deficiencies and tackle them head-on. Find what it is that you want to do. Find out what aspects of this action plan resonate with you, and grasp them firmly.</p><blockquote>There should be one culture: Win. We fucking win. When we’re done winning, we win some more. When we do that, we have winning for dessert, and then we go to bed in a blanket of winning. It’s about winning.</blockquote><p>What is “winning life” to you? What do you want to pursue? and better yet:</p><p><em>How are you going to get there?</em></p><p>If you liked this Article:</p><ul><li>Follow me on LinkedIn and Medium</li><li><a href="https://theksociety.us7.list-manage.com/subscribe/post?u=4b1a04b2ea48a24c4fe617e67&amp;amp;id=faa6af79c7">Join my monthly newsletter</a> (No spam, just a once a month overview on my projects and nuggets of knowledge)</li></ul><p>*If you’d like to hear more about Chamath’s viewpoints I&#39;d highly suggest these resources:*</p><p>His Medium — <a href="https://medium.com/@chamath">https://medium.com/@chamath</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fux78B5ogVR4%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dux78B5ogVR4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fux78B5ogVR4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/9cbcaa6e5b39da1b93b00884de33bff3/href">https://medium.com/media/9cbcaa6e5b39da1b93b00884de33bff3/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FnHHCrTjclN4%3Fstart%3D2025%26feature%3Doembed%26start%3D2025&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DnHHCrTjclN4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FnHHCrTjclN4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2cc39bcb08ab6f7b501207596fa059a9/href">https://medium.com/media/2cc39bcb08ab6f7b501207596fa059a9/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FvvIR13eS8FE%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DvvIR13eS8FE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FvvIR13eS8FE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e823128cb9ecf022fafc1837f6aff119/href">https://medium.com/media/e823128cb9ecf022fafc1837f6aff119/href</a></iframe><figure><a href="https://usejournal.com/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_image"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f2IVAl0TbsfES9cFGYr40g.png" /></a></figure><p>📝 Read this story later in <a href="https://usejournal.com/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_read_later_text">Journal</a>.</p><p>🗞 Wake up every Sunday morning to the week’s most noteworthy Tech stories, opinions, and news waiting in your inbox: <a href="https://usejournal.com/newsletter/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_text">Get the noteworthy newsletter &gt;</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=834d6893c6b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Parsing In Tongues: Neural Machine Translation]]></title>
            <link>https://medium.com/@KiaraCuter/parsing-in-tongues-neural-machine-translation-58a0b24f0533?source=rss-a5c1faedece------2</link>
            <guid isPermaLink="false">https://medium.com/p/58a0b24f0533</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Kiara Cuter]]></dc:creator>
            <pubDate>Mon, 04 Feb 2019 02:29:02 GMT</pubDate>
            <atom:updated>2019-02-05T15:44:36.078Z</atom:updated>
            <content:encoded><![CDATA[<h4>How Google Translate — Translates.</h4><p>First period and I’m sitting in class. Today we have a “work period”, which (for the uninitiated) can mean one of two things… Every student in the class is either doing something or — <em>an abundance of nothing</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*T5M57p4zO9LhxHt6" /><figcaption>Photo by <a href="https://unsplash.com/@danielcgold?utm_source=medium&amp;utm_medium=referral">Dan Gold</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In an attempt to shake the boredom from the room, my friend takes it upon herself to teach a small group of us basic Spanish. We all crane in, to marvel at the ease of her speech and the foreign feeling of the same sentences on our tongues. The call and response, and the entertained look of my peers when a word shared similarities to an English counterpart all take me back to the clamor of my 5th-grade Spanish class in Chicago.</p><p>I remember how there was always a few students to scramble to Google Translate in an attempt to finish the previous night’s homework, and how panic settled on their faces when they noted that the translation didn&#39;t seem <em>exactly</em> like what should be used in the context given.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*zaK7KBzFtPFodu4CyauDYw.gif" /></figure><p>I doubt my old class recognized that this is a problem that is being tackled on the daily.</p><h4><strong>Translation is a Large Task</strong></h4><p>Language is the structured use of words, yeah — but it’s also a fundamental aspect of our communication as humans. What you probably don&#39;t think about on a day to day is why it’s important to make sure things we say don&#39;t get lost in translation.</p><h3>Grammarly on Twitter</h3><p>Fun with #words: put the word &#39;only&#39; in different places in this sentence to change its meaning.</p><p>In the example above, the addition of “only” in different parts of the sentence DRASTICALLY changes the meaning of the phrase entirely. This presents a simple truth:</p><p>In order to preserve ideas, stories, and the results of that one Buzzfeed quiz you took when you were bored you must preserve<strong> <em>meaning </em></strong>when translating into another language.</p><p>But how do we do this when text is riddled with connotations and nuances?</p><p><strong>Machine/Deep Learning = ‘Teachable’ Computers</strong></p><p>Machine Learning (ML) and Deep Learning (DL) are terms that have been used extensively in the past couple of years and dropped in conversations across the globe. I won&#39;t be delving into these terms in this article.</p><blockquote>So for now, all you need to know is that :</blockquote><blockquote><em>Deep Learning is a machine learning technique that teaches computers to do what comes naturally to humans: learn by example.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*B-dvp1JdI6-ZoUCS" /><figcaption>Photo by <a href="https://unsplash.com/@franckinjapan?utm_source=medium&amp;utm_medium=referral">Franck V.</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Just as a baby would take account for its surroundings in order to distinguish between items in a new room. Deep Learning allows for computers to be fed large amounts of data so that they can apply their observations to new contexts/scenarios.</p><h3>Statistical Machine Translation</h3><p>Statistical Machine Translation (SMT) is a usage of Deep Learning that dominated the machine translation scene <strong>for decades</strong>.</p><p>SMT uses predictive algorithms to teach a computer how to translate text. These decryption models are made using parallel bilingual text corpora. <em>Which is a fancy way of saying large sets of text in different languages that have corresponding meanings.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Z9JmCCE6tNTp7EgQ" /><figcaption>Photo by <a href="https://unsplash.com/@frankieguarini?utm_source=medium&amp;utm_medium=referral">Frankie Guarini</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>SMT models use this to create the “most probable output” <em>(the best translation our computer friends can muster)</em>, based on different bilingual examples. Using this already translated text, a statistical model guesses or predicts how to translate to foreign language text.</p><p>SMT systems would evaluate the fluency of a sentence in the target language (what we are translating to) a few words at a time using an <strong>N-gram language model.</strong></p><blockquote><em>“If we have an N-gram model of order 3, when it generates a translation, it will always assess the fluency by looking at n-1 previous words,” he said. “So in this case it would be 2 previous words.”</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*SFGzZ8Zh527XnpOI" /><figcaption>Fluent? All together? Not quite.</figcaption></figure><p>This means that given a sentence of any length, an SMT system with a 3-gram language model will make sure every three words would be fluent together. When adding new words to a sentence, this means new words will be contextually fluent to the two words before it.</p><p><em>But…The sentence overall usually isn’t all that great.</em></p><p>One benefit of Statistical MT is our ability to set and leave a model to work its magic. But a large drawback is that this system <strong>needs </strong>bilingual material to work from. This means translation to <a href="http://unitedlanguagegroup.com/stories/editorials/challenge-bidirectional-languages/">obscure languages</a> can be near impossible.</p><blockquote>“Get on with it then… How has Google changed their ways?”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*mH4KnRFf49dDhz4Aogjllw.gif" /></figure><h3>Neural Machine…What?</h3><p>It’s Neural Machine Translation (NMTs). They’re pretty awesome.</p><p>Let me tell you why.</p><h4>NMT Systems Consider Entire Sentences</h4><p>NMT models assess the fluency of translated sentences differently.</p><p>These models use recurrent neural networks. In other words, words that are generated at each position in a sentence in our target language are dependent in some manner on <strong>all the previous words</strong> in that sentence.</p><blockquote>In essence, where SMT is limited to how many words its N-gram model dictates, NMT evaluates fluency for the entire sentence.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*Uru7IjI2v7dIUawrnX9z5Q.gif" /><figcaption>Happy dance for sentences we can understand</figcaption></figure><h4>Pay ‘Attention’ To This One</h4><p>In the calculations of the hidden layer in an algorithm (the calculations we don’t see) a specific ‘sorting method’ can be used on top of NMT architecture to rank the correlation of words in a phrase. This method is called <strong>self-attention.</strong></p><blockquote><em>Hold up. I just heard you “ugh”.</em></blockquote><p>Before you have more of a guttural response to the word “algorithm” in the blurb above — Don’t worry! I’ll explain self-attention minus the mind-numbing math (and with some pictures!)</p><p>When looking at this picture, you may find yourself inclined to look at the dog, then maybe the rocks, perhaps followed by the magnificent view.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ln98uT3_IsaaVrsW" /><figcaption>Photo by <a href="https://unsplash.com/@jfbrou?utm_source=medium&amp;utm_medium=referral">Jf Brou</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong><em>But are you aware of how your eyes are determining the context of the photo based on various focal points, colours, and recognized textures?</em></strong></p><p><em>HOW DID YOU EVEN KNOW THAT WAS A DOG???</em></p><p>This deciphering of the image is extremely similar to the attention process used in NMT. This is because,</p><blockquote>Attention is, to some extent, [modeled after] how we pay visual attention to different regions of an image or correlate words in one sentence.</blockquote><p>In the way that you may have made mental note of the dog’s ears and canine snout to determine it was, in fact, a dog, you may have disregarded the scenery to make this conclusion.</p><p>We can explain the relationship between words in this sentence or another close context in a similar way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*raN_CLcD_1dSAXor.png" /><figcaption>Note that the relationship between “eating” and apple is stronger than that between “eating” and “green”.</figcaption></figure><p>The word “eating” in the above alludes to the presence of a food in the sentence (like “apple”). Whereas the color “green” describes the food and is present in the sentence, but it is not as directly related to “eating”.</p><p>In an NMT model that utilizes self-attention, words in a sentence are given “ratings” pertaining to how correlated they are to one another. In the following example, “eating” is highly correlated to “apple” — therefore, it is described as an area of high attention.</p><p>So, in attention, we assign weighted values according to how strongly one word is correlated to others. This “weight” contributes to a hierarchical structure that helps the neural network draw conclusions about a word’s importance within the context of the sentence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_-NSMoH7jB9FDmheqkckIQ.png" /><figcaption>This matrix of alignment scores is a shows the correlation between source and target words.</figcaption></figure><p>Using this weighted system allows NMT models to understand the links between multiple components of a sentence — so we’ve got translations that make sense! Yay!</p><h4>NMT Learns The Complex Ties Between Languages</h4><p>SMT and NMT systems are trained pretty differently.</p><p>SMT systems have three separate main components:</p><ul><li>the translation model that calculates the proper translation for words between languages</li><li>the reordering model that reorders words in the output</li><li>the language model (the N-gram model)</li></ul><p>These models are all taught independently from each other. This means there was little to no interdependence between them. The problem here, of course, is that some languages require a heavier usage of certain models (more reordering, more fluency checks, etc).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bFFCPfPpzXxnJDgj" /><figcaption>Photo by <a href="https://unsplash.com/@rawpixel?utm_source=medium&amp;utm_medium=referral">rawpixel</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In NMT systems, however, we are looking at a singular ‘all-encompassing’ model that has several components. But these components are all jointly trained. This allows for interdependencies between the unique features of different languages to be better understood by our translation systems. In essence, (through the usage of Neural Networks) NMT systems can model the complex relationships between words in language translation.</p><p>Google’s NMT system: GMNT began by using a sequence to sequence model, which I have implemented below.</p><p>This NMT model consists of two recurrent neural networks: the encoder RNN which reads the source sentence and build a context vector or a sequence of numbers that represents the sentence meaning. The decoder uses this context vector to produce a language.</p><p>Broadly, the task of an encoder network is to understand the input sequence. The decoder, then, processes the sentence vector to come up with a translation.</p><p>That’s pretty brilliant, isn&#39;t it?</p><h3>Let’s make out own NMT model! (+self-attention)</h3><p>Let’s begin with importing libraries and sorting out dependencies. Don’t forget to download our English and Spanish datasets!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/859538c42e8c9218b5a35dd8cbfb80a0/href">https://medium.com/media/859538c42e8c9218b5a35dd8cbfb80a0/href</a></iframe><p>Now for some data preprocessing! I’ve converted the Unicode file to ascii for starters. I’m also creating a space between words and the punctuation following them.<br>ex: “NMT is fun.” → “NMT is fun .”</p><p>Let’s also replace everything with space except punctuation and letters (a-z, A-Z “.” “?” “!” “,”) and add a start and an end token to the sentence so the model knows when to start and stop ‘reading’ the sentence to predict the translation.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c59482d8f5d9187c2322372909171cc2/href">https://medium.com/media/c59482d8f5d9187c2322372909171cc2/href</a></iframe><p>The class <em>LanguageIndex </em>gives each word a number or <strong>index</strong> (ex. “dog” → 2) and vice-versa <br>(2 → “dad”) for each language.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/58989ad1109930f55943e0983e51cf7d/href">https://medium.com/media/58989ad1109930f55943e0983e51cf7d/href</a></iframe><p>We’re now going to vectorize the input and target languages. Then, we’ll set the max_lengths(s) for the input (English) and output tensor (Spanish) to the longest sentence in their respective dataset.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5e456fa5ea274ae25df63e8b58b9f140/href">https://medium.com/media/5e456fa5ea274ae25df63e8b58b9f140/href</a></iframe><p>Here we’re setting the size of our example dataset and creating training and validation sets using an 80–20 split. The last bit of code checks for a GPU.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7db2eed7fd7517a025b639aab8fe842d/href">https://medium.com/media/7db2eed7fd7517a025b639aab8fe842d/href</a></iframe><p>With the groundwork laid down, we can <em>finally</em> start making our Encoder and Decoder RNNs!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/48c7342164e0bca4aa76c65c9997d20c/href">https://medium.com/media/48c7342164e0bca4aa76c65c9997d20c/href</a></iframe><p>Next, we feed our datasets into the Encoder and Decoder RNNs and create a loss function.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76f4d3e401b6d1113ff6c26a9b76d584/href">https://medium.com/media/76f4d3e401b6d1113ff6c26a9b76d584/href</a></iframe><p>Set some checkpoints to save the current state of our weights so that we can pick up from where you left off with training. If you don’t checkpoint your training models at the end of a job, you’ll have lost all of your conclusions!</p><p>Then we can begin training our model! I set my test to have 10 EPOCHS but feel free to play around with the number of training iterations to see a variance in translation accuracy/cohesiveness.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6a4ebb0dac1bda7ebdac90a06bdb9af3/href">https://medium.com/media/6a4ebb0dac1bda7ebdac90a06bdb9af3/href</a></iframe><p>Next, we’ll make an evaluation function that processes the input sentence and stores an initial intuition as to its translation as a vector. Our self-attention weights will also be stored so we can plot them for a neat visualization later!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f82dff9ad5377f0322d9b1e335c14874/href">https://medium.com/media/f82dff9ad5377f0322d9b1e335c14874/href</a></iframe><p>Let’s take those stored attention weights and create a plotting function for our visualization —</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3203bd9552c64d705ad1009ea196674e/href">https://medium.com/media/3203bd9552c64d705ad1009ea196674e/href</a></iframe><p>Last but not least: our translation function! Calling upon the evaluate function, our translate function prints our input sentence and our model’s predicted translation in our target language (Spanish).</p><p>Ask the user for keyboard input and translate it towards the end for that authentic Google Translate feel!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ac8a4bbb204db04b40fa762cc96e4589/href">https://medium.com/media/ac8a4bbb204db04b40fa762cc96e4589/href</a></iframe><p>And we’re done! Your result should look something similar to this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/0*_bOog4WY1og-jpHd" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/0*H7eR0yQcZhvmxXXu" /></figure><h3>Congratulations on creating your own NMT model with self-attention!</h3><p>Hey! You’ve reached the end. If you liked this article:</p><ul><li>Feel free to reach out to me on <a href="https://www.linkedin.com/in/kiaracuter/">LinkedIn</a></li><li>Comment your thoughts + Be sure to follow me on <a href="https://medium.com/@KiaraCuter">Medium</a> for more content</li></ul><figure><a href="https://usejournal.com/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_image"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f2IVAl0TbsfES9cFGYr40g.png" /></a></figure><p>📝 Read this story later in <a href="https://usejournal.com/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_read_later_text">Journal</a>.</p><p>🗞 Wake up every Sunday morning to the week’s most noteworthy Tech stories, opinions, and news waiting in your inbox: <a href="https://usejournal.com/newsletter/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_text">Get the noteworthy newsletter &gt;</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58a0b24f0533" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AlphaFold — Predicting Protein Structure]]></title>
            <link>https://medium.com/@KiaraCuter/alphafold-predicting-protein-structure-2f84746377da?source=rss-a5c1faedece------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f84746377da</guid>
            <category><![CDATA[google]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Kiara Cuter]]></dc:creator>
            <pubDate>Sun, 16 Dec 2018 16:04:47 GMT</pubDate>
            <atom:updated>2018-12-22T00:17:16.792Z</atom:updated>
            <content:encoded><![CDATA[<h3>AlphaFold — Predicting Protein Structure</h3><h4>DeepMind’s dive into scientific discovery</h4><p><strong><em>Researchers at DeepMind have recently brought together experts from the fields of structural biology, physics, and machine learning to apply cutting-edge techniques to predict the 3D structure of a protein based solely on its genetic sequence.</em></strong></p><h4>What’s so big about protein structures?</h4><p>“Proteins are large, complex molecules essential in sustaining life. Nearly every function our body performs — contracting muscles, sensing light, or turning food into energy — can be traced back to one or more proteins and how they move and change.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jQDpuTYL6dOsWwzC" /></figure><p>A protein&#39;s function is completely dependent on its form (3D structure). Take, for example, antibody proteins. These proteins, which can be found throughout our immune systems are ‘Y-shaped’, and function like hooks. By latching on to viruses and bacteria, antibody proteins can detect and tag malignant microorganisms for extermination. You may have also heard of the CRISPR and Cas9 proteins, which act like scissors to cut and paste DNA.</p><p>We’ve done a lot of work with proteins, but figuring out the 3D shape of a protein purely from its genetic sequence is a task that scientists have been trying to spearhead for <strong>decades.</strong></p><p>The challenge is that DNA only contains information about the sequence of a protein’s building blocks called amino acid residues, which form long chains. Predicting how those chains will fold into the intricate 3D structure of a protein is what’s known as the “protein folding problem”.</p><p>In larger proteins, calculating the ways in which amino acids can interact to come up with possible structures would be a near impossible task. It just takes <strong>way </strong>too much time.</p><h4>Okay cool, what does this mean for us?</h4><blockquote>The ability to predict a protein’s shape is useful to scientists because it is fundamental to understanding its role within the body, as well as diagnosing and treating diseases believed to be caused by misfolded proteins, such as <a href="https://www.ncbi.nlm.nih.gov/pubmed/25230234"><strong>Alzheimer’s</strong></a>, <a href="https://www.nature.com/news/misfolded-protein-transmits-parkinson-s-from-cell-to-cell-1.11838"><strong>Parkinson’s</strong></a>, <a href="https://cen.acs.org/articles/95/i39/Revised-view-Huntingtons-protein-misfolding.html"><strong>Huntington’s</strong></a> and <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5137970/"><strong>cystic fibrosis</strong></a>.</blockquote><p>This means that the acquiring of knowledge about the shapes of proteins and how they operate opens up new potential within drug discovery. This allows us to possibly receive new insight that improves our understanding of the body and how it works! This will enable scientists to create novel, more effective cures for diseases more efficiently. This could ultimately improve the quality of life for <strong>millions of patients around the world.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8u7lDR9h8EYbXla_" /><figcaption>Photo by <a href="https://unsplash.com/@kelsoknight?utm_source=medium&amp;utm_medium=referral">Kelsey Knight</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Where does AI come in?</h4><p>Scientists have been able to determine shapes of proteins in labs using various experimental techniques but each method relies on a great deal of trial and error, <em>which can take years and cost tens of thousands of dollars per structure.</em></p><p>This is precisely why researchers are turning to AI methods to decipher difficult proteins. The model used specifically in Google’s AlphaFold is a Deep Neural Network.</p><h4>What is a Neural Network?</h4><p>Neural networks are a set of algorithms (processes run by computers) that are modeled loosely after the human brain. These networks are designed to recognize patterns. The patterns they recognize are numerical, contained in vectors, into which all real-world data, be it images, sound, text or time series, must be translated.</p><p>Neural networks help us cluster and classify. You can think of them as a clustering and classification layer on top of the data you store and manage. They help to group unlabeled data according to similarities among the example inputs, and they sort/classify data once you’ve given them labeled datasets (take for example a box with the word “Cat” on it containing some pictures of cats).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bhFifratH9DjKqMBTeQG5A.gif" /><figcaption>Here’s an example of a classification usage! The boxes are dog + cat</figcaption></figure><p>This is great for finding patterns in a large dataset. The field of genomics has become extremely rich in data. Which means it wouldn’t be a far cry to note that deep learning approaches to our prediction problem that rely on genomic data have become increasingly popular in the last few years.</p><p>A deep neural network (as the name suggests) has a greater depth than the average neural network. This is to say, the number of node layers through which data passes in a multistep process of pattern recognition increases. You can envision these node layers as different steps in a water filtration system, where each step is designed to sift for a certain type of sediment. In the case of DNNs, this sediment is a distinct set of features based on the previous layer’s output.</p><p>The further you advance into the neural net, the more complex the features your nodes can recognize since they incorporate features from the previous layer.</p><h4>How did AlphaFold use DNNs?</h4><p>In order to create 3D representations of proteins. DeepMind trained two neural networks. One network was to predict two things:</p><p>• the distances between pairs of amino acids</p><p>• the angles between chemical bonds that connect those amino acids.</p><p>These would be used to estimate whether pairs of amino acids are near each other.</p><p>Another neural network was trained to predict the probable distances between every pair of residues in a protein (amino acid <strong>residues</strong> are the part of an amino acid that makes it unique from all the others).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*VhLOiJceYCvX1Ee7.png" /><figcaption>“Unfolded” protein sheet</figcaption></figure><p>These probabilities were then combined into a score that estimates how accurate a proposed protein structure is. A separate neural network was trained that uses all distances in unison to estimate how close the proposed structure is to the right answer.</p><p>These scoring functions were then used to search protein data to find structures that matched AlphaFold’s predictions.</p><p>It is important to note this method can be compared to techniques commonly used in structural biology, where pieces of a protein structure are repeatedly replaced with new protein fragments. DeepMind used a generative neural network to invent new fragments, which were tested in the structure to improve the score of the proposed protein structure — essentially to get closer to what the structure may be.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*afYDLpQwS1icXZBj.png" /></figure><p>The second method attempted to reach improved scores through a mathematical technique by the name of <strong>gradient descent, </strong>wherein small, incremental improvements are made to reach a local/global optima (the ideal score in this case).</p><p>This technique was applied to entire protein chains rather than to pieces that must be folded separately before being assembled, reducing the complexity of the prediction process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*9ocTy0r7fijGYm_TGoq58g.gif" /></figure><h4>What does the future hold?</h4><p>AlphaFold as a whole is a true representation of how machine learning systems can integrate diverse sources of information to help scientists come up with creative solutions to complex problems at speed.</p><p>Google’s magnificent work with this model begs the question:</p><p><strong>How can we optimize upon the familiar constraints in scientific research of time and money? Advances in AI, particularly those within machine learning, are now opening up the possibility to automate these processes.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Pl7QJRCXkuSwJna3" /><figcaption>Photo by <a href="https://unsplash.com/@hansreniers?utm_source=medium&amp;utm_medium=referral">Hans Reniers</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>The application of AI to science research has the potential to support a greater societal impact than ever seen before. It will allow us to understand and explore the use of new materials and new techniques, whilst delving deeper into old structures (like the function of the protein). Drug discovery and medical research will also benefit from these new AI-driven scientific techniques.</p><blockquote>An understanding of protein folding will also assist in protein design, which could unlock a tremendous number of benefits. For example, advances in biodegradable enzymes — which can be enabled by protein design — could help manage pollutants like plastic and oil, helping us break down waste in ways that are more friendly to our environment.</blockquote><p>A Toronto <a href="https://genecis.co/">biotech startup called Genecis</a> is currently using bacteria to digest food waste and use it to form biodegradable plastic in their tiny microbial guts. (That’s two world issues in one. Whoah.) Their next step is to rapidly engineer bacteria that converts organic waste into high-grade chemicals!</p><p>At the end of the day, these types of research breakthroughs will allow our population to live more useful and fulfilled lives. We have every reason to believe AI will speed up our arrival to them.</p><figure><a href="https://usejournal.com/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_image"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f2IVAl0TbsfES9cFGYr40g.png" /></a></figure><p><strong>This story is published in Noteworthy, where 10,000+ readers come every day to learn about the people &amp; ideas shaping the products we love.</strong></p><p><strong>Follow our publication to see more product &amp; design stories featured by the </strong><a href="https://usejournal.com/?utm_source=medium.com&amp;utm_medium=noteworthy_blog&amp;utm_campaign=guest_post_text"><strong>Journal</strong></a><strong> team.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f84746377da" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>