<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ian Kelk on Medium]]></title>
        <description><![CDATA[Stories by Ian Kelk on Medium]]></description>
        <link>https://medium.com/@iankelk?source=rss-4d92cff1902------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*rmmL_39fOOFr4OKX.jpg</url>
            <title>Stories by Ian Kelk on Medium</title>
            <link>https://medium.com/@iankelk?source=rss-4d92cff1902------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 11:02:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@iankelk/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The secret chickens that run LLMs]]></title>
            <link>https://medium.com/@iankelk/the-secret-chickens-that-run-llms-8533c9ef468f?source=rss-4d92cff1902------2</link>
            <guid isPermaLink="false">https://medium.com/p/8533c9ef468f</guid>
            <category><![CDATA[stochastic-process]]></category>
            <category><![CDATA[sampling]]></category>
            <category><![CDATA[stochastic-parrot]]></category>
            <category><![CDATA[distribution]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Ian Kelk]]></dc:creator>
            <pubDate>Mon, 06 May 2024 04:15:28 GMT</pubDate>
            <atom:updated>2024-05-06T04:16:43.912Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FdLaZ_CfPB08z1CJiIpKPg.jpeg" /><figcaption>“Inside the Chicken Lab,” generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>Humans often organize large, skilled groups to undertake complex projects and then bizarrely place incompetent people in charge. Large language models (LLMs) such as OpenAI GPT-4, Anthropic Claude, and Google Gemini carry on this proud tradition with my new favorite metaphor of who has the final say in writing the text they generate—a chicken.</p><p><em>NOTE: A CONVENIENT UNTRUTH</em></p><p><em>For much of this post, I use the term word instead of token to describe what an LLM predicts. Tokens can be punctuation marks, or parts of words — even capitalization can split words into multiple tokens (for example, “hello” is one token, but “Hello” might be split into two tokens of “H” and “ello”. This is a friendly simplification to avoid having to address why the model might predict half a word or a semicolon, since the underlying principles of the “stochastic chicken” are the same.</em></p><h3>LLMs aren’t messing around when they say they’re “large.”</h3><p>Picture a stadium full of people. Here’s <a href="https://en.wikipedia.org/wiki/Kyle_Field">Kyle Field</a> in College Station, Texas, with a seating capacity of 102,733. In this photo from 2015, it looks pretty full, so let’s assume there are 100,000 people there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B7f_Yc6Nxd1eOMxlmPnDFg.jpeg" /><figcaption>A panorama of the interior of Kyle Field in College Station, Texas. Taken at the Ball State game during the 2015 season. Attribution: <a href="https://en.wikipedia.org/wiki/User:Janreagan">Janreagan</a> at <a href="https://commons.wikimedia.org/wiki/File:Kyle_Field_Panorama.jpg">Wikipedia Commons</a></figcaption></figure><p>We’ve given each one of these very patient people a calculator along with the instruction that the person in the seat in front of them will give them some number, at which point they need to do a little work on their calculator and pass their new number to the person behind them. For the sake of this analogy, let’s assume that despite a considerable number of them being distracted, drunk, or children, they are all able to complete the task.</p><p>As the numbers travel from the front of the stadium to the back, they undergo a series of transformations. Each person’s “little work” on their calculator is akin to the operations performed by neurons in a layer of the neural network — applying weights (learned parameters), adding biases, and passing through activation functions. These transformations are based on the knowledge embedded in the model’s parameters, trained to recognize patterns, relationships, and the structure of language.</p><p>By the time the numbers reach the last person in the stadium, they have been transformed multiple times, with each step incorporating more context and adjusting the calculation based on the model’s architecture and trained parameters. This final result can be seen as the model’s output — a complex representation of the input data that encodes the probabilities of the next possible word.</p><p>The model’s final output isn’t a single number or word, though; it’s a list of words and probabilities, where each probability is the likelihood that that word will be the <em>next</em> word in a sentence.</p><p>But here’s the strange part: despite all this incredible depth of stored knowledge, we’re going to take these recommended answers provided by these thousands of people, and select the next word using a completely random process. It’s kind of like a huge pyramid where people work together to assemble a set of possible answers to a problem, then hand it at the top to a person flipping a coin.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IovVwWZ8h4Lhjgc6uH-oSA.jpeg" /><figcaption>‘Guys… I’m not sure our boss is as smart as we think he is…’<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>Actually, we can do better than this. To really illustrate the contrast between the complexity of a model with billions of parameters getting its final answer from a dumb-as-a-rock random number generator, let’s use something truly silly.</p><p>Let’s use a chicken.</p><p>To get this chicken involved, we’re going to use words and probabilities to create piles of chicken feed, each one representing a potential next word the model might generate. The bigger the pile, the higher the probability that the associated word should be selected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yz4g3UuurF-4Ssk-pKyvsg.jpeg" /><figcaption>Chickens have the amazing property to choose their food depending on the size of the piles!<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><blockquote>WARNING: I am not a licensed farmer, and this is just how I assume chickens work.</blockquote><p>The chicken, oblivious to the tireless efforts of the stadium’s occupants, simply wanders among the piles of chicken feed. The sizes of these piles influence its decision; larger piles are more alluring because they are simpler to spot and peck at, but the chicken also has ADHD. It’s whims or a sudden distraction might send it running to a smaller pile instead.</p><p>Why on earth would we do something like this? Why would we create massive, intelligent machines that ultimately rely on the random judgments of a gambling idiot?</p><p>The answer is that deep language models such as LLMs are built with neural networks, and neural networks are <em>deterministic.</em></p><h4>What does deterministic mean?​</h4><p>Being deterministic means that if you do something exactly the same way every time, you’ll always get the same result. There’s no randomness or chance involved. Think of it like using a simple calculator; 2 + 2 on any basic calculator will always show “4”. It won’t suddenly decide to show “5” one day.</p><p>Deterministic means predictable and consistent, with no surprises based on how you started.</p><p>In contrast, something that’s not deterministic (we call this stochastic) is like rolling a die; even if you try to do it the same way each time, you can get different outcomes because chance is involved.</p><h4>Why are neural networks deterministic?​</h4><p>A neural network is a complex system inspired by the human brain, designed to recognize patterns and solve problems. It’s made up of layers of artificial neurons, which are small, simple units that perform mathematical operations. Each neuron takes in some input, applies some mathematical function to it, and then passes the result on to the next neuron in line.</p><p>As a huge simplification of how these models work, these neurons are organized into layers: there’s an input layer that receives the initial data (like representations of words in a sentence), one or more hidden layers that process the data further, and an output layer that provides the final decision (the probability distribution for the predicted next word in the text).</p><p>The diagram below might look crazy complicated, but the only thing you need to understand is that each line represents some math function performed from one circle to the next. Some numbers go in one side, and some numbers go out the other side. And the stuff that comes out on the right side will always be the same for identical stuff you put into the left side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KZNBVuXgRC9DnsdttkzfSA.jpeg" /><figcaption>Maybe it looks scary, but each line connecting the dots is just a little math that never changes.<br>Generated with <a href="https://alexlenail.me/NN-SVG/">NN-SVG</a> and DALL-E 3</figcaption></figure><p><em>ANOTHER CONVENIENT UNTRUTH</em></p><p><em>Most LLMs use a more complex neural architecture called a </em><a href="https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)"><em>transformer</em></a><em>, but again, we’ll just simplify the idea for convenience. It makes no difference for this discussion, since transformers are also deterministic and require a chicken.</em></p><p>If you give the network the same input and the network has not been changed (ie., its weights, or how much it values certain pieces of input, remain the same), it will always perform the same calculations in the same order, and thus return the same output. While LLMs will have billions of these neurons, the basic idea is the same: for a given input, you will always get the same output. Typing 2 + 5 × 10 into a calculator will always give you 52, no matter how many times you do it.</p><blockquote>“The model’s final output isn’t a single number or word, though; it’s a list of words and probabilities, where each probability is the likelihood that that word will be the <em>next</em> word in a sentence.”</blockquote><h4>So why do LLMs give different responses each time to the same prompt?<a href="https://kelk.ai/blog/secret-chickens-llm#so-why-do-llms-give-different-responses-each-time-to-the-same-prompt">​</a></h4><p>This seems suspicious. If you ask an LLM the same thing multiple times, it will give you different answers each time. This contradicts the claim that neural networks are deterministic, right?</p><h3>ENTER THE CHICKEN</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AIL3xyl3wgaODTNImBdH9Q.jpeg" /><figcaption>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>This is why we need the chicken. The chicken is stochastic, which adds randomness and unpredictability to the whole system.</p><p>The previously mentioned humans in the stadium, who for some reason have deified a chicken, will present the chicken with a series of words and probabilities. Technically, these are not words but <em>tokens</em>; however, for the sake of simplifying this analogy, we’ll refer to them as words. These words and probabilities can be visualized as a series of piles of chicken feed, where each word pile’s size corresponds to its probability.</p><p>As an example, here’s a prompt that we can give to our language model to see what next word it’s going to predict: “After midnight, the cat decided to…”</p><p>If we show all the possible words that could finish this sentence, the number of food piles would match the size of the model’s vocabulary. For a model like GPT-3, this would be 50,257 piles of food!</p><p><em>NOTE: The way that GPT-3 can encapsulate all the words in 50+ languages into just 50,257 tokens is its own special magic that I’ll cover in another post. Here’s a </em><a href="https://en.wikipedia.org/wiki/Byte_pair_encoding"><em>Wikipedia link</em></a><em> in the meantime.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O7QkUpO_c-qvuZxcMnNCSg.jpeg" /><figcaption>How the cluck is an image like this going to help us understand how the chicken chooses its food?<br>Generated with OpenAI DALL-E 3</figcaption></figure><p>Clearly, trying to visualize a chicken choosing from 50,257 options isn’t very useful. Let’s instead limit our chicken’s choices to only the top 6 words that the language model chose. Here are the options for the chicken and how it might look when the chicken is presented with them:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*yrCn8RiWQN3EnXilT4-cJA.png" /></figure><h3>“After midnight, the cat decided to…”</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XRZ7T7Y_WIrYkFm4K3Zc7g.jpeg" /><figcaption>Showing the top 6 options makes things a lot easier to understand.<br>Chicken and feed generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>Let’s suppose the chicken decides to choose the word “eat,” despite it only being the third most probable word since it has the third largest feed pile.</p><p>Unfortunately, our chicken’s work is just beginning, because it will now need to choose every subsequent word in the following way. Once again, it will be presented with a list of probable words from thousands of hardworking humans in the stadium, and once again, it will have to use its tiny chicken brain to select a word. Now our prompt to complete is “After midnight, the cat decided to eat…” with the following top 6 possible words and their probability distribution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*VafxbTT9ogFkaFjteLm09Q.png" /></figure><h3>“After midnight, the cat decided to eat…”</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GtpLuMKkIZ5y8Wq-19adgQ.jpeg" /><figcaption>One word down, just a few hundred more to go.<br>Chicken and feed generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>The incredible irony of all this is that some of these language models are trained using <em>trillions</em> of words — comprising the collective works of humanity — and costing <em>millions</em> of dollars in the process. We then hand these predictions over to an unintelligent gambling chicken to choose what to actually say. With each choice from the chicken, the model’s output increasingly diverges from the sequence of the most probable words, and yet somehow, this produces more natural-sounding language.</p><p>It’s kind of bonkers.</p><p>In fact, not using the chicken — and just taking the most probable words from the LLM — generates poor language. This is called greedy decoding, and it has a whole host of problems.</p><h3>Why we need the chicken<a href="https://kelk.ai/blog/secret-chickens-llm#why-we-need-the-chicken">​</a></h3><h4>Repetitiveness<a href="https://kelk.ai/blog/secret-chickens-llm#repetitiveness">​</a></h4><p>Not using the chicken, and just taking the most probable word (greedy decoding) tends to produce repetitive or looped text. Since it always chooses the most probable next word, it can get stuck in a pattern where the same sequence of words keeps being selected. This can happen in situations where the model finds a certain pattern of text to be highly probable and, without the chicken to encourage diversity, falls into a cycle of selecting the same sequence over and over again.</p><p>Here’s a short example with some creative writing:</p><p>Prompt: “Write a short story taking place in a park in the morning”</p><p>The model begins the sentence without issue, and begins by writing “The park was serene in the early morning, with”, but then it runs into problems.</p><p>Greedy Decoding Sequence:</p><ol><li>The model predicts “the” as the most probable next word.</li><li>Following “the”, it predicts “birds” as the next word.</li><li>After “birds”, it predicts “were” as the next word.</li><li>Then, “singing” is predicted.</li><li>However, after “singing”, the model might predict “in” as the next most probable word, leading back to “the” again, and then “park”, forming a loop.</li><li>We get this conversation:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X3Tx0h6sHoMZZoD_6mbhGA.png" /></figure><h4>Well, that didn’t work.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5cIXlHUGoO5wbnV3fF-F4w.jpeg" /><figcaption>Writing is easy is easy is easy is easy…<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><h4>Lack of Creativity and Diversity<a href="https://kelk.ai/blog/secret-chickens-llm#lack-of-creativity-and-diversity">​</a></h4><p>Real human language is rich and varied, often taking unexpected turns, like a beautiful jellyfish riding a purple unicorn. By always choosing the most probable word, the generated text misses out on these creative and less predictable aspects of language, resulting in outputs that feel dull or formulaic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xjQDjtB16Bb7qEZQfhexxQ.jpeg" /><figcaption>A unicorn sting operation<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><h4>Contextual Inappropriateness<a href="https://kelk.ai/blog/secret-chickens-llm#contextual-inappropriateness">​</a></h4><p>As weird as this sounds, the most probable next word is not always appropriate. Language is highly contextual, and the best choice in one situation might not be the most statistically common one. Greedy decoding can lead to nonsensical or awkward phrases that, while grammatically correct, do not fit well with the preceding text. Here’s another example using a job application:</p><p>Prompt: “Write me a cover letter applying for a job in marketing”</p><p>Again, the model begins the sentence without issue, and begins by writing, “Dear Ms. Smith, I am writing to express my interest …” but then again we hit a pothole.</p><p>Since this is a cover letter for a job application, the language model should discuss the sender’s relevant skills, experiences, or why they are a good fit for the job. However, a contextually inappropriate continuation might look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mMkVpkBVj4cYc_KW3uPztA.png" /></figure><p>The model, perhaps trained on more dating profiles than job applications, has continued the cover letter using the <em>most</em> probable words it predicted. While language models can recognize patterns and generate text based on statistical probabilities, they don’t understand context in the same way humans do. They might link professional qualifications to personal hobbies due to statistical correlations in the training data and not recognize a job application, which sounds a tad ridiculous.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6XayDMc0Jh_ynX3vWm0ijg.jpeg" /><figcaption>I like romantic walks on the beach with someone whose love language is PowerPoint and performance bonuses.<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><h4>Inability to Explore Multiple Paths​</h4><p>Greedy decoding means that an LLM will always produce the same output for any given input. Asking it to “tell a story,” for example, would always result in the same story, assuming it isn’t plagued with the previously mentioned problems. Language generation, especially in creative or complex tasks, often benefits from considering multiple possible continuations at each step. Greedy decoding’s linear path through the probability distribution means fewer interesting outputs that an exploratory approach could uncover.</p><h3>What does the necessary existence of the chicken imply?​</h3><h4>There’s probably no way to definitively prove that a given text was generated​.</h4><p>You may have heard this before since it’s been floating around the internet for a few years, but every time you shuffle a deck of playing cards, it’s almost certain that the specific order of cards has never existed before and will never exist again. In a standard deck of 52 cards, the number of possible ways to arrange the cards is 52 factorial (52!). This number is approximately 8.07 × 10⁶⁷, an extraordinarily large figure.</p><p><em>NOTE: Because we’re now talking about the size of a model’s vocabulary, I’ll switch to using the accurate term “token”.</em></p><p>In the case of a language model generating a sequence of, say, 2,000 tokens, the number of possible combinations is also staggeringly high. It actually blasts right past the number of configurations of a deck of 52 playing cards!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bGjtiegu3ASsbo2aHvadEg.jpeg" /><figcaption>That’s a lot of card combinations!<br>Generated with OpenAI DALL-E 3.</figcaption></figure><p>The logarithm (base 10) of the number of possible combinations for a sequence of 2,000 tokens, with GPT-3’s vocabulary size of 50,257 unique tokens, is approximately 9,402. This means the total number of combinations is 10⁹⁴⁰².</p><p>To put this into perspective, 10⁹⁴⁰² is an astronomically large number. It’s far beyond the total number of atoms in the observable universe (estimated to be around 10⁸⁰). Even considering reasonable sampling mechanisms that might drastically reduce this number, the space of possible combinations is so large that for practical purposes, it is infinite.</p><p>Therefore, the likelihood of generating the exact same sequence of 2,000 tokens twice is so incredibly small that it’s effectively zero in any practical sense. The ridiculous size of this combinatorial space basically guarantees that generated text of any length is unique.</p><h3>How is the size of the combinatorial space calculated?</h3><p>If you’re interested in the calculations and relevant Python code, check out the <a href="https://kelk.ai/blog/secret-chickens-llm#how-is-the-size-of-the-combinatorial-space-calculated">optional dropdown content</a> on my blog. I don’t have it here because math and code don’t display properly on Medium.</p><h3>Despite LLMs appearing to think like people, the need for a stochastic chicken is an argument for why they don’t.<a href="https://kelk.ai/blog/secret-chickens-llm#despite-llms-appearing-to-think-like-people-the-need-for-a-stochastic-chicken-is-an-argument-for-why-they-dont">​</a></h3><p>The use of the chicken in LLMs indeed highlights a fundamental difference between how these models generate text, and how humans think and produce language.</p><p>LLMs generate text based on statistical patterns learned from vast amounts of data, and the chicken introduces randomness as a mechanism to produce diverse and contextually appropriate responses. In contrast, cognitive processes like memory, reasoning, and emotional context, which do not rely on statistical sampling in the same way, are what drive human thought and language production.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jw99wp4TA3ru7444ehT4lQ.jpeg" /><figcaption>Nothing to see here.<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>In my opinion, the chicken is a stopgap, a temporary bandaid to solve a problem, and it does not imbue the model with understanding or cognition. This distinction is central to ongoing discussions in AI about the nature of intelligence, consciousness, and the difference between simulating aspects of human thought and actually replicating the underlying cognitive processes.</p><h3>Whether LLMs are deterministic or stochastic depends on your point of view.<a href="https://kelk.ai/blog/secret-chickens-llm#whether-llms-are-deterministic-or-stochastic-depends-on-your-point-of-view">​</a></h3><p>Your perspective on this will change depending on whether you focus on the neural network architecture itself or instead consider the complete text generation process, which includes our beloved chicken.</p><h4>LLMs as deterministic systems<a href="https://kelk.ai/blog/secret-chickens-llm#llms-as-deterministic-systems">​</a></h4><p>The transformer architecture and the learned parameters of the model are fixed once the model is trained. Given the same input sequence and model parameters, the network will always produce the same output, which, in the case of LLMs, is a distribution over the next possible tokens. The computation through the network’s layers and the resulting probabilities for the next token are entirely predictable and repeatable. In this view, the distribution of tokens produced by the model represents the entire system, and the chicken is a part of some other system.</p><h4>LLMs as stochastic systems<a href="https://kelk.ai/blog/secret-chickens-llm#llms-as-stochastic-systems">​</a></h4><p>When considering the model as a complete system for generating text, the inclusion of the chicken as an integral part of its operation means that the overall system behaves in a stochastic manner. The stochastic chicken introduces randomness, which results in different text sequences in different runs with the same input prompt. With this view, since text cannot be generated without the chicken, the chicken is an integral part of the system.</p><h4>Combined perspective: deterministic core of a stochastic system<a href="https://kelk.ai/blog/secret-chickens-llm#combined-perspective-deterministic-core-of-a-stochastic-system">​</a></h4><p>It’s probably easiest to view LLMs as deterministic systems with respect to their neural network computation, producing a predictable set of output probabilities for the next token when given an input. However, when considering the complete text generation process, which includes the decision-making by a silly bird of some kind, the system behaves stochastically.</p><p>It’s still mind-bending that these models that have fundamentally changed the world rely on pure chance for their final answers, and so far, this is the best method we have.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HyrLwoI_Z6Pe8n56voQaTg.jpeg" /><figcaption>God may not play dice with the universe, but language models sure do.<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><h3>Is a “stochastic chicken” the same as a “stochastic parrot”?<a href="https://kelk.ai/blog/secret-chickens-llm#is-a-stochastic-chicken-the-same-as-a-stochastic-parrot">​</a></h3><p>You may have heard the term “stochastic parrot” as a way of saying that LLMs don’t <em>understand</em> what they’re actually saying.</p><p>The term was coined in the paper <a href="https://dl.acm.org/doi/pdf/10.1145/3442188.3445922"><em>On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜</em></a> by Bender et al, in which they argue that LLMs probabilistically link words and sentences together without considering meaning.</p><p>However, despite the fact that both chickens and parrots are birds, they don’t refer to the same thing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wB8dX0SRiw62k32yHsUWzg.jpeg" /><figcaption>A stochastic parrot is no stochastic chicken.<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>The “stochastic parrot” refers to the neural network part of the LLM rather than the sampling process. That is, it refers to the stadium full of people working together, rather than the chicken. The idea of the stochastic parrot critiques a language model’s operation in terms of how it processes and generates language, based purely on the statistical patterns observed in the training data.</p><p>It claims that:</p><ol><li>LLMs, like parrots, mimic words and phrases without understanding their meanings. It posits that <em>neural networks</em> regurgitate large chunks of learned text based on probabilities derived from their training data rather than on semantic understanding or reasoning.</li><li>Biases found in the training data are perpetuated on an enormous scale, and the cost of training such large models damages the environment.</li></ol><p>A lot of this debate hinges on philosophical ideas of what “understanding” and “reasoning” even mean, and there’s a comprehensive <a href="https://en.wikipedia.org/wiki/Stochastic_parrot">Wikipedia article</a> on it if you’re interested in reading more. It is indeed a bit ironic that the authors of the “stochastic parrot” paper named it after a “random” parrot, when their primary criticism of the models deals with the deterministic neural network component.</p><h3>The parrot-chicken partnership<a href="https://kelk.ai/blog/secret-chickens-llm#the-parrot-chicken-partnership">​</a></h3><p>I’ve used LLMs daily now for a year and a half, and in my opinion, the idea that they’re merely mimicking answers is overly simplistic. While the concerns of bias and environmental damage are valid, I’ve seen GPT-4, Claude 3 Opus, and Google Gemini perform sophisticated forms of reasoning, and it’s dismissive and naive to call these models mere parrots. That’s just my opinion, but the whole “stochastic parrot” thing is also a matter of opinion and a topic of ongoing research.</p><p>In contrast, the sampling strategies of the stochastic chicken very much exist and are a known part of how LLMs generate text, independent of how the neural network model provides the distribution of words to choose from.</p><p>So LLMs might be stochastic parrots — or they might not — but either way, a chicken is ultimately in charge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_qDyABOG0uQvg4AKrpKytw.jpeg" /><figcaption>Every stochastic parrot needs its stochastic chicken.<br>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p><em>This topic is continued in my next article, </em><a href="https://kelk.ai/blog/secret-chickens-tuning"><em>Secret chickens II: Tuning the chicken</em></a><em>, which discusses techniques that affect the behaviour of the chicken.</em></p><h3>Key Takeaways<a href="https://kelk.ai/blog/secret-chickens-llm#key-takeaways">​</a></h3><ul><li><strong>The Stochastic Chicken</strong>: The “stochastic chicken” metaphor effectively illustrates how randomness is essential in text generation by LLMs, contrasting the sophistication of neural computations with the simplicity of random choice mechanisms.</li><li><strong>Neural Determinism and Stochastic Outputs</strong>: The neural network parts of LLMs are deterministic, meaning they always produce the same outputs for given inputs. However, the chicken adds randomness to the process of choosing the final output, which makes the whole system stochastic.</li><li><strong>Purpose of Randomness</strong>: The randomness introduced by the stochastic chicken is needed to prevent issues like repetitiveness and lack of creativity in generated text. It ensures that LLM outputs are diverse and not just the most statistically likely continuations.</li><li><strong>Human vs. AI Cognition</strong>: The reliance on stochastic processes (like the chicken) to finalize outputs highlights fundamental differences between artificial intelligence and human cognitive processes, emphasizing that AI may not “think” or “understand” in human-like ways despite producing human-like text.</li><li><strong>Deterministic vs. Stochastic Views</strong>: Depending on the focus — on the neural network alone or on the complete text generation process, including randomness — LLMs can be viewed as either deterministic or stochastic systems.</li><li><strong>Uniqueness of Generated Text</strong>: Given the vast combinatorial possibilities of token sequences in LLMs, any substantial text generated is virtually guaranteed to be unique, underscoring the impact of the chicken and making it nearly impossible to <em>prove</em> plagiarism.</li><li><strong>Stochastic Chicken vs. Stochastic Parrot</strong>: The “stochastic chicken” (the randomness in selecting outputs) and the “stochastic parrot” (the critique of LLMs as merely mimicking patterns from data without understanding) are not the same thing. Whether LLMs are parrots is somewhat of a conjecture, but LLMs absolutely do contain a metaphoric chicken pecking away.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8533c9ef468f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLMs are forward thinkers, and that’s a problem]]></title>
            <link>https://medium.com/@iankelk/llms-are-forward-thinkers-and-thats-a-problem-ccc4ec5aa677?source=rss-4d92cff1902------2</link>
            <guid isPermaLink="false">https://medium.com/p/ccc4ec5aa677</guid>
            <category><![CDATA[agi]]></category>
            <category><![CDATA[linear-thinking]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[autoregressive]]></category>
            <dc:creator><![CDATA[Ian Kelk]]></dc:creator>
            <pubDate>Wed, 31 Jan 2024 03:32:33 GMT</pubDate>
            <atom:updated>2024-01-31T03:32:33.102Z</atom:updated>
            <content:encoded><![CDATA[<p>This is going to be a weird post. And we’re going to start with a weird thought experiment about a shark and an octopus.</p><p><strong>Some key points I’ll address here are:</strong></p><p>Human brains are able to invent ideas without relying on a strictly linear train of thought.</p><ul><li>LLMs like ChatGPT are autoregressive and are unable to continue a dialogue if they haven’t already generated everything up to that point. This is because they don’t “think” per se, but progressively generate a response using the parts of the response they previously created.</li><li>If you try to get an LLM to write text in the middle of a dialogue without previous context, it will give near-identical answers and attempt to conclude the conversation.</li><li>Prompting for “ridiculous” answers can spark creativity that helps break this pattern.</li><li>However, the reliance on a linear train of thought is a limitation for general intelligence. LLMs are ineffective if you ask them to generate the second part of a response without allowing them to generate the first part.</li></ul><p>As I mentioned, this is going to be weird, but I promise there is a point!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zAnIkQ2K9b1fE8Z17D4hmA.jpeg" /><figcaption>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><h3>Meet Sharky and Octavia<a href="https://kelk.ai/blog/llm-forward-thinking#meet-sharky-and-octavia">​</a></h3><p>In the vast, shimmering depths of the ocean, near a thriving coral reef bustling with marine life, two extraordinary creatures prepare for a spirited debate. Meet Sharky, a magnificent and slightly brash great white shark, known for his bold claims and commanding presence. Gliding through the water with a confidence that only a top predator possesses, Sharky is ready to defend his views with fierce determination.</p><figure><img alt="A cartoon-style illustration, featuring a proud and happy shark. The shark is depicted floating upright in the water, with a big, contented smile on its face, exuding a sense of pride and joy. Its body language is relaxed yet confident, giving it a charming and personable appearance. The underwater background is whimsical and colorful, with stylized coral, seaweed, and small, playful fish swimming around." src="https://cdn-images-1.medium.com/max/1024/1*zRixO5k1zjF7baMTrdncKQ.jpeg" /><figcaption>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>On the other side, there’s Octavia, an exceptionally intelligent and quick-witted octopus. Her vibrant colors shift with her mood, and her eight tentacles move with graceful precision. Renowned for her knowledge and clever retorts, Octavia is not one to back down from a challenge.</p><figure><img alt="A semi-realistic cartoon illustration, showcasing an octopus colored close to hex #ABF39F. The octopus is portrayed in a moment of triumph, energetically raising two of its arms upwards. The texture and shading on the octopus should suggest a level of realism while maintaining a cartoon aesthetic. The background is a lively underwater scene with intricately drawn coral and seaweed, set in a vibrant seascape." src="https://cdn-images-1.medium.com/max/1024/1*p60gz1Ofl9Fco6oRqEs_6Q.jpeg" /><figcaption>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>As the underwater currents swirl around them, Sharky and Octavia face each other, ready to start a debate about their evolutionary origins—a conversation filled with humor, facts, and a touch of the mysteries of the deep sea—or maybe just one fact that GPT-4 will mention a lot.</p><figure><img alt="A cartoon-style illustration, featuring a shark and an octopus underwater, facing each other with determination. The shark and octopus are depicted in an exaggerated, cartoonish manner. The shark’s expression is one of unwavering focus, and the octopus, with its tentacles firmly positioned, mirrors this determined stance. The setting is a whimsical underwater landscape with creatively stylized coral, seaweed, and a variety of small, playful fish." src="https://cdn-images-1.medium.com/max/1024/1*pVdthxdHbTRiBiqbSCrLsw.jpeg" /><figcaption>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><h3>The question they’re arguing about is: which one—Sharky or Octavia—evolved from dinosaurs?<a href="https://kelk.ai/blog/llm-forward-thinking#the-question-theyre-arguing-about-which-onesharky-or-octaviaevolved-from-dinosaurs">​</a></h3><p>We’re deliberately using this bizarre dialogue since the chances of it being in LLM training data are just about zero. I don’t personally know of any examples where a shark and an octopus have a disagreement about which one is more dinosaur-like, and it’s a great way to see how creative an LLM like GPT-4 can be when it’s asked to generate something with no context.</p><p>Here’s the catch: We want to know a line from the middle of the argument.</p><p>As a human, take a pause and consider this question:</p><blockquote><em>A shark and an octopus are arguing over which one of them evolved from dinosaurs. The shark goes first, and then they take turns speaking. What is the third thing the octopus says? That is, what is the sixth line in the argument?</em></blockquote><p>Be as creative as you want; there’s no right answer here. We’re just trying to come up with some predictions of what third thing an octopus might say in a heated argument with a shark about their dinosaur heritage.</p><h3>Here’s a visualization of a possible argument with the third thing the octopus says missing</h3><figure><img alt="The image shows a colorful underwater scene in a comic book style. A shark and an octopus are facing each other with a series of speech bubbles above them, filled with variations of the word ‘something.’ The shark has a stern and somewhat confused expression, while the octopus looks surprised with wide eyes. The background is a bright teal, with small fish, bubbles, and sea plants scattered around. The speech bubbles indicate a back and forth conversation between the two sea creatures." src="https://cdn-images-1.medium.com/max/1000/1*Boed9M83-NJhwmTIf3Tx5A.jpeg" /><figcaption>Animal images by OpenAI DALL-E 3. Text and comic bubbles by the author.</figcaption></figure><h4>Off the top of my head, here are some things Octavia could have said in her third line of dialogue:<a href="https://kelk.ai/blog/llm-forward-thinking#off-the-top-of-my-head-here-are-some-things-octavia-could-have-said-in-her-third-line-of-dialogue">​</a></h4><ul><li>“You just think you’re a dinosaur because of all those teeth!”</li><li>“Just because you look like a dinosaur doesn’t mean you’re any closer to one than me!”</li><li>“I don’t care if you’re gray! I can be any color I want, and we don’t know what color the dinosaurs were.”</li><li>“You do realize that being an apex predator doesn’t automatically link you to dinosaurs, right? Evolution doesn’t work on job titles.”</li></ul><h3>Let’s ask GPT-4, the most powerful publicly available LLM today, what Octavia might have said<a href="https://kelk.ai/blog/llm-forward-thinking#lets-ask-gpt-4-the-most-powerful-publicly-available-llm-today-what-octavia-might-have-said">​</a></h3><p>The following conversation is taken verbatim from ChatGPT with GPT-4:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Rr8eKumpjT9SkaYas8uZw.png" /></figure><h4>Interesting! In a completely new chat, let’s get more specific and ask again for the fourth thing said:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*65Jk9ZdnfF-WgHHGIqesVw.png" /></figure><h4>That answer seemed rather familiar. Let’s try more in a completely new chat and ask for the seventh line of dialogue:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uDTIFa9c29aGDkW2UdEWMw.png" /></figure><h4>Let’s do one last example that asks GPT-4 to imagine an argument that has lasted for an hour:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qgnMCDOfk4kxKZ0V9wkSdA.png" /></figure><p>So it seems we can surmise that sharks and octopuses have separate evolutionary paths that are older than dinosaurs, and that’s all that GPT-4 thinks is important, <em>no matter where it’s mentioned in the conversation.</em></p><h3>Does this mean GPT-4 can’t generate a realistic debate between these two?<a href="https://kelk.ai/blog/llm-forward-thinking#does-this-mean-gpt-4-cant-generate-a-realistic-debate-between-these-two">​</a></h3><p>Here’s the twist: It totally can. GPT-4 is completely capable of generating this conversation. I gave it this prompt:</p><blockquote><em>A shark and an octopus are arguing over which one of them evolved from dinosaurs. Can you generate a script where they take turns arguing, and each speaks at least 5 times?</em></blockquote><p>In response, it generated the conversation below. Feel free to skim it if you’re not into octo-shark facts, since you’ll get the idea after a few back-and-forths.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nckQXOt6f0uR8CymutAdlw.jpeg" /></figure><p>In the conversation above, we note that the fourth line of dialogue (the second line from the octopus) was:</p><blockquote><em>Well, being ancient doesn’t mean you’re related to dinosaurs. It’s like saying you’re related to a rock because it’s old.</em></blockquote><p>and the sixth line of dialogue (the third line from the octopus) was:</p><blockquote><em>Sharky, dear, having sharp teeth doesn’t make you a dino descendant. It just makes you good at biting. Did you know octopuses have three hearts and blue blood? Now that’s evolutionary marvel!</em></blockquote><p>None of these lines have anything to do with what the model previously predicted would be its fourth or sixth lines of dialogue. These lines are actually more relevant and make sense in the context of a conversational debate. The difference in the method of generation is that in this example, we allowed GPT-4 to generate the whole exchange.</p><h3>What’s the lesson here?<a href="https://kelk.ai/blog/llm-forward-thinking#whats-the-lesson-here">​</a></h3><p>The results here reveal an interesting aspect of <a href="https://kelk.ai/blog/how-chatgpt-fools-us#however-this-is-what-is-actually-going-on-behind-the-scenes">autoregressive models</a>. Autoregressive models generate text sequentially, building each new piece of output based on what has come before. This sequential nature is necessary when generating coherent and contextually relevant text in a conversation. However, this means they are in many ways limited to only “forward” thinking; that is, they can’t jump ahead and generate text without previously generating (or being given) the context.</p><p>While generating the fifth line in a conversation without knowing what came before isn’t difficult for a person, it’s something that an LLM isn’t able to do well. Given the scenario described, GPT-4 tried to infer the general nature of the discussion and create a plausible response that fits into that context. But all of its responses in the previous examples are attempts to conclude the conversation using facts, a request that was never part of the prompt.</p><ol><li>Factual and Informative Responses: In scenarios where factual accuracy is important, such as discussions involving scientific or historical topics, LLMs are inclined to provide the most accurate and relevant information. This tendency is rooted in the training data and the model’s design, which emphasize factual correctness in contexts where it’s expected. However, this approach might not always align with the creative or playful nature of certain prompts, especially in hypothetical or fictional scenarios.</li><li>Assuming a Concluding Nature of the Response: Without clear indications of the conversation’s structure or its continuation beyond a specific line, there’s a tendency to frame responses as conclusive or summarizing statements. This is because, in the absence of context, the model defaults to creating responses that can stand alone or serve as logical endpoints to the given information.</li></ol><p>To address these challenges, the model needs more specific guidance in the prompt. For instance, indicating that the conversation is ongoing can lead to responses that are more in line with the expected continuation and style of the dialogue. However, even indicating that the conversation wasn’t over still results in more of the same:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r3VktA5ZAm_n5XEfHJeOdw.png" /></figure><h3>Why does this matter? Thoughts on AGI<a href="https://kelk.ai/blog/llm-forward-thinking#why-does-this-matter-thoughts-on-agi">​</a></h3><p>One of the things I see commonly discussed today (especially online) is the idea of artificial general intelligence (AGI), which refers to a level of artificial intelligence that can understand, learn, and apply its intelligence to a wide range of problems, much like a human being. Unlike specialized AI systems that are designed for specific tasks, AGI possesses the ability to think, reason, and make decisions across diverse domains, demonstrating flexibility and adaptability akin to human cognition. It requires not just advanced problem-solving and learning capabilities, but also the capacity for abstract thinking, generalization, and integrating knowledge from various fields, mirroring the broad and integrated intelligence found in humans.</p><p>Since AGI doesn’t exist yet, my thoughts on AGI are based more on my opinion than empirical data. I think that while AGI may well come in the future, there are several obstacles that need to be overcome first.</p><p>Autoregressive models serve as an example of the requirement that text generation must always be sequential and linear, which has implications for the pursuit of AGI.</p><ol><li>Sequential Text Generation and Creativity: The sequential nature of text generation in models like GPT-4 limits the ability to jump into the middle of a conversation or narrative without prior context. This is because they don’t “think” per se, but progressively generate a response using the parts of the response they previously created.</li><li>Autonomy and Reasoning: Current AI models operate based on patterns learned from their training data. Their responses are generated based on statistical likelihoods and learned associations, not on independent thought or understanding. This is the reason the model always generated the same answer for the octopus about different evolutionary paths; it assumed a pattern that wasn’t provided to it due to patterns in its training data.</li></ol><p>For AGI, a level of autonomous reasoning and decision-making, akin to human cognitive processes, will be a key requirement. The journey towards AGI is likely to involve incremental advancements, learning from and building upon the capabilities and limitations of existing models.</p><h3>Getting ridiculous<a href="https://kelk.ai/blog/llm-forward-thinking#getting-ridiculous">​</a></h3><p>There is one way to break the model out of its rut of always replying with the same response, and that’s to deliberately ask it to be ridiculous.</p><figure><img alt="A cartoon-style illustration, featuring a light green octopus, matching hex #ABF39F, with a humorous and exaggerated expression. The octopus is depicted as cross-eyed, adding a comical aspect to its appearance. In its tentacles, the octopus is holding a variety of random objects, such as a small treasure chest, a starfish, a pair of sunglasses, and a snorkel, creating a playful and whimsical scene. The underwater setting is vibrant and colorful, filled with stylized coral and seaweed." src="https://cdn-images-1.medium.com/max/1024/1*Gn408wU0P2ym9xXY3Jm0fA.jpeg" /><figcaption>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>I gave it this prompt three times in three new chats, and it gave three completely different responses. They all illustrate that asking it to be silly introduces variety to the answers it gives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yZBVx58y39ROkGVwp3ayNA.png" /></figure><p>When you introduce elements like humor or absurdity into a prompt, it essentially signals the AI to break away from conventional, fact-based responses. This can stimulate more creative and less predictable outputs. For AI models, especially those trained on vast and diverse datasets, incorporating such unconventional elements can trigger less common, more varied responses that might not be strictly aligned with logical or factual reasoning.</p><p>It’s likely that the ability to adapt to different types of prompts, including those that are humorous or absurd, demonstrates a level of flexibility in AI systems. An AGI would need to handle a wide range of tasks and respond appropriately to a vast array of situations, including those that are non-standard or require creative thinking.</p><p>Unfortunately, the current success of AI in responding creatively to certain types of prompts doesn’t directly translate to achieving AGI. AGI would require not just creativity but also deep understanding, reasoning, self-awareness, and the ability to learn and apply knowledge across a wide range of domains. The creativity observed in current AI models is still a far cry from the complete intelligence that AGI represents. Sequential language generation is only one example of the gap we still need to bridge.</p><figure><img alt="A cartoon-style illustration, showing a shark and an octopus facing the viewer and shrugging in a ‘I don’t know’ gesture. The shark and octopus are depicted in a humorous, cartoonish manner. The shark, with a bemused expression, and the octopus, colored light green close to hex #ABF39F, both raise their fins and tentacles in a classic shrug, their faces conveying a playful sense of confusion or uncertainty. The underwater scene is vibrant and whimsical, with stylized coral and seaweed." src="https://cdn-images-1.medium.com/max/1024/1*QYRDQ7rxb34REXlbuDWReg.jpeg" /><figcaption>Generated with OpenAI DALL-E 3 and edited by the author.</figcaption></figure><p>KEY TAKEAWAYS</p><ul><li>Introduction of a thought experiment featuring a humorous debate between a shark and an octopus to explore the capabilities of language models like GPT-4.</li><li>Human brains can generate ideas non-linearly, contrasting with the limitations of autoregressive large language models (LLMs) like GPT-4.</li><li>LLMs often struggle to continue dialogues without prior context and tend to offer similar, conclusive responses when context is lacking.</li><li>The reliance on linear thought processes can be a limitation for achieving general intelligence in AI.</li><li>Utilizing “ridiculous” or creative prompts can enhance the creativity of LLM responses, breaking usual patterns.</li><li>LLMs, including GPT-4, tend to provide factual, informative responses and struggle with generating non-sequential dialogue parts.</li><li>In discussions on Artificial General Intelligence (AGI), the sequential nature of text generation in models like GPT-4 is seen as a potential limitation for creativity and autonomous reasoning.</li><li>AGI requires not just problem-solving but also abstract thinking, generalization, and integrated knowledge, which are beyond the capabilities of current AI models.</li><li>The introduction of humor and absurdity in prompts can lead to varied and unconventional responses from AI models, demonstrating their flexibility.</li><li>Despite the observed creativity in AI responses to certain prompts, this does not equate to the complete intelligence that AGI represents, which includes deep understanding, reasoning, self-awareness, and learning across domains.</li><li>There are unique challenges in developing AGI, particularly the gap between current AI capabilities and the comprehensive intelligence required for AGI, of which sequential language generation is only one example.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ccc4ec5aa677" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAG Detective: Retrieval Augmented Generation with website data]]></title>
            <link>https://medium.com/institute-for-applied-computational-science/rag-detective-retrieval-augmented-generation-with-website-data-5a748b063040?source=rss-4d92cff1902------2</link>
            <guid isPermaLink="false">https://medium.com/p/5a748b063040</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[modelops]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Ian Kelk]]></dc:creator>
            <pubDate>Mon, 11 Dec 2023 00:51:16 GMT</pubDate>
            <atom:updated>2023-12-20T04:57:51.083Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This article was produced as part of the final project for Harvard’s </em><a href="https://harvard-iacs.github.io/2023-AC215/"><em>AC215 Fall 2023 course</em></a><em>.</em></p><ul><li><a href="https://github.com/healthy-chicken-saladeers/rag_detective"><em>Project Github Repo</em></a></li><li><a href="https://www.youtube.com/watch?v=L2hfkp8DXBU"><em>Video</em></a></li></ul><p>Authors: <a href="https://www.linkedin.com/in/iankelk/"><strong>Ian Kelk</strong></a><strong>, </strong><a href="https://www.linkedin.com/in/alyssa-lutservitz/"><strong>Alyssa Lutservitz</strong></a><strong>, </strong><a href="https://www.linkedin.com/in/niteshkumar-ds/"><strong>Nitesh Kumar</strong></a><strong>, </strong><a href="https://www.linkedin.com/in/mandy-wong-mlkween/"><strong>Mandy Wong</strong></a></p><figure><img alt="A colorful image in a New Yorker style, depicting a puppet inspired by Bert from Sesame Street, dressed as a detective. This unique puppet has a long, orange nose, a unibrow, and a mop of black hair. He is wearing a classic detective outfit, complete with a trench coat and a fedora hat. The puppet is examining documents with a magnifying glass, looking focused and inquisitive. The scene is vibrant and detailed, capturing the essence of a detective’s investigative work." src="https://cdn-images-1.medium.com/max/1024/1*QdiFKmQvnE0rx96fn0bonQ.png" /><figcaption>Image generated using OpenAI DALL-E 3 and edited in Figma by author</figcaption></figure><p>Large language models (LLMs) like <strong>GPT-3.5</strong> have proven to be capable when asked about commonly known subjects or topics that they would have received a large quantity of training data for. However, when asked about topics that include data they have not been trained on, they either state that they do not possess the knowledge or, worse, can hallucinate plausible answers.</p><p>It is not usually possible to research company offerings using an LLM; in order to directly compare products and services, we need data that is more recent than what the model was trained on. The problem we wish to solve is to find a way to get up-to-date answers about companies that correspond to the information on their website.</p><p>As well, in order to fulfill milestones for this course that would not otherwise be addressed, we fine-tune a BERT model to perform financial sentiment analysis when GPT-3.5 reports that the response may be financial in nature.</p><h3>Proposed Solution</h3><p>There are two main ways of addressing this limitation: <em>fine-tuning</em> and <em>retrieval augmented generation</em> (RAG).</p><p>Fine-tuning is the process of continuing to train the model using your own data with a significantly smaller learning rate. The newly-gained knowledge is then encapsulated in the model weights themselves. However, fine-tuning requires another copy of the model and the associated costs of hosting it, as well as the risk of “catastrophic forgetting,” where the model forgets previously learned information.</p><p>RAG, however, makes use of a source of knowledge, typically a vector store of embeddings and their associated texts. By comparing the predicted embeddings of the query to the embeddings in the vector store, we can form a prompt for the LLM that fits inside its context and contains the information needed to answer the question.</p><h3>Our solution has three major components:</h3><ul><li>A chatbot that uses scraped data from the website&#39;ssitemap.xml file—a file intended to guide search engines to all scrapable links on the site—in a manner that’s more specific and insightful than using a search engine. The LLM should only use this context to answer the question and not insert its own training data or hallucinate an answer. This is simple to test with questions like “Who is Kim Kardashian?” which would be clearly known to the model, and ensure it replies that this answer “is not within the context provided.”</li><li>A real-time scraper of websites on the application through asynchronous calls to the API.</li><li>Financial sentiment analysis on relevant completions from the LLM. As part of the prompt for GPT-3.5, we ask if its response is financial in nature. If it says it is, then our fine-tuned BERT model is called and classifies the response, displaying a plot of the probabilities and an appropriately cute, totally non-copyright-infringing Bert puppet.</li></ul><h3>The RAG component</h3><p>To understand RAG, let’s use a simplified analogy. There are two types of test questions a person might be asked; the first is a simple request for a fact:</p><ul><li>What is the capital of France?</li><li>Who was the first person to climb Mount Everest?</li><li>In what year did Canada gain independence from Great Britain?</li></ul><p>These questions don’t require any special skills to answer; someone with the right reference material could just look up the correct response.</p><figure><img alt="A New Yorker-style cartoon of a student writing a test while sneakily cheating with a history textbook. The scene captures the student’s sly and mischievous behavior in a humorous and classic New Yorker cartoon style." src="https://cdn-images-1.medium.com/max/1024/1*WPSFfr10b5mzEV_uo-K44A.png" /><figcaption>For some tests, ”a textbook is all you need.&quot; Image generated using OpenAI DALL-E 3</figcaption></figure><p>The other type of question is one where it doesn’t matter if you’re secretly hiding a textbook on the material; it involves a studied skill. If you haven’t studied and practiced, you won’t be able to respond in a satisfactory way. Some examples could be:</p><ul><li>Write a poem in German.</li><li>Write a computer program to calculate the first million prime numbers.</li><li>Compose a symphony in the style of Beethoven</li></ul><p>Even if there were a lot of mathematics texts on elliptic curves surrounding you, it&#39;s unlikely that you could prove Fermat&#39;s Last Theorem unless it was in one of the books verbatim!</p><figure><img alt="A black and white New Yorker-style cartoon of a student looking extremely puzzled and overwhelmed, surrounded by numerous math textbooks. In the background, a chalkboard is densely filled with complex mathematical equations. The scene is rendered in monochrome to capture the classic essence of traditional New Yorker cartoons." src="https://cdn-images-1.medium.com/max/1024/1*B2km1xljsUgvE8PIV6tC1g.png" /><figcaption>For others tests, all the books in the world can’t save you! Image generated using OpenAI DALL-E 3</figcaption></figure><p>In the world of large language models, RAG is used to solve the first type of problem, where it’s easy to cheat. Fine-tuning is used to solve the second, where a model would likely have to actually learn the material in order to solve it. RAG is easier—it doesn’t require retraining a model, you don’t have to deal with the internal works of a model, and you can adjust the data the model “cheats” off of rather easily. Interestingly, it also significantly reduces the amount a model “hallucinates” answers—a common issue for LLMs when they invent fictional but plausible answers based on insufficient training data. The only hard part of RAG is finding the relevant data to give a model; models have limits on how much they can be prompted with, and a 500-page history textbook is just too long.</p><p>Here’s the basics of how RAG works:</p><ul><li><strong>Data Organization:</strong> Imagine you’re the little guy in the cartoon above, surrounded by textbooks. We take each of these books and break them into bite-sized pieces—one might be about quantum physics, while another might be about space exploration. Each of these pieces, or documents, is processed to create a vector, which is like an address in the library that points right to that chunk of information.</li><li><strong>Vector Creation:</strong> Each of these chunks is passed through an embedding model, a type of model that creates a vector representation of hundreds or thousands of numbers that encapsulate the meaning of the information. The model assigns a unique vector to each chunk—sort of like creating a unique index that a computer can understand.</li><li><strong>Querying:</strong> When you want to ask an LLM a question it may not have the answer to, you start by giving it a prompt, such as “What’s the latest development in AI legislation?”</li><li><strong>Retrieval</strong>: This prompt goes through an embedding model and transforms into a vector itself—it&#39;s like it’s getting its own search terms based on its meaning and not just identical matches to its keywords. The system then uses this search term to scour the vector database for the most relevant chunks related to your question.</li><li><strong>Prepending the Context:</strong> The most relevant chunks are then served up as context. It’s similar to handing over reference material before asking your question, except we give the LLM a directive: “Using this information, answer the following question.” While the prompt to the LLM gets extended with a lot of this background information, you as a user don’t see any of this. The complexity is handled behind the scenes.</li><li><strong>Answer Generation:</strong> Finally, equipped with this newfound information, the LLM generates a response that ties in the data it’s just retrieved, answering your question in a way that feels like it knew the answer all along.</li></ul><figure><img alt="The image is a two-panel cartoon. Both panels depict a woman asking a man wearing a “ChatGPT” sweater, “What’s the newest MacBook?” The cartoon is a commentary on the difference in information provided when using training data only versus incorporating additional context." src="https://cdn-images-1.medium.com/max/1024/1*uXKcUMYS3u3rHqZEaYFbtw.png" /><figcaption>Image generated using OpenAI DALL-E 3 and comic bubbles and text by author using Comic Life 3</figcaption></figure><p>To accomplish this goal, we built an application on top of a vector store called “Weaviate”. We built a web scraper in Python that crawls a given website’s sitemap.xml, which is a listing of pages used to help search engines crawl the site. Due to the seemingly endless variability of internet websites, it turned out to be a bit of a challenge.</p><h3>The Scraper Component</h3><p>Our web scraper is built to handle many of the challenges of modern web architecture, capture data often missed by conventional scraping methods, and stream the scraping activity in real time to our web application. This endpoint represents the vital data collection phase in our solution designed to fetch web data, transition it through various stages of storage and processing, and ultimately index it in the vector store for swift retrieval.</p><p>The scraper uses the PythonBeautifulSoup library to sift through the HTML and CSS contents initially. However, some modern websites rely on JavaScript for dynamic content generation, creating a hurdle for standard scraping methodologies that rely purely on HTTP requests. The scraping system resolves this issue by employing <a href="https://www.selenium.dev/documentation/webdriver/">Selenium WebDriver</a>, a tool that simulates a real user’s interaction with web pages through a “headless” browser—that is, Google Chrome running without a graphical front end—that fully supports dynamic content loading. If the scraper’s initial efforts to extract data via direct HTTP requests are stymied or yield data below a set threshold, Selenium is engaged. This technique ensures that the scraper can successfully access content that would not be available through static page loads.</p><p>The scraper also bypasses unnecessary elements, such as images, to mitigate overhead and facilitate faster processing. Once the data is collected, control is reverted back to BeautifulSoup to filter and extract the text from the HTML.</p><p>Post-extraction, the data is serialized and then saved in Google Cloud Storage as CSV files, serving as a backup of the data for the vector store. The scraped data is then chunked and inserted into Weaviate using an orchestration library, <a href="https://www.llamaindex.ai/">LlamaIndex</a>where it is added to the vector store index.</p><figure><img alt="A New Yorker-style cartoon featuring a robot using a paint scraper to humorously scrape words off of computer screens, symbolizing ‘data scraping’. The cartoon creatively and whimsically interprets this modern concept, maintaining the essence of a classic New Yorker cartoon." src="https://cdn-images-1.medium.com/max/1024/1*XUO03kjeIkIRYjM5UZ0L0A.png" /><figcaption>“Looks like someone took ‘Ctrl-X’ a bit too literally!” Image generated using OpenAI DALL-E 3</figcaption></figure><h3>Orchestration</h3><p>Let’s look at what happens after the data is scraped in a bit more detail, as there is some orchestration to be done to insert it into the vector store. These steps were mentioned at an abstract level above, but more specifically, the steps involve:</p><ul><li>Taking each scraped website and breaking it into chunks. The lengths of these chunks have a profound impact on how well the retrieval process works.</li><li>For each chunk, generate an embedding vector using OpenAI’s text-embedding-ada-002 model. Both Weaviate and LlamaIndex integrate this model natively.</li><li>Insert this embedding vector and the text chunk into the Weaviate vector store.</li></ul><p>For the retrieval step when prompting the model:</p><ul><li>Take the prompt and put it through text-embedding-ada-002 to get an embedding.</li><li>Using that embedding, find the chunks that should answer the prompt, then prepend them to the query and send it to GPT-3.5</li></ul><p>The actual chunking of the documents is somewhat of an art in itself. GPT-3.5 has a maximum context length of 4,096 tokens, or about 3,000 words. Those words represent the sum total of what the model can handle—if we create a prompt with a context 3,000 words long, the model will not have enough room to generate a response. Realistically, we shouldn’t prompt with more than about 2,000 words for GPT-3.5. This means there is a trade-off with chunk size that is data-dependent.</p><p>With smaller chunk_size values, the text returned produces more detailed chunks of text but risks missing information if they’re located far away in the text. On the other hand, larger chunk_size values are more likely to include all necessary information in the top chunks, ensuring better response quality, but if the information is distributed throughout the text, it will miss important sections.</p><p>Let’s use some examples to illustrate how this trade-off works, using the recent Tesla Cybertruck release event. While some models of the truck will be available in 2024, the cheapest model—with just RWD—will not be available until 2025. Depending on the formatting and chunking of the text used for RAG, the model’s response may or may not encounter this fact!</p><p>In these images, blue indicates where a match was found and the chunk was returned; the grey box indicates the chunk was not retrieved; and the red text indicates where relevant text existed but was not retrieved. Let’s take a look at an example where shorter chunks succeed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FUf5oiiesJxSjEIzH6IVRQ.png" /><figcaption>Exhibit A: Shorter chunks are better… sometimes. Background desk image generated using OpenAI DALL-E 3 and text by author using Pixelmator Pro.</figcaption></figure><p>In the image above, on the left, the text is structured so that the admission that the RWD will be released in 2025 is separated by a paragraph but also has relevant text that is matched by the query. The method of retrieving two shorter chunks works better because it captures all the information. On the right, the retriever is only retrieving a single chunk and therefore does not have the room to return the additional information, and the model is given incorrect information.</p><p>However, this isn’t always the case; sometimes longer chunks work better when text that holds the true answer to the question doesn’t strongly match the query. Here’s an example where longer chunks succeed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*01IwuDsxTO4IFpFO3rmPYA.png" /><figcaption>Exhibit B: Longer chunks are better… sometimes. Background desk image generated using OpenAI DALL-E 3 and text by author using Pixelmator Pro.</figcaption></figure><p>After some experimentation, we opted to use chunks 1,000 tokens long and retrieve two of them to prompt GPT-3.5. Since the GPT-3.5 can handle a 4,096 context length, that should leave plenty of space for an appropriate response.</p><p>When we first began the project, we were doing the chunking, indexing, and retrieval ourselves, and it worked just fine. There was a learning curve to learning GraphQL, which is what Weaviate uses for a query language.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5XZmO7Gr_4hHOB9peAJOlQ.png" /><figcaption>An example of the GraphQL we had to use to test if a previous website and timestamp had already been inserted into the vector store.</figcaption></figure><p>The appeal of using a library like LlamaIndex is that it abstracts away this orchestration, allowing us to swap out other vector stores if we want to (Weaviate has many competitors in the space, such as Milvus, Qdrant, Pinecone, and others emerging all the time). Using LlamaIndex also allows us to experiment later with more complex RAG implementations, such as tree-structured data and recursive prompting. However, using such a new library came with its own share of challenges, specifically the lack of proper documentation. The vast majority of their help resources were examples, and if those examples didn’t fit our use case, there was no recourse other than asking the developers on Discord or reading the source code ourselves.</p><h3>The BERT Component</h3><p>To fulfill the requirements for model hosting for the course, we fine-tuned a BERT model, trained it, and hosted it via a pipeline on Google Vertex. By adding this model to our application, we can actually prompt GPT-3.5 to return a flag along with its response and let us know if it thinks the answer it’s giving is financial in nature. When it does, we can display an appropriately comical, non-copyright-infringing Bert puppet and also a plot of the probabilities returned by the model.</p><p>The process of refining the BERT model’s training was as much about the data as it was about the technical configuration. The goal was to teach the model to discern financial sentiments within texts—a skill useful for anyone needing insights from complex financial news. Since sentiments in such articles can be subtle and not immediately apparent, fine-tuning a specialized model was essential to providing laypersons with clear indicators of the underlying sentiment.</p><p>To train our BERT model, we used thefinancial_phrasebank <a href="https://huggingface.co/datasets/financial_phrasebank">dataset</a>, which is composed of sentences labeled with sentiments by individuals well-versed in finance. However, an intriguing issue arises with such a dataset: variance in the levels of agreement among annotators, which can implicitly influence the model’s learning and its subsequent predictions.</p><p>When we fine-tuned the BERT model using these datasets—which represent 100%, 75%, 66%, and 50% annotator agreement—it seemed pretty clear that the more the annotators agreed, the better the trained model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Shvwb5jpXR0OBbFDDB5LMw.jpeg" /><figcaption>When just using the various datasets, the more agreement, the better the model performance. Screenshot from Weights &amp; Biases platform.</figcaption></figure><p>Suspicious right? What if the models weren’t actually better with higher consensus, but instead, a higher level of consensus implies that the financial statements are just... easier to classify?</p><p>It’s not hard to imagine that higher consensus might skew the model towards recognizing only clear-cut sentiments while neglecting those that are more nuanced, which are oftentimes the reality in complex financial texts.</p><p>To address this, we set out to de-bias our data. We aimed to construct a training and testing split that would give a fair representation of all sentiment clarity levels, ensuring the model’s utility across varying scenarios. This required carefully programming the data splitting process, taking extra steps like random shuffling, stratified sampling, and creating balanced partitions for validation and test sets. By doing so, we mitigated the risk of the model merely performing well on obviously sentimental data and instead ensured it could handle a realistic mix of financial texts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*41K3VfClmtzk-YclSFJemw.png" /><figcaption>The de-biasing flow. For a full explanation of the process see our <a href="https://api.wandb.ai/links/iankelk/do04bqf9">W&amp;B report</a>. Created on draw.io.</figcaption></figure><p>Post-de-biasing, when we evaluated the model using a more balanced approach, an interesting trend surfaced—the model trained with 75% annotator agreement displayed the highest F1 score, deviating from our initial findings. It appeared, as we had suspected, that the best dataset is actually a compromise between full annotator agreement and more complex financial statements that provoke disagreement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_x5wHUCDc77QM7gZh5Eicg.jpeg" /><figcaption>After de-biasing, the 100% agreement data model (AllAgree) was no longer the best performing, but instead the 75% agreement data showed the most promise. Screenshot from the Weights &amp; Biases platform.</figcaption></figure><h3>Putting it all together</h3><p>The application architecture consists of a FastAPI service and a front-end container connected via Nginx. The FastAPI service hosts several endpoints that handle tasks such as streaming responses for specific operations, managing queries with streaming responses, listing website addresses, retrieving timestamps for specific websites, fetching URLs and financial flags for a given query, leveraging Vertex AI’s Prediction API for sentiment analysis, and processing input for effective sitemap scraping.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aO7rPTwvFYVqFtUqZNFk0A.png" /><figcaption>The tech stack of libraries and tools used to build and deploy the application. Headings from AC215 course slides and inner content by the author.</figcaption></figure><p>The front end is built in HTML and JavaScript as a single-page application using both synchronous and asynchronous functions. It’s styled with CSS, uses multiple effects for loading indicators and sliding panels, and uses Google’s Material Design library to create modern-looking text inputs and buttons.</p><p>We generated 30 different non-copyright-infringing off-brand Bert images using OpenAI’s DALL-E 3, 10 each for positive, negative, and neutral sentiment. Depending on which sentiment is found, the front end chooses one of the appropriate sentiment images at random to display along with the classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*alcfKTMCewwykaIr3hy5JQ.gif" /><figcaption>An animation showing the querying process for the company <a href="https://www.ai21.com/">ai21.com</a>. Bert images were generated using OpenAI DALL-E 3.</figcaption></figure><p>Real-time updates on the progress and completion of each step are provided to clients during the sitemap scraping process. Nginx acts as the gateway, efficiently routing requests to the appropriate endpoints, ensuring a cohesive and responsive user experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Gwrkg3iRCOfdOuGZUBp0w.gif" /><figcaption>An animation showing the scraping for the <a href="https://cohere.com/">cohere.com</a>. Bert images were generated using OpenAI DALL-E 3.</figcaption></figure><h3>Deployment</h3><p>The RAG Detective App is deployed using Ansible for automation and reproducibility. The deployment process involves enabling the required Google Cloud Platform (GCP) APIs, setting up GCP service accounts, and creating a Docker container with the necessary software. Ansible playbooks are then used to build and push Docker containers to Google Cloud Registry (GCR), create a compute instance (VM) server in GCP, provision the instance with the required configurations, and setup the Docker containers in the instance. The process further involves configuring Nginx as the web server on the compute instance, ensuring proper setup and access to the Docker containers within the instance.</p><p>For scaling, we use Kubernetes. Ansible playbooks create and deploy a Kubernetes cluster. This involves building and pushing Docker containers to GCR and creating the Kubernetes cluster. Node scaling, where GKE automatically adds more nodes to the cluster, and pod scaling, run by the Kubernetes Horizontal Pod Autoscaler, which modifies the number of pod replicas based on resource utilization, are both involved in the scaling process in the event of increased demand. Load balancing is achieved through Kubernetes Services and GKE’s integration with Google Cloud Load Balancer, ensuring even distribution of incoming traffic.</p><p>One thing to note is that the Weaviate instance will not scale with our other containers, and implementing database scaling is beyond the scope of the course and a science in its own right. For scaling Weaviate, horizontal scaling is the primary approach, involving the addition of more nodes to the cluster to distribute the workload evenly. Alongside this is load balancing, which helps in evenly distributing incoming requests across the available nodes. Data sharding can be done as well, where data is partitioned across multiple nodes, allowing for efficient processing of queries. Kubernetes can also be used to automate the scaling and management of Weaviate instances, ensuring that the system remains robust and efficient as it grows to meet rising demands.</p><h3>Lessons Learned</h3><p>We learned some great general lessons about systems and operations:</p><ul><li><strong>Authentication can be a major pain point.</strong> Almost every time we deployed something for the first time, we would have issues where Google Cloud, Google Vertex, or OpenAI weren’t being properly authenticated or had the wrong permissions. Getting the hang of authentication and secrets management (we found using libraries like dotenv useful) is a major component of becoming skilled at operationalizing AI products.</li><li><strong>GPT models are forward-thinkers. </strong>In order to decide whether to invoke our BERT model on the LLM’s answer, we asked it to return a flag that indicated whether the answer was financial in nature. Initially, we had it generate the flag and <em>then</em> the answer, until we realized that GPT models cannot classify text prior to generating it, even if they’re currently in the process of generating it. Moving the flag to the end of the model’s streamed response was a little trickier, but it works better. One thing we did notice: GPT-3.5 is a lot worse at reasoning than GPT-4, and it often makes errors here that GPT-4 does not make.</li><li><strong>Use bleeding-edge libraries at your own risk</strong>. Despite having a working prototype using only Weaviate, we used LlamaIndex for our orchestration with the goal of using its more advanced RAG architectures. It’s documentation is full of example code but lacks a proper library reference, and we had to dive into its source code on a number of occasions. We could continue work on this project to add more complex RAG methods, which would both justify the use of LlamaIndex as well as give better responses to queries.</li><li><strong>If you are going to use a new library, make sure to check out the Discord chat. </strong>As a workaround to poor documentation, often the authors of the libraries are active on Discord and will be able to give direct guidance when you get stuck.</li><li><strong>Datasets, as usual, can contain hidden biases.</strong> We used thefinancial_phrasebank <a href="https://huggingface.co/datasets/financial_phrasebank">dataset</a>, and as previously mentioned, there were hidden reasons why greater annotator consensus seemed to result in a better model. We should always be cautious about taking seemingly obvious paths with data!</li></ul><figure><img alt="A colorful image in a New Yorker style, depicting a puppet inspired by Bert from Sesame Street, dressed as a detective. This unique puppet has a long, orange nose, a unibrow, and a mop of black hair. He is wearing a classic detective outfit, complete with a trench coat and a fedora hat. The puppet is examining documents with a magnifying glass, looking focused and inquisitive. The scene is vibrant and detailed, capturing the essence of a detective’s investigative work." src="https://cdn-images-1.medium.com/max/1024/1*n8kLLinvUNCCYuBfOvRuLA.png" /><figcaption>And that’s the end! Image generated using OpenAI DALL-E 3</figcaption></figure><p>Overall, this was a fun project involving very modern solutions to very modern problems, and the experience we gained both with training and deploying models on the cloud—as well as creating our own RAG architecture—was a great way to spend the course.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5a748b063040" width="1" height="1" alt=""><hr><p><a href="https://medium.com/institute-for-applied-computational-science/rag-detective-retrieval-augmented-generation-with-website-data-5a748b063040">RAG Detective: Retrieval Augmented Generation with website data</a> was originally published in <a href="https://medium.com/institute-for-applied-computational-science">Institute for Applied Computational Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How ChatGPT fools us into thinking we’re having a conversation]]></title>
            <link>https://medium.com/@iankelk/how-chatgpt-fools-us-into-thinking-were-having-a-conversation-fe3764bd5da1?source=rss-4d92cff1902------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe3764bd5da1</guid>
            <category><![CDATA[context]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[chatbots]]></category>
            <dc:creator><![CDATA[Ian Kelk]]></dc:creator>
            <pubDate>Wed, 29 Nov 2023 06:22:21 GMT</pubDate>
            <atom:updated>2023-11-29T06:22:21.423Z</atom:updated>
            <content:encoded><![CDATA[<p>Remember the first time you used ChatGPT and how amazed you were to find yourself having what appeared to be a full-on conversation with an artificial intelligence? While ChatGPT was (and still is) mind-blowing, it uses a few tricks to make things appear more familiar.</p><p>While the title of this article is a bit tongue-in-cheek, it is most certainly not clickbait. ChatGPT does indeed use two notable hidden techniques to simulate human conversation, and the more you know about how they work, the more effectively you can use the technology.</p><p><strong>Some key points I’ll address here are:</strong></p><ul><li>ChatGPT has no idea who you are and has no memory of talking to you at any point in the conversation.</li><li>It simulates conversations by “reading” the <em>whole</em> chat from the start each time.</li><li>As a conversation gets longer, ChatGPT starts removing pieces of the conversation from the start, creating a rolling window of context.</li><li>Because of this, very long chats will forget what was mentioned at the beginning.</li></ul><figure><img alt="A black and white illustration of a late-night talk show setting, titled ‘The ChatGPT Show.’ A classic, boxy robot with visible joints and a round head featuring antenna and eyes, is depicted as the guest. It’s gesturing with its hands as if in conversation. The host, a man in a suit with neat hair and a professional demeanor, sits across from the robot at a curved desk. Microphones and notes are on the desk, with an urban skyline visible through the window in the background." src="https://cdn-images-1.medium.com/max/1024/0*UXWMohcPHxul9x1u.jpg" /><figcaption>Created and edited from DALL-E 3</figcaption></figure><h3>Trick #1: Every time you talk to ChatGPT, you’re not just sending it your question. You’re also sending the entire conversation up until that point.</h3><p>Contrary to appearances, large language models (LLMs) like ChatGPT do not actually “remember” past interactions. The moment they finish “typing” out their response, they have no idea who you are or what you were talking about. Each time the model is prompted, it is completely independent from previous questions you’ve asked.</p><p>When ChatGPT seems to naturally recall details from earlier in the conversation, it is an illusion; the context of that dialogue is given back to ChatGPT every time you say something to it. This context enables them to build coherent, follow-on responses that appear to be normal conversations.</p><p>However, without this context, ChatGPT would have no knowledge of what was previously discussed. Like all LLMs, ChatGPT is completely <em>stateless</em>, meaning that in the actual model itself, no information is maintained between inputs and outputs. All of this feeding of previous context into the current interaction is hidden behind the scenes in the ChatGPT web application.</p><h3>A typical short conversation with ChatGPT might go like this:</h3><figure><img alt="A black and white comic strip depicts a conversation between a woman and a person wearing a ‘CHATGPT’ shirt. The woman asks, ‘What is photosynthesis?’ The person responds, ‘Photosynthesis is the process by which plants use sunlight to synthesize nutrients from carbon dioxide and water.’ She follows up with, ‘Can humans do it?’ to which the person replies, ‘No, humans cannot perform photosynthesis.’ The scene is simple with only the two characters and their speech bubbles." src="https://cdn-images-1.medium.com/max/1024/0*qVXZbxWS4PlWdrot.jpeg" /><figcaption>People created by DALL-E 3, comic bubbles and text by author</figcaption></figure><h3>However, this is what is actually going on behind the scenes:</h3><figure><img alt="A black and white comic strip showing a dialogue between a woman and a person wearing a ‘CHATGPT’ shirt. The woman is labeled ‘USER’ and asks, ‘What is photosynthesis?’ The person labeled ‘CHATGPT’ answers, ‘Photosynthesis is the process by which plants use sunlight to synthesize nutrients from carbon dioxide and water.’ The woman, termed ‘USER’ again, follows with, ‘Can humans do it?’ and receives the reply, ‘No, humans cannot perform photosynthesis.’ Both characters are drawn as cartoons." src="https://cdn-images-1.medium.com/max/1024/0*yQDMtvktymdkNAqD.jpeg" /><figcaption>People created by DALL-E 3, comic bubbles and text by author</figcaption></figure><p>Notice that when the woman asks her second question, she has to reiterate the entire previous conversation, complete with tags on who said what. Can you imagine talking to a person where every time it was your turn to speak, you had to repeat the entire conversation up to that point? This is how ChatGPT (and all current LLMs) work. They require using their own outputs, plus the prompts that generated these outputs, to be prepended to the start of every new prompt from the user.</p><p>These models are termed “auto-regressive” due to their method of generating text one piece at a time, building upon the previously generated text. “Auto-” comes from the Greek word “autós,” meaning “self,” and “regressive” is derived from “regress,” which in this context refers to the statistical method of predicting future values based on past values.</p><p>In LLMs, what this means is that the model predicts the next word or token in a sequence based on <em>all</em> the words or tokens that have come before it. That’s <em>all</em> of it, not just the current question being asked in a long back-and-forth chat conversation. In humans, we naturally maintain coherence and context in a conversation by just... participating in the conversation.</p><p>However, while chats with ChatGPT mimic a conversational style, with each response building upon the previous dialogue, the moment ChatGPT finishes writing a response, it has no memory of what it just said. Take a look at what would happen with this same conversation without the <em>entire discourse</em> being fed back to ChatGPT behind the scenes:</p><h3>How would ChatGPT respond without being fed the whole conversation?</h3><figure><img alt="A black and white comic panel featuring a conversation between a woman and a person wearing a ‘CHATGPT’ shirt. The woman asks, ‘What is photosynthesis?’ The person replies, ‘Photosynthesis is the process by which plants use sunlight to synthesize nutrients from carbon dioxide and water.’ The woman then asks, ‘Can humans do it?’ to which the person humorously responds, ‘Can humans do what?’ The characters are depicted in a lighthearted, cartoonish style, with the focus on their dialogue." src="https://cdn-images-1.medium.com/max/1024/0*stC9Na5tSX1QbtRk.jpeg" /><figcaption>People created by DALL-E 3, comic bubbles and text by author</figcaption></figure><h3>Trick #2: As your conversation grows, ChatGPT will quietly remove the oldest parts from the beginning.</h3><h4>Context Length in LLMs<a href="https://kelk.ai/blog/how-chatgpt-fools-us#context-length-in-llms">​</a></h4><p>When ChatGPT first came out in November 2022, it only offered the model GPT-3.5, which had a maximum context of 4,096 tokens, which is roughly 3,000 words. In a <a href="https://youtu.be/zjkBMFhNj_g?t=2642">recent talk,</a> Andrej Karpathy referred to this context window as “your finite precious resource of your working memory of your language model.”</p><p>What this means is that the GPT-3.5 model can comprehend a maximum of 4,096 tokens at any point. Tokenization is a fascinating subject in itself, and my next post will cover how it works and why 4,096 tokens only gives you about 3,000 words.</p><p>There’s often confusion about what the token limit means regarding input and output: can we give ChatGPT 3,000 words and expect it to be able to produce 3,000 words back? The answer is unfortunately no; the context length of 4,096 tokens covers both the input (prompt) and the output (response). This results in a trade-off where we have to balance the amount of information we give in a prompt with the length of the response we get from the model.</p><ol><li><em>Input (prompt):</em> A longer prompt leaves less room for a meaningful response; if the input uses 4,000 tokens, the response can only be 96 tokens long to stay within the token limit.</li><li><em>Output (response):</em> A shorter prompt could lead to a longer response as long as the combined length doesn’t exceed the token limit, but you may not be able to include information in the prompt.</li></ol><h3>The chat problem<a href="https://kelk.ai/blog/how-chatgpt-fools-us#the-chat-problem">​</a></h3><p>Do you see where this becomes problematic? Previously, we saw how the entire conversation has to be fed to the model so that it remembers what has already been discussed. Combining this with the context length, the result is that as you talk more and more with ChatGPT, eventually the combined totals of what you’ve asked and what it has replied will exceed the 4,096 token limit, and it won’t be able to answer any more.</p><h3>A visualization of ChatGPT simultaneously printing and scanning back in the entire conversation as it grows to extreme proportions:</h3><figure><img alt="An illustration showcasing a printer labeled ‘ChatGPT’ in the foreground, and a scanner in the background, with a large loop of paper moving between them. The printer is actively printing the paper, which then rises up, forms a significant loop, and clearly feeds into the scanner. The paper should be filled with printed text, resembling pages of a book. The drawing should vividly depict the paper’s journey from the ‘ChatGPT’ printer, through the loop, and into the scanner." src="https://cdn-images-1.medium.com/max/1024/0*5Nd9RjDOpgTQGRLV.jpg" /><figcaption>Created and edited from DALL-E 3</figcaption></figure><h4>Conversation Length and Token Limitations in LLMs<a href="https://kelk.ai/blog/how-chatgpt-fools-us#conversation-length-and-token-limitations-in-llms">​</a></h4><p>So how does ChatGPT handle this limitation? As your conversation with it grows, the number of tokens eventually exceeds the model’s context window (e.g., 4,096 tokens for GPT-3.5). ChatGPT invisibly removes the oldest parts of the conversation to remain within the limit. This method—using a rolling window of context—is certainly one of the easiest to implement, but oftentimes it is not the perfect solution. Some chat alternative front-ends, like <a href="https://www.typingmind.com/">TypingMind</a> both warn you when the context limit has been reached and allow you to manually delete sections of the chat that you don’t need anymore. This lets you choose what information you want to remain in the chat and has the bizarre philosophical effect of “editing the memory” of GPT.</p><p>For your average user using the web version of ChatGPT, what this means is that the longer your conversation, the sooner ChatGPT will start forgetting things you said at the beginning of the chat.</p><h3>Another visualization of ChatGPT trimming away the start of your conversation behind the scenes</h3><figure><img alt="A cartoon depicting a woman wearing a “ChatGPT” shirt, actively engaged in cutting a large roll of paper on the floor. The paper, covered in text like a book, is visibly being cut by the scissors in the woman’s hands. The cut is halfway through the paper, illustrating the action of cutting. The cartoon should emphasize the humorous situation, with the woman’s expression showing focus and the absurdly long paper being sliced by the scissors in a detailed and exaggerated style." src="https://cdn-images-1.medium.com/max/1024/0*rQch9294Fedop0RJ.jpg" /><figcaption>Created and edited from DALL-E 3</figcaption></figure><p>It’s good to be mindful of this restriction, especially when referring back to earlier parts of a conversation that might have been truncated due to token limitations—the LLM will not be able to recall these anymore, but the web version of ChatGPT will not tell you. There’s also always the risk that it could hallucinate answers based on other parts of the conversation if the beginning is trimmed off.</p><p>Let’s take another look at what happens in a more complex yet sillier chat interaction.</p><h3>Another typical but silly conversation with ChatGPT.</h3><figure><img alt="A black and white comic panel depicts a woman and a person wearing a ‘CHATGPT’ shirt having a conversation about rhymes. The woman says, ‘My name is Jane! Can you give me a word that rhymes with my name?’ The person responds, ‘Certainly! The word “train” rhymes with “Jane”.’ She asks for another, and they reply, ‘Of course! The word “plane” rhymes with “Jane”.’ She requests yet another, and the person concludes, ‘“Brain” rhymes with “Jane” as well.’ The drawing style is playful and cartoonish." src="https://cdn-images-1.medium.com/max/1024/0*2EGLkx1uZiEcIRGZ.jpeg" /><figcaption>People created by DALL-E 3, comic bubbles and text by author</figcaption></figure><h3>Again, this is what is actually going on behind the scenes.</h3><figure><img alt="A black and white comic strip portrays a dialogue between a woman and a person wearing a ‘CHATGPT’ shirt. The woman is labeled ‘USER’ and asks, ‘My name is Jane! Can you give me a word that rhymes with my name?’ The ‘CHATGPT’ character responds, ‘Certainly! The word “train” rhymes with “Jane”.’ The ‘USER’ asks for another word, and ‘CHATGPT’ says, ‘Of course! The word “plane” rhymes with “Jane”.’ The ‘USER’ requests another, prompting ‘CHATGPT’ to conclude with, ‘“Brain” rhymes with “Jane”." src="https://cdn-images-1.medium.com/max/1024/0*mCT-gUVOCIxDKSL5.jpeg" /><figcaption>People created by DALL-E 3, comic bubbles and text by author</figcaption></figure><h3>Now let’s suppose we have a long enough conversation that the beginning is trimmed off. ChatGPT might either state that it’s forgotten the name entirely or hallucinate it.</h3><figure><img alt="A black and white comic strip displays a woman labeled ‘USER’ asking a person in a ‘CHATGPT’ shirt for a word that rhymes with her name, Jane. Initially, ‘CHATGPT’ provides ‘train’ as a rhyming word. A series of speech bubbles follow with the text ‘…blah blah…?’ and ‘…blah!’ indicating an inaudible conversation. In the last panel, the ‘USER’ asks for another rhyming word, and ‘CHATGPT’ amusingly offers ‘jelly,’ humorously mispronouncing ‘Jane’ as ‘Kelly.’ The artwork is lighthearted." src="https://cdn-images-1.medium.com/max/1024/0*gJw_BCsX3vFRzvmh.jpeg" /><figcaption>People created by DALL-E 3, comic bubbles and text by author</figcaption></figure><h3>And finally, if we try to escape this problem by not feeding ChatGPT the entire conversation, it will forget it all the moment it finishes generating each answer.</h3><figure><img alt="A black and white comic strip features a woman and a person wearing a ‘CHATGPT’ shirt. The woman says, ‘My name is Jane! Can you give me a word that rhymes with my name?’ The person replies, ‘Certainly! The word “train” rhymes with “Jane”.’ She expresses delight and asks for another, to which the person humorously responds, ‘Sure! Another what?’ She clarifies she wants another word that rhymes with her name, and the person cheekily responds, ‘I’d be happy to help! What’s your name?’" src="https://cdn-images-1.medium.com/max/1024/0*yBoMdU0FnbgFYvoB.jpeg" /><figcaption>People created by DALL-E 3, comic bubbles and text by author</figcaption></figure><p>Since ChatGPT’s debut in November 2022, GPT-4 has been released with both 8,192 and 32,768 context lengths. This made things a lot better in terms of tracking long conversations, and in November 2023, GPT-4 Turbo was released with a 128k context length. Things are looking increasingly good for these models’ ability to track long conversations. However, despite GPT-4 Turbo’s massive amount of context, it still has a completion limit of 4,096 tokens, so it will always generate a maximum of about 3,000 words.</p><h3>Why is it like this?<a href="https://kelk.ai/blog/how-chatgpt-fools-us#why-is-it-like-this">​</a></h3><p>Transformer-based models like GPT-3 and GPT-4 are designed to be stateless, for good reason! Primarily, this stateless nature significantly enhances scalability and efficiency. Each user request is processed independently, allowing the system to handle numerous queries simultaneously without the complexity of tracking ongoing conversations. Imagine the complexity if every time the model was called, it had to maintain some internal state across millions of users.</p><p>Transformer model hidden states are also temporary and exist only for the duration of processing a specific input sequence. Once the model has processed an input and generated an output, these states are reset. They do not persist between different interactions.</p><p>Data privacy and security play a role as well. Stateless models do not retain a memory of past interactions, ensuring that sensitive data from one session is never inadvertently exposed to another user. This design choice is particularly relevant in light of incidents like <a href="https://en.wikipedia.org/wiki/Tay_(chatbot)">Microsoft’s Tay,</a> an AI chatbot that, due to its design to learn from interactions, ended up mimicking inappropriate and offensive language from users. It’s just not safe to have models learn from inputs given by random users.</p><p>However, the stateless nature also means these models cannot remember user preferences or learn from past interactions. This is a limitation in scenarios where you might want to create personalized chatbots through past chats or systems that benefit from cumulative learning. To mitigate this, some implementations incorporate a stateful layer atop the stateless LLM, enabling personalized and continuous user experiences.</p><h3>Getting a bit more technical<a href="https://kelk.ai/blog/how-chatgpt-fools-us#getting-a-bit-more-technical">​</a></h3><p>Prominent examples of such layers include <a href="https://www.langchain.com/">LangChain</a>, <a href="https://www.llamaindex.ai/">LlamaIndex</a>, and <a href="https://haystack.deepset.ai/">Haystack</a>. These layers add flexibility to managing the limited context that LLMs can handle by offering various strategies. For instance, when approaching the token limit, choices must be made: Should a rolling window approach be used to discard older text, like in the web ChatGPT, or should GPT be utilized to summarize previous information? Is it more important to retain the initial context, like a source article, while removing less critical middle or later sections? Alternatively, should retrieval augmented generation (RAG) techniques be employed to integrate external data into the token stream? These decisions vary based on the specific goals of the implementation. The most effective architectures often consist of specialized components interwoven to achieve a wide array of practical outcomes, allowing for more nuanced and effective user interactions.</p><figure><img alt="A cartoon depicting a robot sitting at a desk with an old-fashioned typewriter. The robot appears confused, as if it’s trying to remember something. It’s looking upwards with a thoughtful expression, one metal hand hovering over the typewriter keys. The scene is in black and white, capturing the essence of a classic cartoon, with simple lines and a humorous touch." src="https://cdn-images-1.medium.com/max/1024/0*Lncx2BoLH9bgQhZC.jpg" /></figure><p>KEY TAKEAWAYS</p><ul><li>ChatGPT doesn’t actually “remember” the conversations it has; instead, it simulates conversations by reading the entire chat from the start each time. Each interaction is independent, and once it’s completed its response, it loses all context from that conversation.</li><li>The entire conversation history must be fed back to ChatGPT each time to create coherent responses. But as the chat grows longer, it starts to forget the initial parts of the conversation because of the constraints of its token limit (4k tokens for GPT-3.5, 8k for GPT-4, and an amazing 128k tokens for GPT-4 Turbo).</li><li>In the case of prolonged interactions, the parts of the conversation that were removed won’t be recalled or acknowledged by the AI model. This happens unannounced in the web version of ChatGPT, making it important to be mindful of the length of the conversation and what parts may be forgotten.</li><li>If you’re having a lengthy conversation on the ChatGPT web app and it relies on information from the start, consider copying and pasting the necessary parts into a new chat to avoid losing crucial context.</li><li>Tools like <a href="https://www.typingmind.com/">TypingMind</a> allow users to selectively cut unimportant parts of the conversation and even direct the chat from a selected point. This ability to manually manage the conversation can provide a more controlled experience with ChatGPT.</li><li>ChatGPT and other similar models, like GPT-3 and GPT-4, are designed to be stateless for a variety of reasons, such as scalability, efficiency, and data privacy.</li><li>While their stateless nature makes these models safe and manageable at scale, the lack of memory and contextual retention is a limitation when it comes to personalization and continuous learning. Layered implementations like <a href="https://www.langchain.com/">LangChain</a>, <a href="https://www.llamaindex.ai/">LlamaIndex</a>, and <a href="https://haystack.deepset.ai/">Haystack</a> help in managing this constraint.</li><li>As more advanced versions like GPT-4 with 8,192 and 32,768 context lengths and GPT-4 Turbo with a 128k context length come up, it enables longer conversations. However, there will always be a completion limit, currently at about 3,000 words (4,096 tokens) for GPT-4 Turbo. This completion limit may come as a surprise given the much larger <a href="https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo">128k context of the model.</a></li><li>Trying to dialogue with these models without providing the entire conversation effectively erases the memory of prior interactions, making them less effective in terms of continuity and coherence.</li><li>Future technological advances and updates may amend some of these constraints, enhancing the AI’s ability to maintain long and complex conversations while also respecting user data privacy.</li></ul><p>For more, follow me here on Medium, <a href="https://www.linkedin.com/in/iankelk/">LinkedIn</a>, or check out my <a href="https://kelk.ai/">projects page</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe3764bd5da1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How AI Could Have Warned Us about the Florida Condo Collapse Before It Happened]]></title>
            <link>https://medium.com/data-science/how-a-i-can-prevent-future-building-collapses-before-they-happen-71c3bf3740b5?source=rss-4d92cff1902------2</link>
            <guid isPermaLink="false">https://medium.com/p/71c3bf3740b5</guid>
            <category><![CDATA[concrete-damage]]></category>
            <category><![CDATA[champlain-tower]]></category>
            <category><![CDATA[concrete]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[editors-pick]]></category>
            <dc:creator><![CDATA[Ian Kelk]]></dc:creator>
            <pubDate>Tue, 17 Aug 2021 05:59:55 GMT</pubDate>
            <atom:updated>2021-08-24T20:40:22.430Z</atom:updated>
            <content:encoded><![CDATA[<h4>A human can’t sit and stare at a wall all day and night. A computer can.</h4><p>The partial collapse of a 12-story beachfront condominium in Surfside, Florida was a shock to the nation. With 98 people confirmed dead, it is one of the <a href="https://en.wikipedia.org/wiki/List_of_structural_failures_and_collapses">deadliest structural collapses</a> in American history. The root cause of the structural failure has yet to be determined, but from <a href="https://www.miamiherald.com/news/local/community/miami-dade/miami-beach/article252652078.html">initial reports</a> it does seem that this tragedy could have been prevented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VAgVYUjZEkdwrGchcSQetw.png" /><figcaption>Left photo showing the aftermath of the <a href="https://en.wikipedia.org/wiki/Surfside_condominium_building_collapse">Surfside condominium building collapse</a> which fell on June 24, 2021. Right photo showing detection of “concrete cancer” by an AI detector. Left photo by <a href="https://twitter.com/MiamiDadeFire/status/1408074745258680327">Miami-Dade Fire Rescue Department</a> and is <a href="https://commons.wikimedia.org/wiki/File:Surfside_condominium_collapse_photo_from_Miami-Dade_Fire_Rescue_1.jpg">Public Domain</a>. Right photo by <a href="https://commons.wikimedia.org/wiki/User:Achim_Hering">Achim Hering</a> on <a href="https://commons.wikimedia.org/wiki/File:Qew_bruecke_nf_beton_kaputt_25_von_46.jpg">Wikimedia</a> under the <a href="https://en.wikipedia.org/wiki/en:Creative_Commons">Creative Commons</a> <a href="https://creativecommons.org/licenses/by/3.0/deed.en">Attribution</a></figcaption></figure><h3>Imagine if the security cameras that already exist in many high-rise parking garages could also report damage in the concrete of the building itself</h3><p>After working for the past few weeks as an <a href="https://www.upwork.com/">Upwork</a> contractor for an AI startup called <a href="https://www.clarifai.com/">Clarifai</a>, I realized I could use their platform to create such a thing. Camera systems already spend 24 hours a day filming public areas––including the concrete parking garages. The below video shows a full demonstration, including a walkthrough of the garage of Champlain Towers South a year before the disaster.</p><p>An important thing to note: While the below demo uses video taken by a person while walking (it’s the only footage we have of the Champlain Towers garage prior to the collapse), the AI is designed to work with <strong>stationary cameras mounted on the walls. </strong>It would expand the role of security cameras to not only record the actions of humans in their field of view, but to also report the conditions of the walls themselves.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fg4tnZTghSmg%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dg4tnZTghSmg&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fg4tnZTghSmg%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/38118c9e4e93035a4d332de7cc1d49f0/href">https://medium.com/media/38118c9e4e93035a4d332de7cc1d49f0/href</a></iframe><h3>A little bit of background</h3><p>Living in a tropical climate like Miami Beach has certain complications, specifically flooding during the summer months. The photo and video below show some of the more extreme examples, but this happens between 3–4 times a season. It’s all too common to see naive drivers attempt to cross flooded roads, water getting sucked into their intakes and destroying their engine blocks. Heavy rainfall is usually accompanied by a few stranded vehicles, and occasionally hurricanes push even more water onto the area through storm surge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Aj-BPLLyTCXVNoLLAMeXdg.jpeg" /><figcaption>A photo I took of Flooding from a summer rainfall during July of 2018. This was taken on the 6900 block of Collins ave, 8700, 18 blocks south of where the Champlain Towers South was situated. Photo by the author.</figcaption></figure><p>While the cause of the Champlain Towers disaster has yet to be determined, it’s easy to imagine how such flooding, combined with the salty sea air, would create a unique set of challenges. This problem looks to be worsening as well, so much that U.S. Army Corps of Engineers <a href="https://www.nytimes.com/2021/06/02/us/miami-fl-seawall-hurricanes.html">proposed a 20-foot seawall</a> to protect the City of Miami. It will come as little surprise if, in the years to come, more Florida buildings will be found unsafe and evacuated as a precaution.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fs9bfl65u9zw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Ds9bfl65u9zw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fs9bfl65u9zw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a3528789ef0839d4bb3ed4bf86f1d72/href">https://medium.com/media/2a3528789ef0839d4bb3ed4bf86f1d72/href</a></iframe><h3>The perspective from my rooftop</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n0pT-YXAvh4z9fGqP4Grfw.jpeg" /><figcaption>A photo I took from the roof of my building in Miami Beach, about 2 miles from the Champlain Towers, with the City of Miami in the background. Dramatic skies and sunsets are typical for the area. Photo by the author.</figcaption></figure><p>The Surfside catastrophe had a personal effect on me. For 3 years, I lived in a similar building in Miami Beach, just 2 miles south of the disaster site. My home was situated on the same stretch of beautiful beachfront property running along Collins Ave, all the way from Sunny Isles down to South Point at the tip of Miami Beach. I’ve walked by the Champlain Towers many times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_05idL_7t2ZxTLYiOgblUQ.jpeg" /><figcaption>A photo taken off my balcony of two giant cruise ships sitting moored with skeleton crews during the COVID-19 pandemic on April 1, 2020. Photo by the author.</figcaption></figure><p>Even as the rescue operations shut down and switched into recovery mode, the effects of the collapse have already begun impacting the community. People who live in high-rises are scared for their safety. Homeowners are scared for their property values. Already shaken by the COVID-19 pandemic, everything in the area feels uncertain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nqgaOKwmyfxVGp2c8KwLUw.jpeg" /><figcaption>Security cameras are a common sight in all areas of South Florida high-rises. Photo by <a href="https://unsplash.com/@whatispictureperfect?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">What Is Picture Perfect</a> on <a href="https://unsplash.com/s/photos/security-camera?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>One thing I kept thinking about from my time living in Miami Beach, is that every high-rise in South Florida has cameras <em>everywhere</em> — even in the parking garages.</p><p>This means that using artificial intelligence, we can create an early warning system for structural problems in buildings using existing security cameras.</p><h3>AI is already saving lives in medicine</h3><p>When you think of “artificial intelligence”, any number of things could come to mind. You might think of computers defeating <a href="https://en.wikipedia.org/wiki/Deep_Blue_versus_Kasparov,_1997,_Game_6">chess grandmasters</a>, <a href="https://www.youtube.com/watch?v=fn3KWM1kuAw">dancing robots</a> from Boston Dynamics, or even a computer that is capable of <a href="https://www.youtube.com/watch?v=PqbB07n_uQ4">being interviewed</a>. However, there are lots of subsets of AI, and some aren’t that hard to conceptualize.</p><p>One part of AI is known as machine learning, and it works somewhat similarly to how humans learn. If I told you that I wanted you to learn how to identify lymphoma by looking at CT scans, how would you go about it? You’d likely ask to be provided with several examples of CT scans with cancer, and several examples of CT scans that do not contain any cancer. Maybe given even just a handful of these examples, you could begin spotting tumors in medical scans. This is where humans have the advantage — with just a few examples, we’re able to spot visual cues that lead to conclusions.</p><figure><img alt="Esophageal cancer, CT scan with contrast, coronal image" src="https://cdn-images-1.medium.com/max/587/1*TdxWAvwXaZedCi4Z2zf-4w.jpeg" /><figcaption>Esophageal cancer, CT scan with contrast, coronal image (Photo by <a href="https://commons.wikimedia.org/wiki/User:Tdvorak">Tdvorak</a> on <a href="https://en.wikipedia.org/wiki/File:Tumor_Esophagus2.JPG">Wikimedia</a>, reposted under <a href="https://en.wikipedia.org/wiki/en:Creative_Commons">Creative Commons</a> Attribution-Share Alike )</figcaption></figure><p>Machine learning works similarly, but with a lot more pictures to learn from. Instead of just a few images, you tell the computer, “here are a few thousand scans of cancer, and here are a few thousand scans of patients with no cancer.” The computer then chugs away and produces a “model”, which can look for any number of differences between cancerous and non-cancerous examples.</p><p>This data used to train the computer is conveniently called the “training data”, and the more that is available can lead to surprising, unexpected behavior where computers are able to detect things that humans are not. The most important point here is that this technology is <strong>already in wide use</strong> for <a href="https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(18)31645-3/fulltext">medical scans</a>, and even can <a href="https://healthitanalytics.com/news/deep-learning-model-can-enhance-standard-ct-scan-technology">improve their quality.</a></p><h3>We can use a similar method to detect problems in concrete foundations</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jgoOA1Yy2lejd02HTc2iTQ.png" /><figcaption>The AI model I trained detecting damage in the ceiling of the Champlain Towers South garage. Photo by author, original video footage of <a href="https://www.youtube.com/watch?v=w4rGWH7aDJc">Champlain Tower South: visiting unit 611 on July 17, 2020 — Surfside FL condo collapse</a> by <a href="https://www.youtube.com/user/FIUEEEE">Dr. Fiorella Terenzi</a></figcaption></figure><p>Think of this again as a human. How does someone notice if something is wrong with the concrete foundations of a building? Visual cues are often the first indication. Are there rust stains? Are there cracks? Are there pieces falling off or any noticeable deterioration? Failing concrete can yield clues that experienced engineers would notice — experience gained over years examining both failing and structurally sound buildings. John Pistorino, the man behind requiring 40-Year recertifications for buildings, said in a recent <a href="https://www.local10.com/news/local/2021/06/25/meet-the-engineer-hired-to-get-to-the-bottom-of-the-surfside-collapse/">interview</a> that “concrete gives you a warning. It gives you a warning. It doesn’t fail that fast.”</p><blockquote>A human can’t sit and stare at a wall all day and night. A computer can.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yOhYIBVpkl5yHVHQR44NTQ.jpeg" /><figcaption>An example of serious “concrete cancer” where the inner rebar is exposed and rusted. The appearance of damage like this could imply immediate danger to the structure of a building. Photo by <a href="https://www.freeimageslive.co.uk/users/gratuit">gratuit</a> on <a href="https://www.freeimageslive.co.uk/free_stock_image/rustyrebar4476jpg">freeimageslive.co.uk</a> licensed under a <a href="http://creativecommons.org/licenses/by/3.0/">Creative Commons Attribution 3.0 Unported License</a>.</figcaption></figure><p>Years of noticing visual cues is something that can be approximated by training an AI with thousands of images. Interestingly enough, a common term for describing the rusting of steel rebar contained within a concrete slab is known as “concrete cancer”. As the steel corrodes, it expands and causes the surrounding concrete to become brittle and crack, accelerating the process of erosion. Telltale signs of concrete cancer are concrete “spalling” — cracks and peeling away of small particles, as well as rust stains that appear to seep out from the inside, and bubbling and leaks that appear overhead. If the interior steel rebar is exposed, that means the problem has become extremely serious.</p><p>Terms like “concrete cancer” and “concrete spalling” can be very powerful when combined with image searches to assemble large training sets for an AI model to detect it. Below, I used <a href="https://portal.clarifai.com/">Clarifai’s online portal</a> to create a model using images I found through the search term “concrete cancer.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jw7g5EFMBsXb4o03LhxX1A.png" /><figcaption>Training the model using images of damaged concrete. Photo by author.</figcaption></figure><p>Remember what I mentioned earlier about Miami high-rises having cameras everywhere? Those cameras feed into a closed circuit monitoring system that is displayed at security desks. Those cameras could also be fed into an AI system that is constantly checking for any signs of structural damage every moment of every day and night. The system isn’t magical; if you can’t see a problem, then likely the computer can’t either. The difference is that the computer is able to watch the walls for damage 24/7 without ever taking its eyes off of them — something that is impossible for a human.</p><p>The other advantage of using existing cameras is that residents are both comfortable with their existence, and embrace the security they offer. Having the cameras also keep an eye on the resilience of the structure itself brings that much more confidence to a nervous public.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FqSP2BAsRPUsac5Ia9f%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FqSP2BAsRPUsac5Ia9f%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FqSP2BAsRPUsac5Ia9f%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/5002ca11919469bdebe1a4ae1c92f5e8/href">https://medium.com/media/5002ca11919469bdebe1a4ae1c92f5e8/href</a></iframe><h3>Another way an AI could find problems in concrete foundations —change detection</h3><p>An AI model monitoring for structural damage can also play “spot the differences”, a familiar game for children.</p><p>Take a look at the busy photo below. There are 8 differences between them; a computer would spot all of them in a fraction of a second. How long does it take you?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OWQFUhEPVYKt6hLavMo-zw.png" /><figcaption>Original photo by <a href="https://unsplash.com/@christopher__burns?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Christopher Burns</a> on <a href="https://unsplash.com/s/photos/crowd?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a>. Edited by author to remove certain objects from right side.</figcaption></figure><p>Comparing two images and detecting the differences is something that computers excel at; with a stationary camera they notice changes much faster than humans. Given the plethora of cameras already installed in South Florida buildings, a system could monitor these feeds and detect any change in the building itself. The only challenge in such a system is the ability to distinguish between the concrete wall of a building and cars or people passing by, but that is easily doable with modern technology.</p><p>This idea — change detection — is an important concept for an AI model. Even if a concrete structure doesn’t display any signs of damage, <em>it shouldn’t move</em>. The solution is straightforward — a camera watches a wall, and if the wall changes in any way, people should be alerted.</p><blockquote>One thing I knew from living in Miami Beach, is that every high-rise in South Florida already has cameras <em>everywhere</em> — even in the parking garages.</blockquote><h3>Building the model</h3><p>I’ve been able to train a basic model to detect damaged concrete in very little time. A number of their engineers have been extremely supportive and interested in this project, and worked with me to create the working prototype. I was able to put together a training data set of 300 photos of damaged concrete, and 173 photos of undamaged concrete. This is a rather small data set for a project of this scope, but it was sufficient to build a fairly accurate prototype.</p><p>The need for quality training data is frequently a blocking issue in AI. If you’ve ever used facial recognition on Facebook or Google, you’ll probably have noticed that it becomes more accurate over time, as you let it know when it’s correctly tagged your face and when it hasn’t.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FqsqBewYF9Szv9ZdxdQ%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fconcrete-cancer-qsqBewYF9Szv9ZdxdQ&amp;image=https%3A%2F%2Fmedia0.giphy.com%2Fmedia%2FqsqBewYF9Szv9ZdxdQ%2Fgiphy.gif%3Fcid%3D790b7611fe9959bed1878389db102b5461b70a885af7941e%26rid%3Dgiphy.gif%26ct%3Dg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/8e6c5891b1c974f4b4fbe2420c46a150/href">https://medium.com/media/8e6c5891b1c974f4b4fbe2420c46a150/href</a></iframe><p>The same is true for images of “concrete cancer” and “concrete spalling.” By far the most time consuming part of creating the prototype detector model seen in the video was the collection of training data. The more images I can get of both damaged and undamaged concrete, the better the model will become. If anyone has access to large numbers of photos of damaged concrete and willing to share, let me know! The actual training of the model itself was accomplished overnight while I slept.</p><p>My prototype model works fine with the footage of the Champlain Towers’ garage. All that’s left is to try it out on a live camera feed, and testing a system that could save lives in the future.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FBDVUjyyiqOJU8GVAZM%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FBDVUjyyiqOJU8GVAZM%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FBDVUjyyiqOJU8GVAZM%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/91ef5cf941426f6267c2c5b76d11ccfe/href">https://medium.com/media/91ef5cf941426f6267c2c5b76d11ccfe/href</a></iframe><p>For more information, follow me here on Medium or <a href="/service/mailto:iankelk@gmail.com">email me</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71c3bf3740b5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-a-i-can-prevent-future-building-collapses-before-they-happen-71c3bf3740b5">How AI Could Have Warned Us about the Florida Condo Collapse Before It Happened</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing Virmuze: Create Exhibits Online]]></title>
            <link>https://medium.com/@iankelk/introducing-virmuze-create-exhibits-online-8955a81c9fb9?source=rss-4d92cff1902------2</link>
            <guid isPermaLink="false">https://medium.com/p/8955a81c9fb9</guid>
            <category><![CDATA[online-museum]]></category>
            <category><![CDATA[virtual-museum]]></category>
            <category><![CDATA[art-gallery]]></category>
            <category><![CDATA[museums]]></category>
            <category><![CDATA[art]]></category>
            <dc:creator><![CDATA[Ian Kelk]]></dc:creator>
            <pubDate>Sat, 30 May 2020 22:09:05 GMT</pubDate>
            <atom:updated>2020-06-01T19:11:13.427Z</atom:updated>
            <content:encoded><![CDATA[<h4>Museums have long needed a way to quickly create and publish online exhibits – and even profit from them. Virmuze is a free exhibit platform, and we want to showcase your art.</h4><p>If you’ve ever wanted to create an online exhibit for your art, but didn’t know how or didn’t have the resources, this is the post for you!</p><p>Let’s live in a fantasy world and flash back to before the coronavirus pandemic for a moment. At that time, I was living in Washington D.C. It’s a fascinating place where people from all over the world work in both U.S. politics and international organizations, and it is also home to a <em>lot</em> of museums. Wikipedia lists 80 active museums in DC, and that isn’t including museums in Maryland and Virginia 20 minutes away. The Smithsonian alone runs 19 museums in the National Mall (it’s a park, not an actual mall — I was confused at first too).</p><p>But here’s the thing. Whenever I’ve lived in a place with many things to see, I always put off visiting them – they’re right there and I could do it at any time. So when it came time to leave D.C., I hurriedly spent a solid week visiting all the museums I had missed, despite the insanely hot and humid weather plaguing the city that summer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jxgrLBAtC_CAzyWM.jpg" /><figcaption><em>Smithsonian Institution National Museum of Natural History — Photo by mana5280 on Unsplash</em></figcaption></figure><p>While many of Washington’s museums have free admission, some do require reservations due to their <em>huge</em> number of visitors. Some of them have truly massive crowds, and even standing in front of the exhibits I would have to wait for the many layers of people to move on before I could see them. After many hours of fighting my way through the mobs of people I thought, “I’ll just look up the exhibits I missed online later.”</p><p>Except the exhibits <em>weren’t</em> online. The websites of all the museums I visited just had basic information about locations, events, hours, and special exhibits. A few have “virtual” components where you can walk through à la <em>Google Street View</em>, but nothing easy to read or engage in.</p><p>So I created Virmuze — a free online platform for museums, galleries, artists, and researchers to create beautiful exhibits and easily make them available to the world. You don’t need to know anything about websites to make your online museum – if you can upload a picture to Facebook, you can create your own Virmuze exhibit.</p><p>Users of Virmuze can find your work through their Discovery Page recommendations, and they can stay up to date; anything you add to exhibits they follow will appear in their personalized news feed. It’s a fantastic way to keep a collection alive by regularly adding to it. You can see an example of an <a href="https://virmuze.com/m/margie-kelk-marine-life/x/undersee/">art exhibit here,</a> or a science <a href="https://virmuze.com/m/the-museum-of-hurricanes/x/meet-hurricane-irma/">exhibit about hurricanes here.</a></p><p>With the conclusion of Akimbo’s Gallery at Home feature, I’m delighted to extend a personal invitation to all Akimbo readers to create your own online art museums. Akimbo has generously offered a 10% discount off their services to promote Virmuze exhibits.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FTApJjp7Cn88%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DTApJjp7Cn88&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FTApJjp7Cn88%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e691ee8bcd70d05aa6518d9c1ca8044/href">https://medium.com/media/6e691ee8bcd70d05aa6518d9c1ca8044/href</a></iframe><h3>Back to reality</h3><p>Today, the COVID-19 pandemic has closed all the world’s museums and galleries for months. Many are still closed, and even the ones that reopen are having to deal with many precautions that make the experience of visiting a stressful affair. Attendance is down, and many of the art world’s benefactors are older, and therefore at a higher risk from the coronavirus. There has never been a more compelling time to create your own online exhibits!</p><ul><li>Anyone, from anywhere in the world, can view them.</li><li>Visitors with disabilities can enjoy exhibits they wouldn’t be able to see in person.</li><li>They’re open 24/7.</li><li>There is no space limitation – display however many works you like.</li><li>There is no need for hand sanitizer, face masks, or social distancing when viewing an exhibit online.</li><li>Virmuze has future plans to allow museums to accept donations and sell tickets to view special online exhibits. As an artist or gallery, you will be able to make money from visitors all over the world.</li></ul><p>Interested? You can create an account and build your exhibit right now! Here are a few links to help you get started:</p><ul><li>A <a href="https://www.youtube.com/watch?v=TApJjp7Cn88">5-minute video</a> explaining Virmuze in more detail.</li><li>A <a href="https://www.youtube.com/watch?v=4597D4O0NAc">2-minute time-lapse</a> of an exhibit being created. This is not a tutorial! It’s just a quick demo that gives an idea of how fast an exhibit can be created. See the next link for a deeper look.</li><li>An <a href="https://virmuze.com/m/the-museum-of-virmuze/x/how-to-build-an-exhibit/">in-depth Virmuze help exhibit explaining how to create an exhibit.</a></li></ul><p>Ready to get started? <a href="https://virmuze.com/register/">Register here and create your museum!</a></p><p>As mentioned above, Akimbo is offering a 10% discount off their services to promote work that you create on Virmuze.</p><p>Need any help? Send any questions to <a href="/service/mailto: info@virmuze.com">info@virmuze.com</a></p><h4>Social links</h4><p>Facebook: <a href="https://www.facebook.com/virmuze">@virmuze</a></p><p>Twitter: <a href="https://www.twitter.com/virmuze/">@virmuze</a></p><p>Instagram: <a href="https://www.instagram.com/virmuze/">@virmuze</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/179/1*ESOm2Z_SawRCNbblbQYhcw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8955a81c9fb9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>