<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ritesh Jha on Medium]]></title>
        <description><![CDATA[Stories by Ritesh Jha on Medium]]></description>
        <link>https://medium.com/@ritzardous?source=rss-7ef54900e343------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*XWEsGcZqkYDkk7WrUHdS1Q.jpeg</url>
            <title>Stories by Ritesh Jha on Medium</title>
            <link>https://medium.com/@ritzardous?source=rss-7ef54900e343------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 10:41:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ritzardous/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Finally Understood How ChatGPT Works]]></title>
            <link>https://ritzardous.medium.com/i-finally-understood-how-chatgpt-works-c8a9c97b8d3e?source=rss-7ef54900e343------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8a9c97b8d3e</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Ritesh Jha]]></dc:creator>
            <pubDate>Mon, 22 Jun 2026 23:53:01 GMT</pubDate>
            <atom:updated>2026-06-22T23:53:01.473Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GjocCrsun-Txzy_fn_Hu2A.png" /></figure><h3>1. What is GPT?</h3><p>So, if we start from the very basics of it, lets first try to understand the fullform. It is Generative Pre-trained Transformer. Now lets start by understanding each term and its meaning.</p><p>So <strong>Generative </strong>means that it generates something new instead of a simple retrieval on already existing data. For Example, you are having a bad day and go and ask chatgpt to tell you a joke, and then it tells you a joke (which we all know is very lame and doesn’t make you laugh, moving on), so this joke was generated by it, token by token.</p><p>In the context of GPTs, <strong>Pre-trained</strong> is a very simple term, which indicates that the model has already seen a huge amount of data before you start interacting with it. You can think of this data being sourced from books, websites, documentation, articles, etc. And it learns quite a few things from this data such as language, facts, and also patterns as well as reasoning structures.</p><p><strong>Transformer </strong>is a neural network architecture introduced in the legendary research paper by google research called ‘Attention is all you need’ — which you can read here : <a href="https://arxiv.org/pdf/1706.03762">https://arxiv.org/pdf/1706.03762</a>. We’ll talk about transformers in detail later.</p><h3>2. The Problem GPT Solves</h3><p>Before discussing things &amp; architecture further, lets first try to gauge in the object of chatgpt. The sole purpose of GPTs is <strong>‘to predict the next token’</strong>.</p><p><strong>Example :</strong></p><p>The capital of Uganda is _____.</p><p>And the model predicts : Kampala</p><p><strong>Another example could be :</strong></p><p>I am going to the ____.</p><p>Possible probabilities :</p><p>store : 40%</p><p>park : 25%</p><p>mall : 15%</p><p>gym : 10%</p><p>If you look at all of it from a higher view, its nothing but a very sophisticated autocomplete. All of this complexity and research exists to solve just this one problem.</p><h3>3. Before Transformers (History)</h3><p>Before 2017, there existed RNNs, LSTMs, GRUs, etc. They faced several challenges, such as words being processed one by one, they were slow, couldn’t remember long contexts and were really hard to parallelize.</p><p><strong>Example </strong>: “India is a vast and beautifully diverse country that thrives on a rich heritage spanning millenia and it is known globally for its concept of ‘Unity in Diversity’.”</p><p>RNN might forget “rich heritage” until it reaches “unity in diversity”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ssECaLady2rYzhANZYkLw.png" /></figure><h3>4. Transformer Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*_GUf9WPLj9TWeyKJcV55pA.png" /></figure><p>This right here is the original transformer architecture proposed in Attention is all you need paper, GPT is derived from this arch but only uses the decoder portion.</p><p>The original Transformer contains:</p><p>Encoder — Used for understanding input</p><p>Decoder — Used for generating output</p><p>A good application of using both of these components could be google translate.</p><p>Whereas ChatGPT uses only the decoder stack, and this is also why GPT is often called a ‘Decoder-Only Transformer’.</p><p>A simplified architecture of GPT specifically :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*O1VAGWuQl7kfSjjJxgaGbA.png" /></figure><h3>5. Step 1: Tokenization</h3><p>We first need to know that computers do not understand text. But they do understand numbers. So before processing text GPT converts text into tokens first. An example of this could be a simple “Hello Duniya” being converted into a numerical value, so here lets say it might become [15496, 2159], these numbers are token IDs.</p><p><strong>Vocabulary</strong></p><p>Every model maintains its own vocabulary, for example :</p><p>Hello -&gt; 15496</p><p>Duniya -&gt; 2159</p><p>Different models make use of different tokenizers, such as GPT tokenizer, Claude tokenizer, Llama tokenizer.</p><p>This tool : <a href="https://tiktokenizer.vercel.app/">https://tiktokenizer.vercel.app/</a> is great for visualizing how tokenization is done, and what their value for different natural language sentences are.</p><p>There also exists something called <strong>vocabulary size</strong>, which simply means that a larger vocabulary can represent common words using fewer tokens, whereas a smaller vocabulary may split words into more fine pieces.</p><h3>6. Step 2: Embeddings</h3><p>Token IDs themselves have no meaning of their own. Instead the models convert tokens into vectors.</p><p>suppose : 15496 -&gt; [0.21, -1.44, 0.89, …]</p><p>This process is called embedding. Embeddings allow the models to capture the semantic meaning, so words with similar meanings tend to have similar embeddings. So King, Queen, Prince, Princess will be located close to each other in the embedding space.</p><h3>7. Step 3: Positional Encoding</h3><p>Embeddings alone don’t preserve the order in which words appear. Suppose ‘Snake bites man’ and ‘Man bites snake’, the words here are identical but their meaning is completely different, and hence GPT adds positional information to embeddings. This allows the model to understand sequence order. Position becomes part of the token representation.</p><h3>8. Self Attention</h3><p>This allows tokens to look at other tokens in the same sequence, instead of processing words one by one, every token can consider the entire context. Consider ‘River Bank’ and ‘HDFC Bank’, the word ‘bank’ means diff things in each sentence. So, self attention allows nearby words to influence the representation of ‘bank’. THe model understands context through the relationships between words.</p><h3>9. Multi-Head Attention</h3><p>A single attention mechanism would only focus on one type of relationship. Transformers are much more advanced so they therefore use multiple attention heads.</p><p>Head 1 -&gt; Grammar</p><p>Head 2 -&gt; Subject-verb relationships</p><p>Head 3 -&gt; Long Distance Context</p><p>Head 4 -&gt; Entity References</p><p>And imagine all of them working together to achieve a higher and richer contextual understanding.</p><h3>10. Feed Forward Layers</h3><p>The task of gathering information lies with attention, while the Feed Forward Network processes that information. A transformer block typically consists of:</p><p>Multi Head Attention -&gt; Feed Forward Network</p><p>This structure is repeated multiple times, with multiple blocks, where each layer gradually develops a deeper understanding of the text.</p><h3>11. Output Generation</h3><p>After passing through all the transformer blocks, GPT must decide what token to generate next, if you’d remember this was the crux of this whole thing. The final representation produced by the Transformer is passed through two components :</p><p><strong>Linear Layer</strong> — converts the model’s internal understanding into scores for every token in the vocabulary</p><p>Paris = 8.2</p><p>London = 3.5</p><p>Berlin = 1.1</p><p><strong>Softmax </strong>— converts those scores into probabilities</p><p>Paris = 92%</p><p>London = 5%</p><p>Berlin = 3%</p><p>The model can now finally decide which token it should generate next.</p><p><strong>Temperature </strong>controls how much randomness is introduced during token selection. The selected token is appended to the existing sequence and the entire process repeats again.</p><p><strong>Continuous Token Generation : </strong>Suppose you ask ‘What is the capital of India?’ and the model predicts Delhi. So, now the complete sequence becomes : ‘What is the capital of India? Delhi’. The model then predicts the next token, then its next, then its next and so on… This whole loop continues until End of Sequence token <em>&lt;EOS&gt;</em> is produced. This is how complete responses are generated one token at a time!</p><h3>12. How GPT Learns</h3><p>Till now all of what have discussed has focused on how GPT generates text. But how does it really learn and become intelligent? Its actually fairly simple, it learns by repeatedly predicting the next token and comparing its predictions with the actual answer. Any error is calculated using a Cross Entropy Loss.Then that error is then propagated backwards through the network using Backpropagation, slightly adjusting the parameters. This process is repeated billions of times across massive datasets until the model becomes good at predicting language patterns. Everything from coding assistants to chatbots ultimately builds upon this same next token prediction mechanism.</p><p>So the next time ChatGPT gives you a painfully average joke, remember there are billions of parameters working really hard behind that disappointment :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8a9c97b8d3e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>