<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kiran Raj on Medium]]></title>
        <description><![CDATA[Stories by Kiran Raj on Medium]]></description>
        <link>https://medium.com/@kiranraj_43856?source=rss-b648824a93f1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HlMmEmpobtNkHPEP6Yj8oA.jpeg</url>
            <title>Stories by Kiran Raj on Medium</title>
            <link>https://medium.com/@kiranraj_43856?source=rss-b648824a93f1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:04:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kiranraj_43856/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Build Your First Local RAG Application — A Hello World Example]]></title>
            <link>https://medium.com/@kiranraj_43856/build-your-first-local-rag-application-a-hello-world-example-5dac31ce0f4d?source=rss-b648824a93f1------2</link>
            <guid isPermaLink="false">https://medium.com/p/5dac31ce0f4d</guid>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[agentic-rag]]></category>
            <dc:creator><![CDATA[Kiran Raj]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 05:59:55 GMT</pubDate>
            <atom:updated>2026-11-12T05:59:55.781Z</atom:updated>
            <content:encoded><![CDATA[<p>In my previous article, we discussed what RAG is and how it works.</p><p>Now let’s build a very simple RAG application that runs completely on our local machine.</p><p>We will use Python, Ollama and ChromaDB. We don’t need any cloud services or API keys.</p><p>Our goal is simple: store some information locally, ask a question, retrieve the relevant information and let an LLM generate an answer using that information.</p><h3>1. What are we going to build?</h3><p>Imagine we have a text file containing some technical troubleshooting information.</p><p>For example:</p><ul><li>How to fix a Kubernetes pod stuck in Pending state.</li><li>How to troubleshoot Redis connection failures.</li><li>How to check whether a Model Gateway service is running.</li></ul><p>Instead of manually searching this file every time, we want to ask questions like:</p><p>“Why is my Redis pod stuck in Pending state?”</p><p>Our RAG application should find the relevant information and generate an answer.</p><p>The architecture will look like this:</p><p><strong>Knowledge File → Embeddings → ChromaDB → Retrieve Relevant Information → LLM → Answer</strong></p><h3>2. Install the required tools</h3><p>First, install <a href="https://ollama.com/">Ollama</a>. It allows us to run LLMs locally.</p><p>After installing it, download two models:</p><pre>ollama pull llama3.2<br>ollama pull nomic-embed-text</pre><p>Why do we need two models?</p><ul><li><strong>llama3.2:</strong> Generates the final answer.</li><li><strong>nomic-embed-text:</strong> Converts text into embeddings so that we can search for similar information.</li></ul><p>Now install the Python dependencies:</p><pre>pip install ollama chromadb</pre><p>That’s all we need for this example.</p><h3>3. Create our knowledge file</h3><p>Create a file called knowledge.txt.</p><p>Let’s add some troubleshooting information:</p><pre>A Kubernetes pod can remain in Pending state when<br>the cluster does not have enough CPU or memory resources.</pre><pre>A pod can also remain in Pending state because of<br>node taints and tolerations. Check the node taints<br>and verify whether the pod has matching tolerations.</pre><pre>If a Redis pod is not starting, check the pod events<br>using kubectl describe pod. Also check whether the<br>persistent volume claim is bound correctly.</pre><p>This file is our knowledge base.</p><p>In a real application, this information could come from PDF files, documentation, support tickets or internal troubleshooting guides.</p><p>For now, we will keep it simple and use one text file.</p><h3>4. Build the RAG application</h3><p>Create a Python file called rag.py.</p><pre>import ollama<br>import chromadb</pre><pre># Model configuration<br>LLM_MODEL = &quot;llama3.2&quot;<br>EMBED_MODEL = &quot;nomic-embed-text&quot;</pre><pre># Read our knowledge file<br>with open(&quot;knowledge.txt&quot;, &quot;r&quot;) as file:<br>    text = file.read()</pre><pre># Split the text into simple chunks<br>chunks = [<br>    chunk.strip()<br>    for chunk in text.split(&quot;\n\n&quot;)<br>    if chunk.strip()<br>]</pre><pre># Create a local ChromaDB database<br>client = chromadb.PersistentClient(path=&quot;./rag_db&quot;)</pre><pre>collection = client.get_or_create_collection(<br>    name=&quot;knowledge&quot;<br>)</pre><pre># Convert chunks into embeddings<br>response = ollama.embed(<br>    model=EMBED_MODEL,<br>    input=chunks<br>)</pre><pre>embeddings = response[&quot;embeddings&quot;]</pre><pre># Store chunks and embeddings in ChromaDB<br>collection.upsert(<br>    ids=[f&quot;chunk-{i}&quot; for i in range(len(chunks))],<br>    documents=chunks,<br>    embeddings=embeddings<br>)</pre><pre>print(&quot;Knowledge base loaded successfully!&quot;)</pre><pre># Ask a question<br>question = input(&quot;\nAsk a question: &quot;)</pre><pre># Convert the question into an embedding<br>query_embedding = ollama.embed(<br>    model=EMBED_MODEL,<br>    input=question<br>)[&quot;embeddings&quot;][0]</pre><pre># Retrieve the two most relevant chunks<br>results = collection.query(<br>    query_embeddings=[query_embedding],<br>    n_results=2<br>)</pre><pre>context = &quot;\n\n&quot;.join(results[&quot;documents&quot;][0])</pre><pre># Send the retrieved information to the LLM<br>answer = ollama.chat(<br>    model=LLM_MODEL,<br>    messages=[<br>        {<br>            &quot;role&quot;: &quot;system&quot;,<br>            &quot;content&quot;: &quot;Answer using only the provided context. &quot;<br>                       &quot;If the answer is not in the context, &quot;<br>                       &quot;say you don&#39;t have enough information.&quot;<br>        },<br>        {<br>            &quot;role&quot;: &quot;user&quot;,<br>            &quot;content&quot;: f&quot;&quot;&quot;<br>            Context:<br>            {context}</pre><pre>            Question:<br>            {question}<br>            &quot;&quot;&quot;<br>        }<br>    ]<br>)</pre><pre>print(&quot;\nAnswer:&quot;)<br>print(answer[&quot;message&quot;][&quot;content&quot;])</pre><h3>5. Run the application</h3><p>Make sure Ollama is running, then execute:</p><pre>python rag.py</pre><p>Try asking:</p><pre>Why is my Kubernetes pod stuck in Pending state?</pre><p>The application will:</p><ol><li>Convert your question into an embedding.</li><li>Search ChromaDB for the most relevant information.</li><li>Retrieve the matching text chunks.</li><li>Pass those chunks and your question to the LLM.</li><li>Generate an answer using the retrieved information.</li></ol><p>You should get an answer explaining possible CPU and memory shortages or node taints and tolerations.</p><h3>6. What just happened?</h3><p>Let’s connect the code to the RAG concepts we discussed in the previous article.</p><p>RAG conceptWhat we usedKnowledge sourceknowledge.txtChunkingSplitting text into paragraphsEmbedding modelnomic-embed-textVector databaseChromaDBRetrievalcollection.query()LLMllama3.2Generationollama.chat()</p><p>One important thing to understand is that the LLM does not directly search our text file.</p><p>ChromaDB finds the relevant information and sends it to the LLM as context. The LLM then uses that context to generate the answer.</p><h3>7. Where is the data stored?</h3><p>When we run the application, ChromaDB creates a local folder:</p><pre>rag_db/</pre><p>This folder contains the stored vector database.</p><p>We don’t need to install or manage a separate database server for this example.</p><p>Also, both Ollama models and ChromaDB run locally. Our application does not need to send the knowledge file to an external cloud API.</p><h3>Conclusion</h3><p>That’s it! We have built a basic RAG application using just a few lines of Python.</p><p>Of course, this is only a starting point. A production-ready RAG application would need better document processing, chunking strategies, metadata filtering, access control, improved retrieval and other optimizations.</p><p>But the fundamental idea remains the same:</p><p><strong>Store your knowledge, retrieve the relevant information, and give it to an LLM to generate an answer.</strong></p><p>We have now built that entire flow locally, without using any complicated frameworks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5dac31ce0f4d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAG — Giving LLMs Access to Your Own Knowledge]]></title>
            <link>https://medium.com/@kiranraj_43856/rag-giving-llms-access-to-your-own-knowledge-009330ce5a15?source=rss-b648824a93f1------2</link>
            <guid isPermaLink="false">https://medium.com/p/009330ce5a15</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[Kiran Raj]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 05:55:06 GMT</pubDate>
            <atom:updated>2026-11-12T05:58:53.337Z</atom:updated>
            <content:encoded><![CDATA[<p>We have seen how Large Language Models (LLMs) like GPT, IBM Granite, LLaMA and Mistral can understand and generate human-like text. They are trained on huge amounts of data and can answer questions on almost any general topic.</p><p>But what if you want an LLM to answer questions based on your own data?</p><p>Imagine working in an organization that has thousands of internal documents, technical guides, API documentation and troubleshooting records. Can you simply ask ChatGPT about an internal application and expect it to know everything?</p><p>Probably not. The model may not have access to your organization’s internal data.</p><p>This is where <strong>RAG (Retrieval-Augmented Generation)</strong> comes into the picture.</p><p>In simple terms, RAG allows an LLM to retrieve relevant information from an external knowledge source and use it to generate an answer. Instead of relying only on what the model already knows, we provide additional context while answering the question.</p><p>In this blog, I have covered some of the fundamentals of RAG and how it works.</p><h3>Why do we need RAG?</h3><p>Let’s take a simple example.</p><p>Imagine you are working on a software project. Over time, your team encounters hundreds of issues related to deployments, database connections, API failures and Kubernetes configurations.</p><p>Every time an issue occurs, someone investigates it and finds a solution. These solutions are usually documented in tickets, internal wikis or technical documents.</p><p>A few months later, another developer encounters the exact same issue.</p><p>They start debugging from scratch, search through old tickets and ask teammates for help.</p><p>Wouldn’t it be great if we could simply ask an AI assistant:</p><p><em>“Have we encountered this issue before? If yes, what was the solution?”</em></p><p>This is exactly where RAG can help. We can connect an LLM to our internal knowledge base and allow it to retrieve relevant information whenever someone asks a question.</p><h3>How does RAG work?</h3><p>RAG mainly consists of three steps:</p><ol><li><strong>Retrieval:</strong> Find relevant information from an external knowledge source.</li><li><strong>Augmentation:</strong> Add the retrieved information to the user’s question as context.</li><li><strong>Generation:</strong> Pass this information to an LLM and generate the final response.</li></ol><p>Let’s understand the process.</p><h3>1. Data Collection and Chunking</h3><p>First, we collect the data that we want our LLM to access. This can be PDF documents, Word files, database records, Confluence pages, API documentation or troubleshooting guides.</p><p>Once we extract the text, we split large documents into smaller pieces. This process is called <strong>chunking</strong>.</p><p>For example, a document containing 10,000 words can be split into smaller chunks of around 500 words each.</p><p>We can also attach metadata to each chunk, such as document name, URL, application name and issue category.</p><p>This makes it easier to find the right information later.</p><h3>2. Embeddings</h3><p>We have already discussed embeddings in my previous blog about LLMs.</p><p>Just to recap, embeddings are numerical representations of text. They convert words, sentences or paragraphs into vectors.</p><p>But why do we need them in RAG?</p><p>Imagine a user asks:</p><p><em>“Why is my Kubernetes pod not getting scheduled?”</em></p><p>Our knowledge base contains this information:</p><p><em>“Pod scheduling failure due to node taints and tolerations.”</em></p><p>Although the words are different, both sentences are talking about a similar problem.</p><p>A normal keyword search may not find this information easily.</p><p>This is where embeddings help. We convert both the user’s question and our document chunks into vectors using an embedding model.</p><p>Since embeddings capture semantic meaning, similar sentences tend to have vectors that are closer to each other in vector space.</p><p>This allows us to search based on meaning rather than just matching words.</p><h3>3. Vector Database</h3><p>Now that we have generated embeddings, where do we store them?</p><p>This is where vector databases come in.</p><p>Some popular options are:</p><ul><li>PostgreSQL with pgvector</li><li>Milvus</li><li>Chroma</li></ul><p>A vector database stores embeddings along with the original text and metadata.</p><p>Whenever a user asks a question, we generate an embedding for that question and search the database for similar vectors.</p><p>The database returns the most relevant chunks based on vector similarity.</p><h3>4. Retrieval and Generation</h3><p>Let’s say a developer asks:</p><p><em>“Why is my Model Gateway Redis pod stuck in Pending state?”</em></p><p>The application converts this question into an embedding and searches the vector database.</p><p>It might retrieve information about previous Redis pod failures caused by insufficient node resources, unbound PersistentVolumeClaims or node taints.</p><p>We then combine the original question with the retrieved information and pass everything to an LLM.</p><p>The LLM uses this additional context to generate an answer.</p><p>For example:</p><p>“Your Redis pod could be stuck in Pending state because its PersistentVolumeClaim is not bound to an available PersistentVolume. Check the PVC status and verify whether a matching PersistentVolume is available.”</p><p>And that’s it!</p><p>Instead of relying only on its pre-trained knowledge, the LLM now has access to information from our own knowledge base.</p><p>The complete flow looks something like this:</p><p><strong>User Question → Embedding Model → Vector Database → Relevant Chunks → LLM → Final Answer</strong></p><h3>RAG vs Fine-tuning</h3><p>Are RAG and fine-tuning the same? No, they solve different problems.</p><p><strong>Fine-tuning</strong> involves training an existing model further using additional data to improve its behaviour for a specific task.</p><p><strong>RAG</strong> does not require retraining the model whenever new information becomes available. Instead, it retrieves relevant information from an external knowledge source whenever a question is asked.</p><p>For example, if your organization adds a new troubleshooting document today, you can generate its embeddings and store them in the vector database.</p><p>The next time someone asks a related question, RAG can retrieve this new information.</p><p>You don’t need to train the LLM again.</p><p>This is one of the biggest advantages of RAG for enterprise applications.</p><h3>Limitations of RAG</h3><p>RAG is powerful, but it is not perfect.</p><ol><li><strong>Data Quality:</strong> Incorrect or outdated documents can result in incorrect answers.</li><li><strong>Chunking:</strong> Splitting documents incorrectly can result in loss of important context.</li><li><strong>Retrieval Accuracy:</strong> The vector database may sometimes return information that is similar but not actually relevant.</li><li><strong>Hallucinations:</strong> An LLM can still generate information that is not supported by the retrieved documents.</li><li><strong>Performance:</strong> Embedding generation, vector searches and LLM calls can increase response time.</li></ol><h3>What’s Next? Advanced RAG</h3><p>There are several ways to improve basic RAG:</p><ul><li><strong>Hybrid Search:</strong> Combines keyword search and semantic search.</li><li><strong>Reranking:</strong> Reorders retrieved documents based on relevance.</li><li><strong>Metadata Filtering:</strong> Filters documents using attributes like application name or issue category.</li><li><strong>Query Rewriting:</strong> Improves the user’s original question before searching.</li><li><strong>Agentic RAG:</strong> Uses AI agents to decide when and how to retrieve information.</li></ul><h3>Conclusion</h3><p>RAG is one of the most practical ways to bring Generative AI into existing applications.</p><p>Most organizations already have huge amounts of information stored in documents, databases and internal systems. The problem is finding the right information at the right time.</p><p>RAG provides a way to make all this information accessible through natural language.</p><p>Instead of asking developers to remember every solution or search through hundreds of documents, we can build AI assistants that retrieve relevant information and help solve problems.</p><p>And the interesting part is that we don’t need to train our own LLM to get started.</p><p><strong>LLMs provide the intelligence, vector databases help find the right information, and RAG brings them together.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=009330ce5a15" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Running Ansible, Kubernetes, Docker & Jenkins Locally]]></title>
            <link>https://medium.com/@kiranraj_43856/running-ansible-kubernetes-docker-jenkins-locally-d006e05929e4?source=rss-b648824a93f1------2</link>
            <guid isPermaLink="false">https://medium.com/p/d006e05929e4</guid>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[ansible]]></category>
            <category><![CDATA[argo-cd]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Kiran Raj]]></dc:creator>
            <pubDate>Wed, 19 Nov 2025 17:35:50 GMT</pubDate>
            <atom:updated>2025-11-19T17:35:50.229Z</atom:updated>
            <content:encoded><![CDATA[<p><strong><em>Requirements:</em></strong><em><br>Podman Desktop<br>Minikube → for container orchestration<br>ArgoCd → for GitOps deployment<br>Jenkins → for CI/CD pipelines<br>Docker → for containerising apps <br>registry (by docker)</em></p><p>In this technical blog, I will be covering the practical setup of Kubernetes, Jenkins, and Docker with local orchestration.</p><h3>Kubernetes</h3><p>In Kubernetes, before you deploy your Docker container (Pod/Deployment), you often need to configure the environment it will run in. This configuration can include various resources such as ConfigMaps, Secrets, and ServiceAccounts, which are typically created before deploying your application.</p><p><strong>RBAC on Kubernetes</strong></p><p>There are 3 main components in Kubernetes RBAC. Role, Cluster Role, Role Binding and Cluster Role Binding. These define the permissions that your users, groups or service accounts have in your Kubernetes cluster.</p><p><strong>To install MiniKube</strong></p><blockquote>minikube start — driver=docker –force</blockquote><p>OR</p><p>(not recommended)</p><blockquote>minikube start — driver=podman<br>podman system connection default podman-machine-default-root<br>podman info</blockquote><p>Common kubectl debugging commands:</p><blockquote>kubectl get pods<br>kubectl describe pod &lt;pod-name&gt;<br>kubectl logs &lt;pod-name&gt;</blockquote><h3>ArgoCD</h3><p><strong>To run ArgoCD Locally</strong></p><blockquote>kubectl create namespace argocd<br>kubectl apply -n argocd -f</blockquote><p><a href="https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml">https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml</a></p><blockquote>kubectl port-forward svc/argocd-server -n argocd 8080:443</blockquote><blockquote>kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath=”{.data.password}” | base64 -d; echo</blockquote><p><strong>To access a private repo from argocd</strong></p><blockquote>kubectl config set-context — current — namespace=argocd</blockquote><blockquote>argocd repo add <a href="https://github.ibm.com/kiranraj/argocd-deployment">https://github.com/&lt;</a>repo&gt; — username &lt;username&gt; — password &lt;github_pat_token&gt;</blockquote><h3>Jenkins</h3><p><strong>To Install Jenkins on MiniKube</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5umDXpOWRkh9wdMVbuvVag.png" /></figure><blockquote>kubectl create namespace jenkins<br>eval $(minikube docker-env)<br>docker build -t my-jenkins-with-docker .<br>helm install jenkins jenkins/jenkins -n jenkins -f values.yaml -f jenkins-docker-access.yaml<br>kubectl exec — namespace jenkins -it svc/jenkins — /bin/cat /run/secrets/additional/chart-admin-password<br>kubectl — namespace jenkins port-forward svc/jenkins 49629:8080<br>or<br>minikube service jenkins -n jenkins</blockquote><p># Below command configures your local shell to use the Docker daemon inside Minikube, instead of your host’s Docker.</p><blockquote>eval $(minikube docker-env)</blockquote><h3>Docker</h3><p><strong>To Run a Local Docker Registry</strong></p><p>Use Docker’s Official registry Image to run a local registry, then you can push your images to that registry as shown below.</p><blockquote>docker run -d -p 5000:5000 — name registry registry:2<br>docker tag myimage localhost:5000/myimage<br>docker push localhost:5000/myimage</blockquote><h3>Ansible</h3><p>Ansible doesn’t directly sit inside the CI/CD pipeline like Jenkins or ArgoCD. It is used for:</p><ol><li><strong>Environment Provisioning</strong> (Before Jenkins):<br>Ansible is used to prepare and configure your local or remote environment.<br> For example: Install Docker, Minikube, Jenkins automatically; Set up Kubernetes namespaces, ConfigMaps, and Secrets; Configure Linux servers, users, and permissions.</li><li><strong>Deployment Automation</strong> (After Jenkins):<br>Jenkins can trigger Ansible playbooks after a build to deploy apps, update configurations, or manage services on Kubernetes or VM environments.<br>Example: Jenkins builds the image → pushes to registry → triggers an Ansible playbook that runs kubectl apply.</li><li><strong>Continuous configuration management</strong>: <br>Ansible ensures system and app configurations match desired states across environments (e.g., dev, QA, prod).</li></ol><p><strong>Install Ansible:</strong></p><blockquote>sudo apt update <br>sudo apt install ansible -y <br>ansible — version</blockquote><p>To run a simple playbook to provision a Kubernetes namespace or deploy an app using kubectl</p><blockquote>- name: Deploy app on Minikube<br> hosts: localhost<br> tasks:<br>- name: Create namespace<br> command: kubectl create namespace demo</blockquote><h3>Conclusion</h3><p>These steps help us set up a local environment to enable CI/CD locally to test and develop an environment locally.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d006e05929e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The inner workings of LLMs]]></title>
            <link>https://medium.com/@kiranraj_43856/the-inner-workings-of-llms-cd0135e539e7?source=rss-b648824a93f1------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd0135e539e7</guid>
            <category><![CDATA[agentic-rag]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[llm-agent]]></category>
            <dc:creator><![CDATA[Kiran Raj]]></dc:creator>
            <pubDate>Wed, 19 Nov 2025 17:27:18 GMT</pubDate>
            <atom:updated>2026-07-03T07:06:15.621Z</atom:updated>
            <content:encoded><![CDATA[<h3>LLMs — The Core Intelligence Layer</h3><p>A Large Language Model (LLM) is a type of neural network that is trained to predict the next word or token in a sentence. LLM is the brain behind AI assistants/agents etc. By learning from a huge amount of text, it picks up grammar, meaning, reasoning patterns, and even some problem-solving skills. Some modern LLMs include Meta’s LLaMA 3, OpenAI’s GPT models, IBM Granite, Mistral, Microsoft’s Phi-3, and Google’s Gemma.</p><p>LLMs come in different sizes — from a few billion to hundreds of billions of parameters. Techniques like quantisation and compression allow smaller versions, like Mistral-7B, to run on a personal computer. Tools such as Ollama and LM Studio make it easy to download and run these models locally.</p><p>It’s quite amazing that even though training these models costs billions of dollars and requires huge infrastructure, some of them can now fit on a modern laptop’s hard drive. In this blog I have curated some of the fundamentals of how an LLM work and how it can change our future.</p><h3>Tokens</h3><p>Tokens are the currency of LLMs. When you send any input to an LLM, lets say “Hello”, that gets broken down into tokens. Then the LLM will do some thinking using these tokens and produce some output tokens like “Hi”. The output tokens are built at a different rate than the input tokens. i.e. The charges for token generation would be different for input and output tokens and they would typically be per 1000 tokens. We get the charge for that API call by adding the cost of input token generation and output token generation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4kbFSj6nPpx7tcF8V-jZww.png" /></figure><h3>Embeddings</h3><p>Vectors are a fundamental mathematical concept. Embeddings are vectors that mean something. These embeddings represent anything like words that we use in our daily life.</p><p>Think of the classic word analogy:</p><blockquote><strong>king − man + woman = queen</strong></blockquote><p>We can represent these words as points in a 2D vector space:</p><ul><li><strong>woman</strong> → (9, 0)</li><li><strong>man</strong> → (1, 0)</li><li><strong>queen</strong> → (9, 9)</li><li><strong>king</strong> → (1, 9)</li></ul><p>Now, using vector arithmetic:</p><blockquote>king−man+woman=(1,9)−(1,0)+(9,0)=(9,9)=queen</blockquote><p>In other words, if you take the vector for <strong>king</strong>, subtract <strong>man</strong>, and add <strong>woman</strong>, you land exactly at the vector for <strong>queen</strong>. This illustrates how relationships between words can be captured as geometric transformations in vector space.</p><blockquote>High-dimensional space: each word is a point <br>king — (vector shift) → queen <br>woman — (vector proximity) → queen</blockquote><p>This is great, but we cannot capture the full contextual meaning of a word with just 2 dimensions. Which is why more than often for embeddings to have hundreds of dimensions. This allows them to have complex relationships with other words. ie. This is why words have different meaning based on the neighbouring words in a sentence. Based on the context of your input, the LLMs surf the nth dimension to find the output tokens for you which comes to your screen when you use models like OpenAI, IBM granite etc.</p><h3>LLM Architecture</h3><p>LLMs are based on the Transformer architecture, which uses self-attention to understand context. Even though I am not going to go in detail about the architecture, here are some of the components of LLM’s.</p><p>Components: <br>1. <strong>Attention Heads</strong>: Capture relationships between tokens. <br>2. <strong>Feedforward Layers</strong>: Process token embeddings individually. <br>3. <strong>Layer Normalization</strong>: Stabilizes training. <br>4. <strong>Positional Encoding</strong>: Provides word order information.</p><h3>Training Process</h3><p>LLMs learn by analysing massive amounts of text data and adjusting weights and parameters inside a neural network to understand patterns in language. LLMs are trained using next-token prediction by feeding input text into the model, predict the next token, calculate loss between predicted and actual token, update model parameters using back-propagation.</p><p>Here are the steps:</p><ol><li><strong>Learning process:</strong> The model reads billions of words and tries to predict the next word in a sentence. For example, given “The cat is on the ___,” it learns that “mat” is more likely than “car.” Each time it makes a prediction, it checks if it was right or wrong. When it’s wrong, it adjusts its internal numbers (weights) slightly to do better next time. This process repeats millions of times.</li><li><strong>Weights: </strong>Weights are a specific type of parameter representing the strength of connections between neuron. They are numbers that control how much influence one part of the model has on another. We can think of them like knobs that tune how the model connects ideas. For example, if the model learns that the word “cat” is often linked with “meows,” the weights between those words become stronger. Note that in this case, the words are converted to vectors that have similar patterns like “dog” and “barks”.</li><li><strong>Parameters: </strong>Parameters are all the adjustable parts of the model — mostly these weights (and a few bias terms). So when you hear that “GPT has 175 billion parameters,” it means it has 175 billion little numbers it has learned to fine-tune to understand and generate language. ie. around 175 billion weights + bias.</li></ol><ul><li><strong>Weights:</strong> How strong the connections are.</li><li><strong>Biases:</strong> How easily neurons fire.</li><li><strong>Parameters:</strong> The full set of weights + biases (everything the model learns).</li></ul><p>In short LLMs learn patterns in text by adjusting billions of small values (weights), which act as the model’s memory of how words and ideas connect.</p><h3>Inference: Generating Text</h3><p>LLM inference means using a trained language model to produce an answer or response from a given prompt. The model takes the input text, processes it through its internal layers, and predicts the next words or tokens. These tokens are then turned into readable text. This is how LLMs can answer questions, summarise information, or complete sentences.</p><p>LLMs generate text using token sampling:<br> — Greedy: Chooses the most probable token. <br>— Beam Search: Considers multiple token sequences. <br>— Top-k / Nucleus Sampling: Adds randomness for creativity.</p><h3>Can LLMs self learn?</h3><p>The true difference between an LLM and a human brain is that LLMs are static. They only know what we have already trained them. They predict the next word accurately and pretend to be smart. But in reality they just predict patterns of words based on billions of text that it has been trained on while humans have the ability to understand, learn and grow. We can wake up tomorrow and choose to change, be a better person, learn more etc. While LLMs once trained and deployed, their weights are frozen.</p><p>While there is active research on enabling LLMs to self-learn or self-improve, though it’s still an emerging area, full autonomy is still ambitious.</p><h3>Limitations of LLMs</h3><p>1. <strong>Hallucinations</strong>: May generate incorrect information. <br>2. <strong>Biases</strong>: Inherited from training data. <br>3. <strong>Compute Cost</strong>: Training requires massive resources. <br>4. <strong>Context Length</strong>: Limited by model design.</p><h3>Conclusion</h3><p>LLMs are powerful AI models that understand and generate human-like language by leveraging tokens, embeddings, and transformer architectures. They are becoming increasingly accessible, running even on personal computers while providing enormous capabilities. In essence, an LLM cannot replace human sense, but can be a reflection of it — a vast web of patterns, probabilities, and relationships made from our collective language. It doesn’t think, yet it reasons; it doesn’t feel, yet it speaks as though it does.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cd0135e539e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>