<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ali Issa on Medium]]></title>
        <description><![CDATA[Stories by Ali Issa on Medium]]></description>
        <link>https://medium.com/@aliissa99?source=rss-d3daa151c134------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Tf4evErOaJRbsZAxIIh4rQ.jpeg</url>
            <title>Stories by Ali Issa on Medium</title>
            <link>https://medium.com/@aliissa99?source=rss-d3daa151c134------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 15:16:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aliissa99/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building and Training Large Language Models (LLMs): A Stanford Lecture Summary]]></title>
            <link>https://aliissa99.medium.com/building-and-training-large-language-models-llms-a-stanford-lecture-summary-6b63c431a4b8?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b63c431a4b8</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Tue, 01 Apr 2025 19:03:37 GMT</pubDate>
            <atom:updated>2025-04-01T19:03:37.804Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JXJAlRdmFU2qJbAY" /><figcaption>Image generated by ChatGPT</figcaption></figure><blockquote>This is a summary of the S<a href="https://youtu.be/9vM4p9NN0Ts?si=UIqeYt79TIEL227d">tanford CS229 Machine Learning lecture</a> : Building Large Language Models (LLMs) by Yann Dubois</blockquote><h3>Introduction to Large Language Models (LLMs)</h3><p>Large Language Models (LLMs), such as ChatGPT, Claude, Gemini, and LLaMA, have become fundamental tools in modern machine learning applications. Understanding how these models are built and trained involves several crucial components, including architecture, training algorithms, data, evaluation methods, and system considerations.</p><h3>Essential Components in Training LLMs</h3><p>When building an LLM, several critical aspects must be considered:</p><h4><strong>Architecture</strong></h4><p>Most LLMs use transformer-based architectures. It is crucial to understand how the model structure impacts training, evaluation, and practical use.</p><h4><strong>Data</strong></h4><p>Data quality and diversity significantly affect model performance. Typically, data is downloaded from the internet using standard crawling methods like Common Crawl. Once obtained, the text must be extracted from HTML pages, including handling special cases like mathematical content.</p><p><strong>Data processing includes:</strong></p><ul><li>Filtering undesirable content, such as harmful information and personally identifiable information (PII), which can often be identified using small classification models.</li><li>Duplicate content, such as recurring headers and footers or identical content from different URLs, must be removed to avoid redundancy.</li><li>Heuristic filtering further refines the dataset by removing low-quality documents based on simple rules . For example, eliminating documents that are too short or excessively long.</li><li>Model-based filtering is another effective method, where small models predict the likelihood of a webpage being referenced by Wikipedia for example (if something referenced by wikipedia, it might be a good website).</li><li>Data mixtures are also crucial, involving weighting different content types (such as code, books, entertainment) appropriately.</li><li>Finally, learning rate annealing on high-quality (it’s like overfitting on high quality data).</li></ul><h4>Tokenization</h4><p>Tokenizers convert text into a sequence of tokens that models can process. A simple approach is to treat each word as a token .But this breaks down when there are typos or out-of-vocabulary (OOV) words, which the model wouldn’t recognize. On the other end, character-level tokenization handles typos and OOV issues better but results in very long sequences, which can be inefficient for training.</p><p>Subword-level tokenization, like Byte Pair Encoding (BPE), offers a balance between these extremes. BPE starts with individual characters and iteratively merges the most frequent pairs to form new tokens. You can control how many merges the tokenizer applies, effectively customizing the vocabulary size. This approach handles typos more gracefully than word-level tokenization and keeps sequence lengths shorter than character-level methods.</p><p>Tokenizers are trained independently from the LLM by feeding them text, the algorithm learns which token pairs to merge based on frequency.</p><blockquote>Pre-tokenization is often performed beforehand, such as splitting text by spaces or punctuation, to improve consistency and efficiency.</blockquote><h4><strong>Training Algorithm and Loss Function</strong></h4><p>The training process of an LLM consists of the following steps: a sentence is tokenized, passed through the model, producing logits, which are then converted into a probability distribution (containing the probabilities of each token in the LLM’s vocabulary). The most likely token is retrieved, and the loss is computed. The primary loss function for LLMs is typically the cross-entropy loss.</p><h4><strong>Evaluation</strong></h4><p>Traditionally, LLMs were evaluated using <strong>perplexity</strong>, which measures how confidently a model predicts the next token. Perplexity values typically range between 1 and the size of the vocabulary, representing how many tokens the model is “hesitating” between on average.</p><blockquote>For more info about perplexity feel free to check my recent<a href="https://www.linkedin.com/posts/aliissa99_artificialintelligence-llms-nlp-activity-7295160342010093568-ex-z?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAACkQ3MUB2S21Nlpy3ztq_SXSjXMUhGDC000"> post</a> on LinkedIn</blockquote><p>However, perplexity has limitations. Although averaging the loss makes it independent of sentence length, longer sentences tend to accumulate higher loss, while shorter ones yield lower loss.This will lead to misleading comparisons. Additionally, perplexity is sensitive to the tokenizer and vocabulary size used. For instance, ChatGPT uses a vocabulary size of 50,000, whereas Gemini uses 10,000. These differences make perplexity unreliable for fair benchmarking across models.</p><p>As a result, <strong>modern evaluation</strong> has shifted away from perplexity, especially in academic benchmarks.</p><p><strong>Evaluation challenges:</strong></p><ul><li><strong>Prompt sensitivity and loss inconsistencies</strong>: Since pre-trained models are not always instruction-following, evaluating tasks like multiple-choice questions requires special techniques. We can’t directly prompt these models to choose the correct answer because they haven’t been trained to follow instructions yet. Instead, one common approach is to use the model’s probability distribution to determine which option A, B, C, or D has the highest likelihood, assuming the full content of each choice is included in the input.</li><li><strong>Training-test contamination</strong>: Unlike companies who know their training data, the public lacks access to proprietary datasets. To detect potential data leakage in closed models, evaluators may check whether the model can reproduce test examples exactly, suggesting it was trained on them.</li><li><strong>Evaluation inconsistencies</strong>: Different LLM evaluators (like HELM and Harness) use varying strategies, which can result in inconsistent measurements across tools.</li></ul><h4><strong>Systems Considerations</strong></h4><p>Deploying LLMs in real-world applications involves handling large-scale computations, data pipeline efficiency, and optimized hardware usage.</p><h3>Scaling Laws</h3><p><a href="https://arxiv.org/pdf/2001.08361">Research by OpenAI</a> has shown that increasing both the model size and the amount of training data typically leads to better performance.</p><blockquote>Note that larger models tend not to overfit in the traditional sense.</blockquote><p>This relationship is formalized through <strong>scaling laws</strong>, which estimate how performance improves as a function of model size, dataset size, and compute. These laws guide decisions like: given a certain GPU budget, which model architecture or size should be trained?</p><p>Historically, hyperparameter tuning was performed on large models. Training a new one each day(for example) and selecting the best after weeks of experimentation.</p><p>Modern pipelines are more efficient: they tune hyperparameters on smaller models, fit scaling laws to the results, and then extrapolate the expected performance of larger models. This allows researchers to reserve most of the budget (e.g., 27 out of 30 days) for training the final large-scale model, using only a small portion (e.g., 3 days) for tuning.</p><p>An example of this approach is comparing LSTMs and Transformers. By training various sizes of each architecture and fitting scaling laws, researchers can determine which is more effective under a fixed compute budget.</p><p>The <a href="https://arxiv.org/abs/2203.15556"><strong>Chinchilla paper</strong></a> further refined this understanding by fixing total compute and training models of different sizes on varying amounts of data. It found that for optimal performance, models should be trained with around <strong>20 tokens per parameter</strong>. If inference cost isn’t considered, this ratio increases to about <strong>150 parameters per token</strong>.</p><h3>Post-Training Alignment and Fine-Tuning</h3><p>Raw language models typically do not follow instructions and can generate harmful or toxic content. Thus, alignment through supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) is essential.</p><h4><strong>Supervised Fine-Tuning (SFT)</strong></h4><p>It is the first step in post-training, where a language model is trained to generate responses in a desired format using labeled input-output pairs. These labels can come from either <strong>human-annotated</strong> data or <strong>synthetically generated</strong> examples. While human-labeled data is high-quality, it’s expensive and time-consuming to collect at scale. To address this, researchers use <strong>synthetic data generation. </strong>It<strong> </strong>leverages smaller human annotated datasets to train models (like Alpaca 7B) that can then generate large-scale instruction-following examples.</p><p>The goal of SFT is not to teach the model new knowledge but to fine-tune it to produce outputs that align with expected behavior and formattin, essentially <strong>behavior cloning</strong>. According to the <a href="https://arxiv.org/abs/2305.11206">LIMA paper</a>, scaling up the amount of SFT data beyond a certain point doesn’t significantly improve performance, reinforcing the idea that relatively small datasets are sufficient for effective SFT.</p><p>One potential drawback is that SFT can introduce <strong>hallucinations</strong>, where the model confidently outputs incorrect information or harmful content.</p><p>This leads to the next step in post-training: <strong>Reinforcement Learning from Human Feedback (RLHF)</strong>, which goes beyond behavior cloning by directly optimizing for human preferences rather than just reproducing human behavior.</p><h4><strong>Reinforcement Learning with Human Feedback (RLHF)</strong></h4><p>RLHF further refines the model by maximizing human preferences rather than simply mimicking labeled data. It involves training a reward model to evaluate model outputs based on human judgments, and then optimizing the LLM using methods like Proximal Policy Optimization (PPO) or Direct Preference Optimization (DPO).</p><blockquote>Note that <strong>DPO is a simplified approach</strong> that directly uses labeled preference data without the need to train a separate reward model. In contrast, PPO requires a reward model, which is trained on human-annotated preferences (or synthetically generated comparisons), to guide the optimization process.</blockquote><p>RLHF helps mitigate hallucinations and aligns outputs more closely with human expectations but is computationally expensive and challenging due to ethical considerations and annotator variability.</p><h3>Evaluation Challenges in Post-Training</h3><p>Evaluating fine-tuned models with standard metrics like perplexity or validation loss can be misleading since post-training methods optimize models differently than standard language modeling. Instead, comparative benchmarks such as Chatbot Arena (human-based comparisons) and AlpacaEval (LLM-based comparisons) have become popular.</p><p>Curious about the latest in AI and ML? Follow me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a> for fresh insights and updates!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b63c431a4b8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Reasoning in LLMs]]></title>
            <link>https://aliissa99.medium.com/understanding-reasoning-in-llms-1a92299329bf?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a92299329bf</guid>
            <category><![CDATA[deepseek-r1]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[large-reasoning-models]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Tue, 25 Feb 2025 21:41:09 GMT</pubDate>
            <atom:updated>2025-02-25T21:42:49.769Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1aDGDNcy71KGR8ZG" /><figcaption>Image generated by ChatGPT</figcaption></figure><blockquote>These notes are based on Sebastian’s article about Understanding Reasoning LLMs from his newsletter. It offers valuable insights explained in simpler terms. I highly recommend checking out this newsletter.</blockquote><blockquote><a href="https://magazine.sebastianraschka.com/">https://magazine.sebastianraschka.com/</a></blockquote><h3>Introduction</h3><p>Reasoning refers to the ability of models to solve problems that require deep analysis before arriving at the correct answer. For simple problems like “5+10”, minimal reasoning is needed. However, more complex tasks, such as solving puzzles, require multiple layers of thought and reasoning. Currently, reasoning models are mostly designed to tackle complex problems, going beyond straightforward computations.</p><h3>Types of Intermediate Steps in Reasoning Models</h3><p>Reasoning in LLMs can be broken down into two approaches:</p><ol><li><strong>Explicit Reasoning</strong>: The reasoning chain is directly shown in the response.</li><li><strong>Implicit Reasoning</strong>: Some models, like OpenAI’s models, run multiple iterations internally without revealing the intermediate steps to the user.</li></ol><h3>When to Use Reasoning Models?</h3><p>Reasoning models tend to be more expensive and slower, so it is important to choose when to apply them in a system. For example, it wouldn’t make sense to route a simple sentence like “hello” to a reasoning model. A solution could be implementing a complexity-based LLM router, which directs queries to the appropriate model depending on their complexity.</p><h3>DeepSeek Training Pipeline</h3><p>DeepSeek has introduced a range of models, each progressively refined for improved reasoning capabilities:</p><ul><li><strong>DeepSeek-R1-Zero</strong>: Built on the pre-trained DeepSeek-V3 model, it was trained using reinforcement learning (RL) without supervised fine-tuning (SFT), also known as a cold-start approach.</li><li><strong>DeepSeek-R1</strong>: This model builds upon R1-Zero by adding further SFT stages and reinforcement learning, improving the reasoning capabilities.</li><li><strong>DeepSeek-R1-Distill</strong>: This model fine-tunes smaller models (Llama 8B and 70B, Qwen 1.5B–30B) using outputs from the larger DeepSeek-R1 671B model, enhancing their reasoning capabilities.</li></ul><h3>Four Key Methods to Build and Improve Reasoning Models</h3><ol><li><strong>Inference-Time Scaling</strong></li></ol><ul><li>In simple terms, inference-time scaling increases computing resources during inference. Techniques include:</li><li><strong>Prompting LLMs with Chain-of-Thought (CoT)</strong>: This helps provide more context, leading to higher quality responses.</li><li><strong>Generating Multiple Responses</strong>: LLMs can generate several responses and select the best one.</li><li><strong>Beam Search</strong>: Tracks the top-k most likely sequences at each step and uses a verifier to select the best answer.</li><li><strong>Lookahead Search</strong>: Similar to beam search but includes a rollout mechanism to anticipate possible future tokens.</li></ul><p><strong>2. Pure Reinforcement Learning (RL)</strong></p><ul><li><strong>DeepSeek-R1-Zero</strong> was trained using RL alone, without SFT. The reward models used are:</li><li><strong>Accuracy Reward</strong>: Utilizes a logic compiler to evaluate math responses.</li><li><strong>Format Reward</strong>: Evaluates responses based on the expected format, assessed by an LLM evaluator.</li></ul><p><strong>3. Supervised Fine-Tuning (SFT) + RL</strong></p><ul><li><strong>DeepSeek-R1</strong> integrates RL with SFT. The multi-stage training process includes:</li><li><strong>Cold-start SFT</strong>: Data generated by the unsupervised DeepSeek-R1-Zero.</li><li><strong>Instruction Fine-Tuning</strong>: Fine-tuned using CoT and knowledge-based SFT examples generated by DeepSeek-V3.</li><li><strong>Reinforcement Learning</strong>: Further RL with verifiable rewards (math and coding) and human preference labels for other tasks.</li><li><strong>Final SFT</strong>: Instruction fine-tuning with updated CoT examples, boosting performance significantly over the DeepSeek-R1-Zero model.</li></ul><p><strong>4.Pure SFT and Distillation</strong></p><ul><li><strong>Distillation</strong> involves training smaller models (e.g., Llama 8B, 70B, Qwen 2.5B) with an SFT dataset generated by larger models like DeepSeek-V3 and DeepSeek-R1. The distillation process focuses on improving the reasoning abilities of smaller models by leveraging the larger models’ knowledge.</li></ul><h3>Conclusion</h3><p>The field of reasoning in LLMs is rapidly evolving. Through techniques such as inference-time scaling, reinforcement learning, and distillation, models like DeepSeek are pushing the boundaries of what’s possible. Understanding these methods is crucial for developing robust reasoning systems.</p><p>🔗 Want to learn more? Follow me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a> for regular insights into AI and machine learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a92299329bf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Key Takeaways from the MLOps Data Lifecycle in Production Course]]></title>
            <link>https://aliissa99.medium.com/key-takeaways-from-the-mlops-data-lifecycle-in-production-course-ae631f4b001d?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/ae631f4b001d</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Wed, 15 Jan 2025 20:54:10 GMT</pubDate>
            <atom:updated>2025-01-15T20:54:10.101Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hFka-V-_kdUhxCEn" /><figcaption>Generated by DALL-E</figcaption></figure><blockquote>As I revisit the Machine Learning Data Lifecycle in Production course (a refresher before diving into the third course), I’d like to share some notes that i took while reviewing the slides.</blockquote><h4>𝐑𝐞𝐬𝐞𝐚𝐫𝐜𝐡 𝐌𝐋 𝐯𝐬. 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐌𝐋</h4><ul><li>In research, the focus is on achieving high-accuracy models by fine-tuning hyper parameters to perfection.</li><li>In production, cost-efficiency becomes critical. Models need to adapt to new incoming data and balance accuracy with scalability and resource constraints.</li></ul><h4>𝐁𝐞𝐬𝐭 𝐏𝐫𝐚𝐜𝐭𝐢𝐜𝐞𝐬 𝐢𝐧 𝐌𝐋 𝐂𝐨𝐝𝐞</h4><p>Machine learning systems, like any well-designed software, should follow these principles:</p><ul><li>Scalability: Handle growing data and workloads effectively.</li><li>Extensibility: Easily add new features.</li><li>Configuration: Simplify environment and parameter management.</li><li>Consistency &amp; Reproducibility: Ensure results can be replicated.</li><li>Safety &amp; Security: Defend against potential attacks or failures.</li></ul><h4>𝐌𝐋 𝐏𝐢𝐩𝐞𝐥𝐢𝐧𝐞𝐬</h4><ul><li>ML Pipelines are directed acyclic graphs (DAGs) defining the sequence of tasks, their relationships, and dependencies.</li><li>The key steps in an ML pipeline include defining the project, preparing the data, training the model, and deploying the model.</li><li>Orchestration Frameworks: Airflow, Argo, Celery, Luigi, and Kubeflow are essential for managing pipeline execution.</li></ul><h4>𝐓𝐅𝐗 𝐟𝐨𝐫 𝐄𝐧𝐝-𝐭𝐨-𝐄𝐧𝐝 𝐌𝐋 𝐏𝐢𝐩𝐞𝐥𝐢𝐧𝐞𝐬</h4><ul><li>In the course, TensorFlow Extended (TFX) was used for building ML pipelines.</li><li>It allows to generate data statistics and schemas (data types, ranges, etc.), perform data transformations for preprocessing. Also It manage straining, evaluation, and deployment.</li></ul><h4>𝐃𝐚𝐭𝐚 𝐪𝐮𝐚𝐥𝐢𝐭𝐲</h4><ul><li>When working with data, it’s essential to translate user needs into actionable data requirements by identifying what type of data is necessary to meet their expectations.</li><li>Key considerations include determining the quantity and type of data available, understanding how frequently new data is generated or updated, and assessing whether the data is labeled. If annotations are missing, evaluate the effort and cost required to label it.</li><li>Consistency throughout the data lifecycle is important, as inconsistencies can severely impact the accuracy and reliability of results.</li></ul><h4>𝐌𝐚𝐱𝐢𝐦𝐢𝐳𝐢𝐧𝐠 𝐏𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐯𝐞 𝐒𝐢𝐠𝐧𝐚𝐥𝐬</h4><ul><li>Understanding users and translating their needs into data problems is crucial to designing robust ML systems.</li><li>Identify features with high predictive value and remove those that add noise.</li><li>Feature engineering enhances the quality of the dataset, while feature selection measures where the predictive information lies.</li></ul><p>🔗 Want to learn more? Follow me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a> for regular insights into AI and machine learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae631f4b001d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Raw Data to Model Efficiency: Mastering Feature Engineering and Selection]]></title>
            <link>https://aliissa99.medium.com/title-from-raw-data-to-model-efficiency-mastering-feature-engineering-and-selection-250a418dca6c?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/250a418dca6c</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mlops]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Fri, 06 Dec 2024 07:08:39 GMT</pubDate>
            <atom:updated>2024-12-06T07:08:53.683Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vZwkYkwCGmlBwChir9zpkw.png" /><figcaption>Generated by GPT-4o</figcaption></figure><blockquote>In the world of machine learning, your model is only as good as the data it learns from. Transforming raw data into a structured, meaningful format can make all the difference in model performance. Two essential processes for this transformation are <strong>feature engineering</strong> and <strong>feature selection</strong>. This article draws on key concepts from the<strong> </strong><a href="https://www.coursera.org/learn/machine-learning-data-lifecycle-in-production"><strong>MLOps course</strong></a><strong> by DeepLearning.ai</strong> to explore these critical steps in machine learning pipelines.</blockquote><h3>Feature Engineering: Transforming Raw Data for Better Models</h3><p>Feature engineering focuses on transforming raw data into a format that maximizes model performance and resource efficiency. Here are the key steps and techniques involved:</p><h4>1️. Consistency in Processing</h4><p>The transformations applied during training — such as normalization or rescaling — must also be applied during serving. These steps ensure the model performs consistently across different stages.</p><h4>2️. Transforming Raw Data</h4><ul><li><strong>Categorical features</strong> (e.g., strings) need to be converted into numerical formats like one-hot encoding.</li><li><strong>Numerical transformations</strong>, such as converting integers to floats.</li><li><strong>Text processing</strong>, including stemming and lemmatization.</li><li><strong>Image preprocessing</strong>, such as cropping or resizing.</li></ul><h4>3️. Popular Feature Engineering Techniques</h4><ul><li><strong>Scaling, Normalization, and Standardization</strong>: Adjust numerical ranges to improve model sensitivity.</li><li><strong>Bucketing</strong>: Group continuous values into discrete intervals (e.g., converting hours into labeled time ranges).</li><li><strong>Dimensionality Reduction</strong>: Techniques like PCA, t-SNE, and UMAP simplify high-dimensional data.</li><li><strong>Feature Crossing</strong>: Combine existing features to create new ones, capturing nonlinear relationships.</li><li><strong>Encoding Categorical Variables</strong>: Transform strings into numerical forms for model training.</li></ul><h3>Feature Selection: Identifying the Most Valuable Features</h3><p>Feature selection ensures that only the most relevant and representative features are included in your dataset. This reduces computational costs and prevents overfitting.</p><h4>Types of Feature Selection</h4><ol><li><strong>Unsupervised Methods:</strong>Focus on removing redundant or irrelevant features without considering target relationships.</li></ol><p>2️. <strong>Supervised Methods: </strong>Utilize the relationship between features and the target variable to identify significant contributors.</p><h4>Supervised Feature Selection Techniques</h4><p>1️. <strong>Filter Methods</strong></p><ul><li><strong>Pearson Correlation</strong>: Measures linear relationships between features and the target.</li><li><strong>Univariate Feature Selection</strong>: Evaluates individual features for relevance.</li><li>Other techniques include Mutual Information, F-Test, and Chi-Squared tests.</li></ul><p><strong>2. Wrapper Methods</strong></p><ul><li><strong>Forward Elimination</strong>: Start with one feature, then iteratively add more until performance stabilizes.</li><li><strong>Backward Elimination</strong>: Begin with all features and remove them one by one.</li><li><strong>Recursive Feature Elimination (RFE)</strong>: Rank features using a model (e.g., Random Forest), then iteratively discard the least important ones.</li></ul><p>3️. <strong>Embedded Methods</strong></p><ul><li>Use feature importance scores from models like Random Forest or LASSO to rank and discard less significant features.</li></ul><h4>Conclusion</h4><p>Whether it’s transforming raw data or narrowing down the most valuable features, mastering these processes is crucial for building efficient, high-performing machine learning models. Both feature engineering and feature selection pave the way for cleaner data and smarter models, optimizing both resource usage and accuracy.</p><blockquote><em>These insights were compiled from the</em><a href="https://www.coursera.org/learn/machine-learning-data-lifecycle-in-production"><em> MLOps course</em></a><em> by DeepLearning.ai, an invaluable resource for understanding end-to-end machine learning workflows.</em></blockquote><p>🔗 Want to learn more? Follow me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a> for regular insights into AI and machine learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=250a418dca6c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enhancing Large Language Models: The Power of Continued Pre-Training]]></title>
            <link>https://aliissa99.medium.com/enhancing-large-language-models-the-power-of-continued-pre-training-64d6cc844656?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/64d6cc844656</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[pre-training]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Sun, 17 Nov 2024 15:48:03 GMT</pubDate>
            <atom:updated>2024-11-17T15:48:03.737Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bsWiBMcCxGWlLY7E" /><figcaption>Generated by DALL-E 3</figcaption></figure><h4>𝐖𝐡𝐚𝐭 𝐢𝐬 𝐜𝐨𝐧𝐭𝐢𝐧𝐮𝐞𝐝 𝐩𝐫𝐞-𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠</h4><p>Pre-training an LLM can begin with initial weights that are not random but were originally used to pre-train the model on different data. This allows us to leverage the knowledge gained from the initial pre-training rather than starting from scratch. It could be an efficient way to keep models current and adaptable to new domains without retraining from scratch. This is crucial for incorporating the latest information and insights into our models.</p><h4>𝐄𝐱𝐩𝐞𝐫𝐢𝐦𝐞𝐧𝐭</h4><p>A recent paper highlighted three different approaches:<br>1. Regular pretraining on an initial dataset (D1).<br>2. Continued pretraining by further training an already pretrained model on a new dataset (D2).<br>3. Retraining from scratch on the combined datasets (D1 + D2).</p><h4>𝐊𝐞𝐲 𝐅𝐢𝐧𝐝𝐢𝐧𝐠𝐬</h4><p>- Continued pre-training is 2× cheaper than full retraining but achieves comparable results.<br>- Adding a small fraction (as low as 0.5–5%) of the original data helps prevent 𝐜𝐚𝐭𝐚𝐬𝐭𝐫𝐨𝐩𝐡𝐢𝐜 𝐟𝐨𝐫𝐠𝐞𝐭𝐭𝐢𝐧𝐠, keeping past knowledge intact.<br>- Re-warming and re-decaying the learning rate: To maintain performance in continued pre-training, the learning rate is reintroduced with a warmup phase, followed by decay — matching the original pre-training schedule. This approach preserves the model’s stability and ensures efficient adaptation to new data.</p><blockquote>This summary was inspired by <a href="https://www.linkedin.com/in/sebastianraschka/">Sebastian Raschka, PhD</a>, whose newsletter offers valuable insights presented in a clear, organised way. He also recently published a book titled Build a Large Language Model (From Scratch).</blockquote><p>There’s plenty to explore and learn, but taking small steps each day to absorb a bit of information can lead to significant progress over weeks and months.</p><p>If you like what you see, hit the follow button! You can also find me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a>, and we can follow each other there too. 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64d6cc844656" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reproducing GPT-2 124M: Key Insights from Andrej Karpathy’s 4-Hour Deep Dive]]></title>
            <link>https://aliissa99.medium.com/reproducing-gpt-2-124m-key-insights-from-andrej-karpathys-4-hour-deep-dive-3952046c5d31?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/3952046c5d31</guid>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[pre-training]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[pretrained-model]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Tue, 10 Sep 2024 20:25:39 GMT</pubDate>
            <atom:updated>2024-09-10T20:25:39.778Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5wOltkLOkN2zHpkj" /><figcaption>Generated by DALL-E 3</figcaption></figure><blockquote>This is a summary of Andrej karpathy’s video about pre-training a GPT-2 124M parameter model from scratch.Feel free to check it using this <a href="https://www.youtube.com/watch?v=l8pRSuU81PU">link</a></blockquote><h3>General notes</h3><ul><li>Andrej Karpathy noted that the code published by OpenAI for GPT-2 was not designed for training and was written in TensorFlow. Preferring PyTorch, he utilized the Hugging Face implementation, available on <a href="https://medium.com/transformers/src/transformers/models/gpt2%20at%20main%20%C2%B7%20huggingface/transformers%20(github.com)">GitHub</a>.</li><li>GPT-2 is a modified version of the original Transformer, distinguished by the absence of an encoder and the encoder-decoder multi-head attention mechanism.</li></ul><h3>Architectural notes</h3><ul><li><strong>Reshuffled layer normalization</strong>: Layer norms are positioned before the feed-forward and multi-head attention layers, with an additional layer norm after these components.</li><li><strong>GELU activation: </strong>Unlike ReLU, GELU doesn’t have a flat tail at 0 and allows for non-zero values below 0, demonstrating improved performance in both GPT-2 and BERT papers.</li><li><strong>AdamW optimizer:</strong> Preferred over SGD for better optimization.</li><li><strong>Residual connection scaling:</strong> A scaling factor is applied to reduce standard deviation increase caused by residual connections, calculated by dividing by the square root of the number of residual connections.</li></ul><blockquote>Device management: Models use .to(device), while tensors require explicit assignment (tensor = tensor.to(device)).</blockquote><h3>Optimizations</h3><h4>Hardware and Precision Optimizations</h4><ul><li>Most numbers in the architecture use 32 bits.</li><li>INT8 has uniform spacing.</li></ul><p><strong>Tensor cores</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/904/1*CWbzvxCunR778ecXthawJg.png" /><figcaption>Image taken from the video</figcaption></figure><ul><li>It’s An instruction in the A100 architecture(the one used by Andrej for training)that performs 4x4 matrix multiplication.</li><li>When a matrix multiplication is provided, we also specify the intermediate precision.</li><li>For a given matrix multiplication, it divides matrices into 4x4 matrices and performs multiplication.</li><li>TF-32 reduces precision internally when performing matrix multiplication, but the result remains the same.</li></ul><blockquote>60% utilization is considered good. Most GPUs are memory-bound; we can’t feed them data fast enough, so tensor cores are often idle due to data unavailability.</blockquote><blockquote>Lower precision allows faster access and more storage.</blockquote><blockquote>When a floating point is being represented, we have sign to specify if number is positive or negative), the exponent responsible to set the range, and precision determines the accuracy of numbers.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-9gJxfXuGoFUqqmd2lFAVw.png" /><figcaption><a href="https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf">Source</a></figcaption></figure><ul><li><strong>BF16:</strong> Same range as TF32 but with less precision.</li><li><strong>FP16:</strong> Doesn’t have the same range (gradient scalers problem) but higher precision.</li></ul><h3>Software Optimizations</h3><h4>torch.autocast</h4><p>Used for mix precision, chooses which operations should be converted to BF16 and which should stay as TF32, depending on precision sensitivity. It allows faster and longer runs, helping mitigate precision issues.</p><p>For more info: <a href="https://pytorch.org/tutorials/recipes/recipes/amp_recipe.html#adding-torch-autocast">Link</a></p><h4>torch.compile</h4><p>while all calculations occur on the GPU, most of its memory resides in <strong>High Bandwidth Memory (HBM)</strong>, with only a small amount of faster memory on the GPU chip itself.</p><p>This chip contains <strong>Streaming Multiprocessors (SMs) </strong>that contains tensor cores for computations. Due to limited on-chip memory, data frequently travels between the chip and HBM, which can be time-consuming. <strong>torch.compile</strong> makes code significantly faster by reducing Python overhead and GPU read/writes. It analyzes the code to optimize operations, removing it from the Python interpreter’s sequential execution.</p><p>It addresses this issue through techniques like <strong>kernel fusion</strong>, which allows larger chunks of data to be processed on-chip before sending results back to HBM, thereby reducing round trips and bandwidth usage.</p><p>This optimization is particularly important because GPU-to-memory transfers are time-intensive, and accessing disk data requires routing through the CPU, which is computationally expensive.</p><h4>Flash-attention</h4><p>A kernel fusion operation that torch.compile can’t identify. It performs more FLOPs than the original implementation, applied to matrix multiplication, dropout, softmax, and mask operations.</p><h3>Code optimization</h3><ul><li>Check for “ugly numbers” because CUDA kernels are written in terms of powers of 2. For example, increasing vocab size to the nearest power of 2 can increase FLOPS.</li><li>In the code, GPT-2 vocab size was increased from 50,257 to 50,304 for optimization reasons. Why? CUDA uses block tiles in powers of 2. When computations don’t fit these tiles, additional operations are required, increasing execution time.</li><li>GPT-2’s original vocabulary size was 50,257, but it has been increased to 50,304 because this number is divisible by 2, 4, 8, 16, 32, and 64. This adjustment optimizes performance, particularly for CUDA operations. However, the tokenizer will never generate indices for these additional numbers because it’s only trained to produce indices up to 50,257.</li><li>Consequently, the prediction probabilities for these extra tokens will never be used. The network should learn that these tokens have a very low probability, so their logits should approach negative infinity. In practice, log(-inf) equals 0.</li></ul><h3>Hyperparameter optimization</h3><h4><strong>Adam optimizer</strong></h4><p>Parameters were taken from GPT-3 (not mentioned in GPT-2).</p><h4><strong>Learning rate scheduler</strong></h4><p>cosine decay with warmup.</p><h4><strong>Weight decay</strong></h4><p>not applied to biases, one-dimensional vectors and layer norm (only to embeddings and matrix multiplications).</p><h4><strong>Gradient accumulation</strong></h4><p>Is a technique used for handling large batch sizes in language model training. Here’s an explanation with an example:</p><ul><li>B<strong>atch size: </strong>When we say “0.5M”, it means 0.5 x 10⁶ (500,000) tokens in each batch.</li><li>Due to memory constraints, we can’t process this large batch size at once. Instead, we use gradient accumulation: we process multiple smaller batches, accumulating their gradients until we reach the desired total batch size. Only then do we perform a single parameter update.</li><li>The formula for gradient accumulation steps is: <em>GRAD_accum = total_batch_size // (B * T) Where B is the batch size per GPU and T is the sequence length.</em></li><li>Example: If we’re using a batch size of 16 and sequence length of 1024, we’re processing 16 x 1024 tokens per forward-backward pass. To reach our target of 0.5M tokens per update, we need to accumulate gradients for:</li><li>0.5M / (16 * 1024) ≈ 30 steps</li><li>Important note on loss calculation: When accumulating gradients, be careful with loss functions like MSE. Since we’re accumulating losses over multiple steps, we should not use the mean for each individual accumulation step. Instead, sum the losses and take the mean only after all accumulation steps</li><li>This technique allows us to effectively train with large batch sizes even when limited by GPU memory.</li></ul><h3>Training Process</h3><blockquote>Distributed Data Parallel (DDP) for multi-GPU training was used</blockquote><h4>Data Used</h4><p>The dataset utilized is <a href="https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu"><strong>FineWeb Edu</strong></a><strong>,</strong> a high-quality common crawl dataset provided by Hugging Face.</p><h4>Steps vs Epochs</h4><p>A step refers to a single gradient update, while an epoch is a full pass through the entire dataset.</p><h3>Model Evaluation</h3><p>The evaluation metric is based on <strong>HellaSwag</strong>, a multiple-choice problem. For small language models like GPT-2, a direct answer selection is difficult. The solution involves creating a batch of 4 sentences. Each option in this batch has its own context and option, and computations are performed accordingly.</p><blockquote>Additional notes taken from the video</blockquote><blockquote><strong>- Gradient Norm Clipping:</strong> This technique is employed to prevent the model from experiencing large gradient updates.</blockquote><blockquote><strong>-Document Permutation:</strong> It’s beneficial to shuffle the documents during each epoch to avoid repetitive training on the same order of documents, which could introduce periodicity into the model. The order of documents should not affect the model’s performance.</blockquote><p>Certainly, much more information was covered in the 4-hour video, particularly regarding the code. Feel free to ask any questions or suggest any additional details you think would be valuable to include in the article.</p><p>If you like what you see, hit the follow button! You can also find me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a>, and we can follow each other there too. 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3952046c5d31" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi-Agent System with Crew AI: A Short Course Summary]]></title>
            <link>https://aliissa99.medium.com/multi-agent-system-with-crew-ai-a-short-course-summary-c19dafe27e20?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/c19dafe27e20</guid>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Thu, 04 Jul 2024 19:24:51 GMT</pubDate>
            <atom:updated>2024-07-04T19:24:51.086Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ACbDK_bLwS7QsjgZ" /><figcaption>Generated by DALL E-3</figcaption></figure><blockquote><strong>Please note that this article contains direct quotes content from the short course on multi-agent systems with Crew AI.</strong></blockquote><h3><strong>Introduction</strong></h3><p>Crew AI is a multi-agent framework built on LangChain, aiming to empower engineers to harness the collective power of AI agents. We can run different types of agents (GPT-4, LLaMA, fine-tuned models) and deploy them to production.</p><blockquote>We will talk later about Crew AI components other than the agents, like tasks, crew, and tools. But now, let’s start by defining agents, their components in general, and how to build a multi agent system.</blockquote><p>In the course, they mentioned that working on software is different from working with LLMs. In software, you need to precisely define the input, its type, and the output, whereas with LLMs, you can have different input and output types each time.</p><p>This is true, but I think when people try to use LLMs in their applications, they need to consider them as a crucial component in software and make sure they receive the appropriate input and output.</p><p>This is less critical for simple generation methods (summarization, writing a document, etc.) where errors don’t have a huge effect on the software. However, for features like sending a transaction through LLMs (and function calling), we need to make sure all inputs and outputs are 100% accurate.</p><h3>1-What is Agentic automation?</h3><p>In the old way of automating, trying to handle edge cases can increase the complexity of the code. With agents, we don’t need to handle such cases; they will be handled automatically.</p><h3>2-What’s an agent?</h3><p>An agent is born when we get an LLM to engage with itself. For example, when we ask the LLM to perform a task, and the LLM starts enhancing and creating further steps to reach the final answer, this makes the LLM an agent.</p><p>For instance, if you ask ChatGPT a question and it responds with an answer, this is not an agent because the LLM is not performing additional work beyond the response generated.</p><p>But reasoning is not the only thing that defines an agent; you have tools and memory, and you can have multiple agents. You might be familiar with tools that are functions that can be executed by the agent if required, and memory used to store the thinking and decisions made by the agent.</p><h3>3-Why several agents?</h3><p>Each can be specialized in a specific domain, and it is known that a specialized model is more capable than a general-purpose model on a specific task.</p><h3>4-Components of an agent system</h3><p><strong>Specialized agents: </strong>Try to add rules and goals. To have a good agent, we need to add some rules, identify the goals, and provide a backstory. These elements are necessary to define the role of an agent and to obtain a good result.</p><h4><strong>Enable collaboration</strong></h4><p>The ability for agents to cooperate and balance ideas off each other makes a huge difference in producing better outcomes. The same way that when you’re talking with ChatGPT, the process of having a conversation and giving it feedback helps it produce better outcomes.</p><p>The ability for these agents, now role-playing, to talk to each other and simulate the same chat behavior creates a better result. They can take feedback from each other and delegate tasks to each other, producing a better outcome.</p><h4><strong>Tools</strong></h4><p>Functions like web scraping, search engines, or any other tool that can be used by the agent to execute tasks.</p><h4><strong>Short-term Memory</strong></h4><p>This memory lives only during the crew execution. Every time you run a crew, it starts from blank. This memory is useful because as different agents try to accomplish different tasks, they store different things they learn in this memory, which is shared across all the agents.</p><h4><strong>Long-term Memory</strong></h4><p>It stores data used after execution of current tasks and can be used in future tasks.</p><h4><strong>Entity Memory</strong></h4><p>Stores the subjects being discussed. If your agent is trying to learn something about a specific company, it might store the company as an entity and its understanding of this company in this database.</p><h3>5- How do we define a multi-agent system?</h3><p>When we want to create an agent system, we should think as a manager. Think about what kind of people you would need to complete your project. For example, when building a website, you need frontend engineers, backend engineers, UI/UX designers, and more. You need to define the role of each one of them to ensure that the project will be completed.</p><h3>6-Crew AI components</h3><h4><strong>Agents </strong>(already explained)</h4><h4><strong>Tasks</strong></h4><p>Think like a manager. We need to set a goal, choose the right people (agents) for this job, define the role of each person, and what each person should do (tasks). We specify what we expect them to do (tasks should be explained clearly and concisely) and what the expected output is.</p><h4><strong>Tools</strong></h4><p>Used by agents to complete tasks. There are different ways to give agents tools:<br><strong> -Agent Level</strong>: The agent can use the tool(s) on any task it performs.<br><strong> -Task Level:</strong> The agent will only use the tool(s) when performing that specific task.</p><p>Tools are fault-tolerant: When getting exceptions, we need to make sure that the tool can fail and try the process again rather than stopping all the procedures.</p><blockquote>Note: In CrewAI, Task Tools override the Agent Tools. If an agent has 10 tasks and a tool has 3 tasks, then we can only use 3 tasks.</blockquote><h3>CrewAI features when building an multi agent system</h3><p><strong>→ Delegation</strong>: The ability to choose if an agent should go back to another agent in case it finds itself unable to solve the task.<br><strong> → Caching:</strong> Choose when to do a request and only do it when necessary, which helps reduce API calls, save time, and cost.<br><strong> → Async execution</strong>: Tasks executed in parallel with any other task.<br><strong> →Human input:</strong> Before providing the final answer, we should ask the user if they would like to make any changes.<br><strong> → Agent collaboration:</strong> Instead of a sequential process, which could lead to going away from the initial context, a better approach is to have collaboration between agents, where a manager, for example, delegates tasks to other agents. We can define how agents communicate with each other, such as hierarchically.</p><p>If you like what you see, hit the follow button! You can also find me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a>, and we can follow each other there too. 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c19dafe27e20" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stanford’s CS25: Lecture 2summary]]></title>
            <link>https://aliissa99.medium.com/stanfords-cs25-lecture-2summary-8cc871391fa4?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/8cc871391fa4</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Tue, 02 Jul 2024 19:11:08 GMT</pubDate>
            <atom:updated>2024-07-02T19:11:08.747Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Lecture 2 of <a href="https://www.linkedin.com/company/stanford-university/">Stanford University</a>’s CS25 V4 Transformers course was delivered by <a href="https://www.linkedin.com/in/jason-wei-5a7323b0/">Jason Wei</a> and <a href="https://www.linkedin.com/in/hyung-won-chung-5801b9a4/">Hyung Won Chung</a>. Highly recommended to watch, and below are some notes that I took.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vN2vg0j-IpxNh-JS" /><figcaption>Generated by DALL-E 3</figcaption></figure><h4>𝐖𝐡𝐲 𝐃𝐨 𝐋𝐋𝐌𝐬 𝐖𝐨𝐫𝐤?</h4><p>LLMs have revolutionised natural language processing and are now widely used for various tasks. Let’s delve into how they work and why they’re so effective.</p><h4>Next work prediction</h4><p>is Massive Multi-Task Learning: LLMs predict the next word in a sequence, a task known as “next-word prediction.” After training on this prediction task, LLMs gain the ability to perform multiple other tasks, such as translation, grammar correction (choosing the correct verb), math, and world knowledge.</p><p>The challenge lies in the fact that when predicting a sentence, the model learns a multitude of tasks simultaneously. For instance, it learns about world knowledge, predicts commas, and handles grammar — all from a single sentence. Now imagine scaling this to a large corpus of data.</p><h4>Increasing Computation to Improve Loss</h4><p>LLMs benefit from increased computation power. Possible reasons could be:</p><p><strong>-Memorization:</strong> Unlike SLMs, LLMs can memorize vast amounts of data. They don’t need to be selective about what they learn.</p><p><strong>-Heuristics: </strong>SLMs tend to learn simple tasks first. If they struggle with basic tasks, they won’t handle complex ones well. LLMs, on the other hand, handle complex tasks effectively.</p><h4>Individual Task Improvement and Overall Loss</h4><p>When the overall loss decreases during training, not all individual tasks need to improve equally. Some tasks may suddenly improve, while others show smoother progress. In the lecture, we observed this behavior using datasets like Big-Bench, which contains tasks from different categories.</p><h4>U-Shaped Scaling</h4><p>During a lecture, Jason demonstrated an interesting phenomenon. A tiny LLM predicted correctly, a small one struggled, and a large one excelled at a specific task (e.g., repeating words). The small model’s weakness was due to its unfamiliarity with the task. Inspecting data helps understand model behavior, and if you plot the curve, you’ll notice a U shape.</p><h4>𝐄𝐧𝐜𝐨𝐝𝐞𝐫-𝐝𝐞𝐜𝐨𝐝𝐞𝐫, 𝐝𝐞𝐜𝐨𝐝𝐞𝐫 𝐚𝐧𝐝 𝐞𝐧𝐜𝐨𝐝𝐞𝐫 𝐚𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞𝐬</h4><p><strong>Encoder:</strong> Bidirectional self-attention allows any token to attend to any other token in the sentence.</p><p><strong>Decoder: </strong>Causal self-attention attends only to tokens before a given token.</p><p><strong>Cross Attention:</strong> Connects encoder and decoder by attending to the final encoder layer.</p><p><strong>Encoder-decoder vs. decoder-only:</strong><br>-&gt; Encoder-decoder combines bidirectional and causal attention.<br>-&gt; Parameters are shared in the decoder.<br>-&gt; Encoder-decoder attends the last layer to the decoder, while the decoder with input and output attends at each layer.</p><p>If you like what you see, hit the follow button! You can also find me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a>, and we can follow each other there too. 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8cc871391fa4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stanford’s CS25: Lecture 1 summary]]></title>
            <link>https://aliissa99.medium.com/stanfords-cs25-lecture-1-summary-7f1b74f34286?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f1b74f34286</guid>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Tue, 11 Jun 2024 18:34:13 GMT</pubDate>
            <atom:updated>2024-06-11T18:34:13.224Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>I recently watched the first lecture of <a href="https://www.linkedin.com/company/stanford-university/">Stanford University</a> CS25 V4 Transformers course, and presented by <a href="https://www.linkedin.com/in/div99/">Div Garg</a>, <a href="https://www.linkedin.com/in/steven-feng/">Steven Feng</a>, <a href="https://www.linkedin.com/in/ebunnapradist/">Emily Bunnapradist</a> and <a href="https://www.linkedin.com/in/seonghee-lee/">Seonghee L.</a> It was packed with insightful information, and I took some interesting notes.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PcbyQCng54Jem_5d" /><figcaption>Generated by DALL-E 3</figcaption></figure><p>One of the challenges in NLP is precision. There are issues with context length, and some models lack reasoning and have short context lengths.An interesting fact I learned is that the first chatbot, Eliza Chatbot, was created in 1966.</p><h4>Semantic parsing</h4><p>It converts the words in a text into a machine-understandable format. I didn’t delve into the details, but I plan to explore it later as it seems to be a significant topic.</p><blockquote>The lecture covered various topics like transformers, RNNs, embeddings, and more, including the architecture of the transformer.</blockquote><h4>Cross-attention</h4><p>In a translation example from English to French. We have attention coming from the encoder (we have English input as text and we encode it), and this encoded input will be computed with the output (the French sentence) to check the relation between the initial input and the translated version of it. More technically, the key and value from the encoder will be provided to the decoder.This is called cross-attention. The second source is the self-attention between the output values.</p><h4>Emergent abilities</h4><p>Abilities that are present in larger models but not in smaller models.</p><h4>AGI</h4><p>Reducing computation complexity, thinking like humans, multi-modality, emotional intelligence, ethical reasoning, and more are necessary to achieve AGI.</p><h4>Transformers applications</h4><p>Text, speech, vision (analyzing images like vision transformers), generating images (DalL-E), videos (Sora), robotics (Voyager), and playing games (AlphaGo).</p><h4>Some of the challenges that LLM face</h4><p><strong>1)Continual learning:</strong> Infinite and permanent self-improvement. How can LLMs learn similarly to us? We don’t spend every two months reading all text online to update our knowledge, so how can LLMs do the same as humans?</p><p><strong>2)Model editing:</strong> Can we edit specific nodes in the model without the need to update all the weights? I found this idea interesting and have linked a related paper in the comments.</p><p><strong>3)Chain of thought</strong>: CoT works effectively for models of approximately 100B parameters or more. It seems small models forget some steps during reasoning or do not understand what is required to do.</p><p><strong>4)Socratic Questioning:</strong> Taken from their slides -&gt; “Divide-and-conquer fashion algorithm that simulates the self-questioning and recursive thinking process. Self-questioning module using a large-scale LM to propose subproblems related to the original problem as intermediate steps and recursively backtracks and answers the sub-problems to reach the original problem.”</p><blockquote>They also gave information about agents, multi-agents like agent-computer interaction: using API, or direct interaction using browser or desktop control. This is more interesting.</blockquote><p>If you like what you see, hit the follow button! You can also find me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a>, and we can follow each other there too. 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f1b74f34286" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[     …]]></title>
            <link>https://aliissa99.medium.com/-22caf09e5005?source=rss-d3daa151c134------2</link>
            <guid isPermaLink="false">https://medium.com/p/22caf09e5005</guid>
            <category><![CDATA[thoughts]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[vision]]></category>
            <dc:creator><![CDATA[Ali Issa]]></dc:creator>
            <pubDate>Fri, 24 May 2024 18:05:04 GMT</pubDate>
            <atom:updated>2024-05-24T18:05:04.667Z</atom:updated>
            <content:encoded><![CDATA[<h3>𝐕𝐢𝐬𝐮𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐓𝐡𝐨𝐮𝐠𝐡𝐭 𝐄𝐥𝐢𝐜𝐢𝐭𝐬 𝐒𝐩𝐚𝐭𝐢𝐚𝐥 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 𝐢𝐧 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥𝐬(Paper by microsoft)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BwWLldRhZlW2yx13" /><figcaption>Generated by DALLE-3</figcaption></figure><blockquote>Imagine a scenario where you’re listening to a story. As you follow along, your mind naturally starts to visualize the the unseen objects, actions, and their relationships. This mental process is often referred to as the “Mind’s Eye.” Now, consider how language models can perform similar spatial reasoning tasks.</blockquote><p>Spatial reasoning involves the ability to visualize and reason about relationships in a three-dimensional environment. It’s the skill that allows us to comprehend and navigate our surroundings. Recently, Microsoft introduced a paper that explores this concept in LLMs.</p><p>To evaluate spatial awareness and the LLM’s ability to understand spatial relationships, several tests have been conducted:</p><h4>𝐍𝐚𝐭𝐮𝐫𝐚𝐥 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐍𝐚𝐯𝐢𝐠𝐚𝐭𝐢𝐨𝐧</h4><p>In this approach, the model is tested on its ability to determine its current location based on a set of steps. Imagine a 4x4 square grid where objects are placed. For instance, if you start in the bottom-left corner and move right, you encounter a car; moving up, you find a chair, and so on.</p><p>After informing the LLM about the objects on the grid, we ask it to perform specific actions (e.g., “go up, then left, then right”). The model must then identify the current location of the specified object. At each reasoning step, the LLM visualizes the grid and performs the requested action.</p><h4>2)𝐕𝐢𝐬𝐮𝐚𝐥 𝐍𝐚𝐯𝐢𝐠𝐚𝐭𝐢𝐨𝐧</h4><p>The visual navigation task presents a synthetic 2D grid world to the LLM. The challenge is to navigate using visual cues. The LLM generates navigation instructions (left, right, up, down) to reach a destination from a starting point while avoiding obstacles.</p><p>This task involves two sub-tasks: route planning (finding a path) and next step prediction (using multi-hop spatial reasoning).</p><h4>3)𝐕𝐢𝐬𝐮𝐚𝐥 𝐓𝐢𝐥𝐢𝐧𝐠</h4><p>In this challenge, called polyomino tiling, a 6x6 grid is filled with colored shapes (e.g., green squares, yellow triangles). The LLM must fit a specific shape (e.g., a 3x1 red line) vertically within the grid. This tests the model’s ability to comprehend, organize, and reason with shapes in a confined area.</p><h4>𝐕𝐢𝐬𝐮𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐓𝐡𝐨𝐮𝐠𝐡𝐭</h4><p>The key idea behind VOT is to visualize the state after each reasoning step. Instead of simply generating an answer, the LLM performs a visualization at each reasoning stage. This approach helps improve the accuracy of the model’s results.</p><p>For a clearer understanding, refer to the image below, which illustrates the three tasks mentioned earlier and how VOT is applied.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*I2-N2ZTZhFWtTu0O" /><figcaption><a href="https://arxiv.org/abs/2404.03622v1">Source</a></figcaption></figure><p>If you like what you see, hit the follow button! You can also find me on <a href="https://www.linkedin.com/in/aliissa99/">LinkedIn</a>, and we can follow each other there too. 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22caf09e5005" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>