<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Winston Wang on Medium]]></title>
        <description><![CDATA[Stories by Winston Wang on Medium]]></description>
        <link>https://medium.com/@winston-wang?source=rss-9fdd9d6a1135------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*iLtkfSLlA0nh0D3Az_InAw.png</url>
            <title>Stories by Winston Wang on Medium</title>
            <link>https://medium.com/@winston-wang?source=rss-9fdd9d6a1135------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 09:09:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@winston-wang/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[DeepSeek, Geopolitics, and the Real AI Battleground: Innovation, Talent, and Strategic Advantage]]></title>
            <link>https://medium.com/thoughts-and-reflections-by-winston-wang/deepseek-geopolitics-and-the-real-ai-battleground-innovation-talent-and-strategic-advantage-b1f557686ffb?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/b1f557686ffb</guid>
            <category><![CDATA[innovation-policies]]></category>
            <category><![CDATA[tech-industry]]></category>
            <category><![CDATA[geopolitics]]></category>
            <category><![CDATA[deepseek]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Tue, 04 Feb 2025 04:14:16 GMT</pubDate>
            <atom:updated>2025-02-04T14:42:40.730Z</atom:updated>
            <content:encoded><![CDATA[<p>Last Monday (January 27), the Nasdaq Composite Index dropped by approximately 3%, primarily due to the release of the new <strong>R1 model</strong> by the Chinese startup <strong>DeepSeek</strong>. This model is considered a challenge to existing AI technologies in terms of performance and cost, leading investors to worry about the market position of leading U.S. AI companies. As a result, shares of AI chip manufacturers such as <strong>NVIDIA</strong> plummeted. <strong>NVIDIA’s stock price fell by 17%, wiping out approximately $592.7 billion in market value, marking the largest single-day market cap loss in U.S. stock market history</strong>.</p><p>As of now, <strong>market sentiment has somewhat recovered</strong>. However, the impact of DeepSeek has sparked widespread discussions across various domains, primarily in the following aspects:</p><ul><li><strong>Technological Innovation</strong></li></ul><p>DeepSeek has pioneered a <strong>development path distinct from closed AI companies like OpenAI</strong> by embracing an open-source approach. Its <strong>37-billion parameter model has demonstrated performance comparable to U.S. trillion-parameter models in multiple benchmarks</strong>, proving the feasibility of <strong>achieving more breakthroughs with fewer resources</strong>. This combination of <strong>efficient algorithms and an open-source ecosystem</strong> showcases a new direction for AI development.</p><ul><li><strong>Cost Efficiency</strong></li></ul><p>The <strong>training cost</strong> of the DeepSeek-R1 model is <strong>only $6 million</strong>, <strong>5% to 10%</strong> of the cost of models with similar performance. For inference, <strong>DeepSeek-R1 costs $2.2 per million tokens</strong>, <strong>about one-thirtieth of OpenAI’s O1 model</strong>. This <strong>low-cost, high-performance model</strong> significantly improves the affordability of large language models, paving the way for broader AI adoption.</p><ul><li><strong>Market Impact</strong></li></ul><p>The emergence of DeepSeek has had a <strong>huge impact on U.S. tech stocks</strong>. On <strong>January 27, 2025, NVIDIA lost $592.7 billion in market value in a single day</strong>, setting a record for a U.S. publicly traded company&#39;s largest single-day market cap loss. This market turbulence has prompted <strong>investors to reconsider their AI investment strategies</strong>.</p><ul><li><strong>Industry Landscape</strong></li></ul><p>DeepSeek’s success <strong>challenges the U.S.’s dominance in AI and reshapes the global AI industry’s competitive landscape</strong>. Several major U.S. companies, including <strong>Microsoft and AWS, </strong>have announced plans to<strong> integrate DeepSeek-R1 </strong>into their cloud services. Meanwhile, <strong>Perplexity has already deployed R1 for inference, allowing users to interact with it</strong>.</p><ul><li><strong>Policy Impact</strong></li></ul><p>The rise of DeepSeek has <strong>disrupted U.S. policies on semiconductor and AI restrictions against China</strong>, triggering a <strong>chain reaction in regulatory policies</strong>. This has also led the <strong>U.S. government to reassess its AI strategy</strong> — a reflection of this was the meeting between <strong>President Donald Trump and NVIDIA CEO Jensen Huang on January 31</strong>, where <strong>DeepSeek was a key discussion topic</strong>.</p><ul><li><strong>Innovation Model</strong></li></ul><p>DeepSeek has proven the power of <strong>open-source collaboration and algorithmic innovation</strong>. When <strong>technical details are made public and global developers can collectively participate in optimization</strong>, the <strong>pace of model improvement</strong> far exceeds that of traditional closed environments. This model may drive <strong>more companies to reconsider their R&amp;D strategies</strong>.</p><p>Personally, I believe that while DeepSeek’s consecutive releases of two models have indeed shocked the AI community, predicting the decline of the U.S. AI industry, as a result, is utterly absurd. DeepSeek’s Chinese identity stands out prominently in the current geopolitical competition between China and the U.S., especially after China’s sixth-generation fighter jet made its first public appearance late last year. Understandably, Americans feel they are once again facing a <strong>Sputnik crisis</strong>.</p><p>The process of a technology maturing is inherently a process of cost reduction. Even without DeepSeek, other teams would have eventually built better architectures to lower the training and usage costs of current large language models (LLMs). Whether it is the <strong>V3</strong> or <strong>R1</strong>, DeepSeek’s models have not significantly improved performance. While cost reduction will inevitably impact OpenAI’s business model, it only determines the lower bound of technological advancement. The upper bound of technology is still dictated by <strong>advanced hardware, a relaxed environment for innovation, abundant research talent, and substantial research investment</strong>.</p><p>In the short to medium term, the <strong>restrictions on chip technology</strong> and <strong>limitations imposed by Chinese government censorship</strong> will define DeepSeek’s trajectory. The U.S. will also further tighten the export of high-end GPUs to China, which is, in fact, the greatest nightmare for Chinese AI researchers. After all, current computing power is far from reaching a point where <strong>scaling laws</strong> become obsolete. The limitations of the <strong>transformer</strong> architecture are becoming increasingly apparent, and publicly available human text data will soon be exhausted. In the future, a new model architecture will inevitably replace transformers, and the research and training of this new architecture will require many GPUs.</p><p>When it comes to the AI industry, the U.S. enjoys <strong>geopolitical and technological advantages</strong>. Apart from the primary <strong>GPU production base in Taiwan</strong>, the biggest threat to the U.S. is not competition that creates a <strong>catfish effect</strong> but rather whether <strong>isolationist policies</strong> will continue to attract and retain top talent AND whether <strong>cuts and restrictions on research funding</strong> will still encourage innovation. This may well determine a nation&#39;s fate.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1f557686ffb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thoughts-and-reflections-by-winston-wang/deepseek-geopolitics-and-the-real-ai-battleground-innovation-talent-and-strategic-advantage-b1f557686ffb">DeepSeek, Geopolitics, and the Real AI Battleground: Innovation, Talent, and Strategic Advantage</a> was originally published in <a href="https://medium.com/thoughts-and-reflections-by-winston-wang">Thoughts and Reflections by Winston Wang</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[C’mon, Stop Wowing! Let’s See Why the AI World Is Going Wild Over DeepSeek’s Latest R1!]]></title>
            <link>https://medium.com/research-highlights-by-winston-wang/cmon-stop-wowing-let-s-see-why-the-ai-world-is-going-wild-over-deepseek-s-latest-r1-7afaa63db6a4?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/7afaa63db6a4</guid>
            <category><![CDATA[deepseek]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deepseek-r1]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Sun, 26 Jan 2025 06:23:20 GMT</pubDate>
            <atom:updated>2025-01-26T06:25:57.884Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/195/0*K44NKyX3jE7rvUNF" /></figure><p>The shock brought to us by the Chinese company DeepSeek continues. Less than a month after the release of their V3 model, they introduced the R1 model, which focuses more on general reasoning capabilities and multi-task adaptability, creating a significant stir in the AI community. If the previous V3 was positioned as a competitor to OpenAI’s 4o, then this time, R1 is clearly targeting o1. Most importantly, the R1 series is completely open-sourced, including model weights and training details, with multiple smaller distilled versions (e.g., 32B and 70B) provided, significantly lowering the entry barrier for developers.</p><p>Let’s first look at the results — let the numbers speak:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F6NY_gxrJ4unT2zjIeoV0w.png" /></figure><p><strong>What technologies have enabled DeepSeek’s latest R1 model to achieve such a significant breakthrough?</strong></p><p>Let’s start by understanding or reviewing two concepts:</p><p><strong>Supervised Fine-Tuning (SFT)</strong> — Supervised fine-tuning is a technique that builds upon a pre-trained model, using a labeled dataset for specific tasks to further train the model. Its goal is to improve the model’s performance in specific tasks or domains.</p><p><strong>Reinforcement Learning (RL)</strong> — A machine learning method that learns to make decisions by interacting with the environment. In this process, the agent learns through trial and error to take actions in a specific environment to maximize cumulative rewards.</p><h3>What Can RL Bring to a Model?</h3><p>Compared to the DeepSeek V3 model, which combines supervised fine-tuning (SFT) and reinforcement learning (RL), the R1 series attempted to completely skip the traditional SFT step during training and directly experimented with large-scale reinforcement learning (RL) to test whether the model could develop strong reasoning abilities:</p><p>The DeepSeek team first trained the DeepSeek-R1-Zero model directly on the base model using reinforcement learning without employing supervised fine-tuning (SFT) as a preliminary step. The model utilized the <strong>Group Relative Policy Optimization (GRPO)</strong> algorithm for reinforcement learning. This algorithm does not require a reward model as large as the policy model; instead, it estimates the baseline from group scores, thereby saving training costs.</p><p>To train DeepSeek-R1-Zero, researchers designed a simple template to guide the base model to follow specified instructions. This template required DeepSeek-R1-Zero <strong>first to produce a reasoning process and then provide a final answer.</strong></p><p>Studies have shown that neural reward models may have reward loopholes in large-scale reinforcement learning, and retraining reward models requires additional training resources, complicating the entire training process. Therefore, in the development of DeepSeek-R1-Zero, no result-based or process-based neural reward model was used.</p><p>Reports indicate that during reinforcement learning, <strong>DeepSeek-R1-Zero&#39;s thinking time progressively increased</strong>, suggesting that it learned to <strong>solve complex reasoning tasks by taking more thinking time.</strong> As testing time increased, R1-Zero would reexamine and reassess its previous steps and explore alternative problem-solving approaches. The resulting DeepSeek-R1-Zero demonstrated remarkable reasoning capabilities, as shown in the chart below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BWjKWufLhztXMD56xw44EA.png" /><figcaption>Reasoning-related benckmarks comparison between o1 and R1-Zero</figcaption></figure><p>However, R1-Zero also had significant issues, primarily manifesting as outputs containing mixed languages or the absence of markup-like markdown format, making it difficult for users to read and understand its reasoning process and answers. To address the issues of <strong>poor readability and language mixing</strong> in the DeepSeek-R1-Zero model, the research team made significant efforts, resulting in the development of the DeepSeek-R1 model.</p><h3>Alright, We Get It — RL Is Great! But How Do You Train R1?</h3><p>The training process of the R1 model adopted a multi-stage workflow as follows:</p><p><strong>Base Model → RL → SFT → RL → SFT → RL</strong></p><ul><li><strong>Base Model</strong>: The DeepSeek-V3-Base model was used as the base model.</li><li><strong>Cold Start Data</strong>: To provide the model with initial reasoning capabilities and avoid instability during the early RL training stages from the base model, the research team collected thousands of high-quality, long-chain-of-thought (CoT) data points and fine-tuned the DeepSeek-V3-Base model using this data. This served as the initial actor for RL training. Unlike DeepSeek-R1-Zero, the cold start data for DeepSeek-R1 used a readable format, with each response ending in a summary and filtering out responses unsuitable for reading. The output format was defined as |special_token||special_token|, where the reasoning process represents the CoT of the query, and the summary summarizes the reasoning results.</li><li><strong>Reasoning-Oriented RL</strong>: This stage focuses on enhancing the model’s performance in reasoning-intensive tasks (e.g., coding, mathematics, and logical reasoning). To address the language mixing issue in R1-Zero, a language consistency reward was introduced to encourage the model to use a consistent target language for reasoning.</li><li><strong>Rejection Sampling and SFT</strong>: Once reasoning-oriented RL converged, the resulting checkpoints were used to collect SFT data, including both reasoning and non-reasoning data. This stage aimed to improve the model’s general capabilities beyond reasoning tasks. Reasoning data was obtained through rejection sampling and reward models, while non-reasoning data came from the SFT dataset of DeepSeek-V3. Approximately 800,000 samples were used to fine-tune the model.</li><li><strong>RL for All Scenarios</strong>: A second stage of reinforcement learning was conducted to enhance the model’s helpfulness and harmlessness while further improving reasoning capabilities. Rule-based rewards were used for reasoning data, while reward models were employed to capture human preferences for general data.</li></ul><p>This training process produced the R1 model, which not only resolved some issues in R1-Zero but also delivered excellent performance。</p><h3>Excellent Performance</h3><p><strong>Significant Improvement in Reasoning Ability</strong></p><p>R1 performed exceptionally well in various reasoning tasks, <strong>matching the performance of OpenAI’s <em>o1–1217</em> model.</strong></p><ul><li>In the <strong>AIME 2024</strong> benchmark, DeepSeek-R1 achieved a <strong>Pass@1 score of 79.8%</strong>, slightly surpassing OpenAI-<strong><em>o1–1217</em></strong>.</li><li>In the <strong>MATH-500</strong> benchmark, DeepSeek-R1 scored an impressive <strong>97.3%,</strong> on par with OpenAI-<strong><em>o1–1217</em></strong>.</li></ul><p>Overall, the R1 model achieved significant accuracy improvements on STEM-related problems.</p><p><strong>Outstanding Coding Capabilities</strong></p><p>R1 demonstrated expert-level performance in coding competition tasks, achieving an Elo rating of <strong>2,029</strong> on <strong>Codeforces</strong>, surpassing <strong>96.3% of human competitors.</strong> On <strong>LiveCodeBench,</strong> R1 also scored <strong>65.9%.</strong></p><p><strong>Strong Knowledge and Understanding</strong></p><ul><li>In the <strong>MMLU</strong> benchmark, DeepSeek-R1 scored <strong>90.8%.</strong></li><li>In the <strong>MMLU-Pro</strong> and <strong>GPQA Diamond</strong> benchmarks, it scored <strong>84.0%</strong> and <strong>71.5%,</strong> respectively.</li><li>It performed excellently in the <strong>FRAMES</strong> long-text understanding task, demonstrating strong capabilities in long-document analysis.</li></ul><p><strong>Improved General Abilities</strong></p><p>In the <strong>AlpacaEval 2.0</strong> and <strong>Arena-Hard</strong> benchmarks, DeepSeek-R1 achieved win rates of <strong>87.6%</strong> and <strong>92.3%,</strong> respectively, showing robust intelligence in handling non-exam-oriented queries.</p><p><strong>Remarkable Results from Distilled Small Models</strong></p><p>The distilled small model DeepSeek-R1-Distill-Qwen-7B, with only <strong>7B parameters</strong>, achieved a score of <strong>55.5%</strong> on <strong>AIME 2024,</strong> outperforming the <strong>32B-parameter</strong> QwQ-32B-Preview. The distilled 32B-parameter model scored <strong>72.6%</strong> on <strong>AIME 2024</strong>, <strong>94.3%</strong> on <strong>MATH-500</strong>, and <strong>57.2%</strong> on <strong>LiveCodeBench,</strong> comparable to the 100B-parameter <strong><em>o1-mini</em>.</strong></p><h3>As for Shortcomings</h3><ul><li>The R1 model is <strong>highly sensitive to prompts,</strong> with small sample prompts consistently reducing its performance.</li><li>R1 did not show significant improvements over DeepSeek-V3 in software engineering benchmarks.</li><li>In tasks involving complex interactions and formatted outputs, such as function calls, multi-turn dialogues, complex role-playing, and JSON outputs, R1 underperformed compared to V3.</li><li>R1 is primarily optimized for Chinese and English, so it may exhibit language mixing issues when processing queries in other languages.</li><li>After safety reinforcement learning, R1 tends to refuse to answer certain queries, resulting in weaker performance than DeepSeek-V3 on the SimpleQA Chinese benchmark.</li></ul><h3><strong>Conclusion</strong></h3><p>Amidst widespread acclaim, it is important to recognize that the release of DeepSeek-R1 is not a replacement for V3 but an exploration and supplement of different technical paths. This does not imply that RL will replace SFT. The training of the R1-Zero model demonstrated the importance of RL in improving reasoning capabilities. However, in practical training, to avoid issues caused by exclusively using RL, R1 adopted a hybrid multi-stage training process. More importantly, the DeepSeek team, through training the R1 series models, provided deeper insights into how different training methods for large language models can endow models with specific capabilities. Compared to OpenAI’s o1 models, which consume massive amounts of tokens and money, R1’s open-sourcing provides the community with more ideas and conditions for experimentation.</p><h3>Check the Source Code</h3><p><a href="https://github.com/deepseek-ai/DeepSeek-R1">GitHub - deepseek-ai/DeepSeek-R1</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7afaa63db6a4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/research-highlights-by-winston-wang/cmon-stop-wowing-let-s-see-why-the-ai-world-is-going-wild-over-deepseek-s-latest-r1-7afaa63db6a4">C’mon, Stop Wowing! Let’s See Why the AI World Is Going Wild Over DeepSeek’s Latest R1!</a> was originally published in <a href="https://medium.com/research-highlights-by-winston-wang">Research Highlights by Winston Wang</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Forget Transformers? Meet Titans, the Next Big Leap in AI Memory and Efficiency]]></title>
            <link>https://medium.com/paper-skimming-by-winston-wang/forget-transformers-meet-titans-the-next-big-leap-in-ai-memory-and-efficiency-963587318e4d?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/963587318e4d</guid>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[future-of-ai]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[ai-innovation]]></category>
            <category><![CDATA[ai-architecture]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Thu, 23 Jan 2025 20:32:41 GMT</pubDate>
            <atom:updated>2025-01-23T20:32:41.288Z</atom:updated>
            <content:encoded><![CDATA[<p>The transformer architecture faces numerous challenges, such as Mamba, linear recurrent models, and the recently revived Test-Time Training (TTT) layers.</p><p>Recently, Google introduced a novel neural network architecture called <strong>Titans</strong>, incorporating short-term and long-term memory modules that can memorize data even during testing. This architecture leverages attention mechanisms as short-term memory, neural network modules as long-term memory, and a persistent memory module — learnable but input-independent — to address the scalability issues and long-sequence modeling challenges faced by Transformers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*62NHkz76Apv8SJO20AYGjg.png" /></figure><h4><strong>Memory in AI: Limitations of Existing Models</strong></h4><p>When discussing memory, models like LSTMs and Transformers often come to mind. These models simulate human memory from different perspectives, but they have notable limitations: Compressing data into a fixed-size hidden state results in a limited capacity. Capturing long-range dependencies leads to quadratic computational complexity as sequence length increases. Moreover, memorizing training data may not help in real-world applications where test data could fall outside the training distribution.</p><p>The core of Transformer models is the <strong>self-attention mechanism</strong>, which allows every <strong>token</strong> in a sequence to dynamically attend to all other tokens, capturing dependencies between them. However, this process requires pairwise interactions for every token pair, resulting in <strong>O(N²)</strong> computational complexity. For long sequences, this translates to significant time and memory consumption.</p><p>Linear Transformers, designed to handle longer sequences, reduce the complexity to <strong>O(N)</strong> by sacrificing some expressiveness and the ability to model long-range dependencies effectively. However, this compromise makes them less balanced and comprehensive solutions.</p><h4><strong>Titans: Inspired by Neuropsychology</strong></h4><p>From a neuropsychological perspective, memory is a neural update caused by input, and learning is the process of acquiring effective and useful memories. Human memory is not a single entity but consists of interconnected subsystems — short-term, working, and long-term memory — that serve different functions. This inspired Google researchers to design an efficient architecture that integrates these distinct memory modules, resulting in Titans.</p><h4><strong>Three Key Modules of Titans</strong></h4><p><strong>1. Core Module (Short-Term Memory)</strong></p><p>The primary data processing module is responsible for handling the current input sequence. Using <strong>attention with a limited window size</strong>, it functions as the model’s <strong>short-term memory</strong>, modeling dependencies between tokens within the current context window.</p><p><strong>2. Long-Term Memory Module</strong></p><p>A <strong>neural long-term memory module</strong> that stores and remembers historical information over longer periods. This module <strong>learns to store data in its parameters</strong> and can be memorized during testing. Inspired by the human long-term memory system, it acts as a <strong>meta-model</strong> capable of learning how to remember at test time.</p><p><strong>3. Persistent Memory Module</strong></p><p>This consists of <strong>learnable, input-independent parameters</strong> that encode task-specific knowledge. Acting as <strong>task-related memory</strong>, these parameters are fixed after training and typically prepended to the input sequence.</p><h4><strong>Memory Integration: Three Architectures</strong></h4><p><strong>Memory as Context (MAC)</strong></p><p>This approach treats long-term memory as the context for current information.</p><p>The input sequence is divided into fixed-size segments. The current segment serves as a query to retrieve historical information from <strong>long-term memory</strong>. Retrieved historical data and <strong>persistent memory parameters</strong> are combined with the current input segment and processed through the <strong>attention module</strong>.</p><p>This allows the model to <strong>make decisions using both historical and current information</strong> while ensuring that long-term memory only stores useful data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sE7BKvsoU2lLW9t4oDCfjQ.png" /><figcaption>MAC</figcaption></figure><p><strong>Memory as a Gate (MAG)</strong></p><p>In this architecture:</p><p>Input data directly updates the long-term memory. <strong>Sliding window attention (SWA)</strong> is applied to the input sequence, functioning as <strong>precise short-term memory</strong>. Outputs from sliding window attention and long-term memory are combined using a <strong>gating mechanism</strong>.</p><p>This approach offers flexible control over the influence of long-term memory on the final output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dxx5AJjWWEZ8jZ2_KBfrig.png" /><figcaption>MAG</figcaption></figure><p><strong>Memory as a Layer (MAL)</strong></p><p>Here, the long-term memory module is treated as a dedicated layer in a deep neural network.</p><p>The input sequence is processed through the long-term memory module first. The output is then fed into the <strong>sliding window attention</strong> module.</p><p>While common in hybrid models, this design limits the model’s overall capacity due to its layer-wise constraints.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rc-bU7j4AtFEu6NPIeE_Tw.png" /><figcaption>MAL</figcaption></figure><h4><strong>The Role of Persistent Memory</strong></h4><p>Persistent memory plays a crucial role in all three architectures:</p><p>• It stores <strong>task-specific knowledge</strong> independent of input data.</p><p>• By being prepended to the input sequence, it mitigates the <strong>bias of attention mechanisms</strong> towards initial tokens, enabling more effective attention weight allocation.</p><h4><strong>Titans’ Performance and Future</strong></h4><p>Experiments show that Titans outperforms Transformers and linear recurrent models in language modeling and commonsense reasoning tasks. Its hybrid architectures (MAC, MAG, MAL) excel over others, such as Mamba with attention.</p><p><em>Key achievements include:</em></p><p>• <strong>Needle in a Haystack (NIAH) Tasks</strong>: Titans effectively retrieve information from long sequences and scales to larger context windows.</p><p>• <strong>BABILong Benchmarks</strong>: Titans demonstrates superior reasoning capabilities over long documents, even surpassing some large language models like GPT-4.</p><p>• <strong>Diverse Applications</strong>: Titans achieves remarkable performance in time series forecasting and DNA modeling.</p><p>While the official Titans implementation is not yet open-sourced, third-party versions, like the <a href="https://github.com/lucidrains/titans-pytorch">Titans-PyTorch project</a>, are available. Google has also confirmed plans to release Titans’ PyTorch and JAX implementations soon, which will include training and evaluation scripts for broader adoption and testing.</p><h4>Read the paper here:</h4><p><a href="https://arxiv.org/abs/2501.00663">Titans: Learning to Memorize at Test Time</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=963587318e4d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paper-skimming-by-winston-wang/forget-transformers-meet-titans-the-next-big-leap-in-ai-memory-and-efficiency-963587318e4d">Forget Transformers? Meet Titans, the Next Big Leap in AI Memory and Efficiency</a> was originally published in <a href="https://medium.com/paper-skimming-by-winston-wang">Paper Skimming by Winston Wang</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeepSeek in a Nutshell: Everything you need to know at a Glance]]></title>
            <link>https://medium.com/research-highlights-by-winston-wang/deepseek-in-a-nutshell-everything-you-need-to-know-at-a-glance-dec2c0314c94?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/dec2c0314c94</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-innovation]]></category>
            <category><![CDATA[cost-effective]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deepseek]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Mon, 06 Jan 2025 16:30:32 GMT</pubDate>
            <atom:updated>2025-01-06T16:30:32.581Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/195/0*GAh8zmLWwHLcaXEy" /></figure><p>Recently, the large language model (LLM) DeepSeek has gained widespread attention. Out of curiosity, I conducted a <strong>brief investigation</strong> to understand its <strong>standout</strong> features.</p><h4><strong>Technological Innovations and Architecture</strong></h4><p>1. <strong>Mixture of Experts (MoE):</strong></p><p>DeepSeek V3 features <strong>671 billion</strong> parameters but only <strong>activates</strong> <strong>37 billion</strong> <strong>per token</strong>, thanks to its efficient MoE architecture combined with <strong>Multi-Head Latent Attention </strong>(<strong>MLA</strong>), which improves inference speed and performance.</p><p>2. <strong>DualPipe Algorithm:</strong></p><p>This algorithm optimizes the training process by overlapping computation and communication, enabling larger models on limited hardware resources through efficient cross-node communication.</p><h4><strong>Performance Metrics</strong></h4><ul><li><strong>Mathematics and Coding Ability: </strong>DeepSeek V3 achieved a high score of 90.2 in benchmarks like MATH-500, outperforming GPT-4o and other open-source models. It also excelled in coding benchmarks such as LiveCodeBench and HumanEval.</li><li><strong>Multilingual Capabilities: </strong>It performed exceptionally well in Chinese and multilingual tasks, scoring 88.8 in the CMMLU test, surpassing other open-source models.</li><li><strong>Educational and Factual Tasks: </strong>DeepSeek V3 demonstrated excellent comprehension in educational benchmarks (MMLU) and factual QA tasks (SimpleQA), performing on par with or better than proprietary models like GPT-4o and Claude-Sonnet-3.5.</li></ul><h4><strong>Cost and Efficiency</strong></h4><ul><li><strong>Low Training Cost: </strong>Complete training required only <strong>2.788 million GPU hours</strong> using Nvidia H800 GPUs, costing $5.58 million, far less than the $500 million required for Meta’s Llama 3.1.</li><li><strong>High Cost-Effectiveness: </strong>Techniques like FP8 low precision and sparse modeling optimized compute usage, earning it the nickname “Temu of AI.”</li></ul><h4><strong>Limitations</strong></h4><p>Despite its impressive performance, DeepSeek V3 lags behind models like Google Gemini 2.0 and OpenAI’s o1 series in areas such as complex reasoning and multimodal fusion. Its focus remains on <strong>“downward optimization”</strong> rather than <strong>“upward breakthroughs.”</strong></p><h4><strong>Core Technology — Multi-Head Latent Attention (MLA)</strong></h4><p>MLA enhances DeepSeek V3’s performance by improving inference speed, memory efficiency, and task handling through:</p><ul><li><strong>Low-Rank Joint Compression:</strong></li></ul><p>Reduces key-value (KV) storage requirements by compressing them into latent vectors, lowering memory usage and computational complexity.</p><ul><li><strong>Dynamic Feature Aggregation:</strong></li></ul><p>Merges key-value matrices from successive layers into shared representations, reducing computation and optimizing storage needs.</p><ul><li><strong>Information Filtering and Reuse:</strong></li></ul><p>Filters irrelevant information, improving the model’s focus and accuracy when processing complex tasks.</p><ul><li><strong>Efficient Inference and Extended Context Handling:</strong></li></ul><p>MLA enables efficient handling of long-context tasks (up to 128K tokens) while maintaining performance and stability.</p><h4>Conclusion</h4><p>Overall, despite facing U.S. tech sanctions and using relatively outdated GPUs, DeepSeek managed to train its large language model with just 2.788M H800 GPU hours — around 1% of the cost of Meta’s Llama 3.1 — achieving comparable or even superior performance. This remarkable result highlights the team’s impressive engineering capabilities. Instead of fundamental innovations, DeepSeek excelled by integrating existing techniques like knowledge distillation and sparse models. Their “combinatorial innovation” approach reflects a deep understanding of resource efficiency, potentially opening new business models and offering fresh perspectives for the current LLM market.</p><h4>Try DeepSeek yourself:</h4><p><a href="https://www.deepseek.com/">https://www.deepseek.com/</a></p><p><em>For more details on DeepSeek’s emergence, development efficiency, technical capabilities, and popularity, see the original article from the </em><strong><em>South China Morning Post</em></strong><em>.</em></p><p><a href="https://www.yahoo.com/tech/meet-deepseek-chinese-start-changing-093000172.html">Meet DeepSeek: the Chinese start-up that is changing how AI models are trained</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dec2c0314c94" width="1" height="1" alt=""><hr><p><a href="https://medium.com/research-highlights-by-winston-wang/deepseek-in-a-nutshell-everything-you-need-to-know-at-a-glance-dec2c0314c94">DeepSeek in a Nutshell: Everything you need to know at a Glance</a> was originally published in <a href="https://medium.com/research-highlights-by-winston-wang">Research Highlights by Winston Wang</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Implicit Chain of Thought (ICoT): A Faster but Unproven Alternative to CoT Reasoning]]></title>
            <link>https://medium.com/paper-skimming-by-winston-wang/exploring-implicit-chain-of-thought-icot-a-faster-but-unproven-alternative-to-cot-reasoning-ad31d99ac2e9?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad31d99ac2e9</guid>
            <category><![CDATA[ai-research]]></category>
            <category><![CDATA[chain-of-thought]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[reasoning]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Thu, 02 Jan 2025 02:10:44 GMT</pubDate>
            <atom:updated>2025-01-02T02:10:44.405Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*co3FoG1S9s9H1CKSfiE4UQ.png" /></figure><p>Since the release of OpenAI’s o1 model, Chain of Thought (CoT) prompting has gained widespread attention for its ability to enhance reasoning capabilities in language models. However, its drawbacks — slower inference speed and high computational costs — have also come to light. For many users, waiting for a model to reason step-by-step and paying for additional tokens can feel unnecessary.</p><p>To address these challenges, researchers introduced a novel approach: Implicit Chain of Thought (ICoT). ICoT aims to boost reasoning efficiency while avoiding generating explicit intermediate steps. By distilling knowledge from a teacher model trained with explicit CoT into a student model, ICoT gradually removes the need for visible intermediate steps, teaching the model to internalize the reasoning process.</p><p>The <strong>benefits of ICoT</strong> are clear:</p><ul><li>Cost Efficiency: Models reason implicitly within hidden states across layers, skipping intermediate token generation and reducing user costs.</li><li>Speed: ICoT is faster than explicit CoT, especially for tasks involving lengthy reasoning chains.</li><li>Streamlined Outputs: Answers are generated directly without visible intermediate steps.</li></ul><p>However, despite these advantages, experimental results reveal significant limitations. While ICoT shows promise in reducing costs and speeding up inference, its performance on knowledge-driven and arithmetic tasks remains behind explicit CoT.</p><p>A team of researchers from Tsinghua University tested this through experiments using simple multi-step arithmetic problems (e.g., addition and subtraction). They forced the model to answer directly using implicit reasoning and probed its hidden states to detect whether intermediate steps were being computed.</p><p>The findings were striking:</p><ol><li>Implicit reasoning skip steps: LLMs did not perform step-by-step calculations internally. Instead, they relied on intuition and learned experience to provide answers.</li><li>Vulnerability to modifications: Slight modifications to problem formats significantly reduced ICoT’s performance, while explicit CoT remained robust.</li><li>Reliability concerns: ICoT is less reliable for complex or highly accurate tasks, as it depends on intuition rather than systematic reasoning.</li></ol><p><strong>Conclusion</strong></p><p>While ICoT offers theoretical benefits, its current limitations suggest it cannot fully replace explicit CoT, particularly in scenarios demanding high accuracy and reliability. Explicit CoT remains the gold standard for enhancing LLMs’ ability to solve complex tasks.</p><p><strong>Read the original paper</strong></p><p><a href="https://arxiv.org/abs/2411.15862">Do LLMs Really Think Step-by-step In Implicit Reasoning?</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad31d99ac2e9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paper-skimming-by-winston-wang/exploring-implicit-chain-of-thought-icot-a-faster-but-unproven-alternative-to-cot-reasoning-ad31d99ac2e9">Exploring Implicit Chain of Thought (ICoT): A Faster but Unproven Alternative to CoT Reasoning</a> was originally published in <a href="https://medium.com/paper-skimming-by-winston-wang">Paper Skimming by Winston Wang</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sequoia’s Vision for AI in 2025: Key Predictions That Will Shape the Future]]></title>
            <link>https://winston-wang.medium.com/sequoias-vision-for-ai-in-2025-key-predictions-that-will-shape-the-future-57b7110d6439?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/57b7110d6439</guid>
            <category><![CDATA[ai-trends-2025]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[sequoia-capital]]></category>
            <category><![CDATA[future-of-ai]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Thu, 02 Jan 2025 01:34:28 GMT</pubDate>
            <atom:updated>2025-01-02T01:34:28.680Z</atom:updated>
            <content:encoded><![CDATA[<p>As 2024, dubbed the “primordial soup year” for AI, draws to a close, the AIGC ecosystem has seen explosive growth across infrastructure, models, tools, and applications. Competitive dynamics are solidifying, and Sequoia Capital has offered insights into what lies ahead in 2025.</p><h4>1. The Rise of Five Major LLM Players</h4><p>2024 was pivotal for large language models (LLMs), with five dominant companies establishing unique strengths:<br>• Microsoft/OpenAI: Leveraging brand strength and public visibility.<br>• Amazon/Anthropic: Combining cloud infrastructure with top AI talent.<br>• Google: Integrating hardware, software, and models for a holistic edge.<br>• Meta: Gaining traction with an open-source strategy.<br>• xAI: Investing heavily in data center infrastructure to lead from the hardware front.</p><h4>2. AI Search Becoming Mainstream</h4><p>AI-powered search engines emerged as game-changers in 2024 and are poised for mass adoption in 2025:<br>• New Entrants: Perplexity stands out with rapid user growth.<br>• Industry Leaders: OpenAI’s ChatGPT Search is redefining how information is accessed.<br>By providing direct answers and concise summaries, these tools are transforming traditional search paradigms.</p><h4>3. Data Center Expansion Accelerates</h4><p>AI’s computational demands have sparked a rapid boom in data centers:<br>• Supply Chain Impacts: Increased investment in chips and hardware.<br>• Infrastructure Challenges: Elevated demands for energy, cooling, and construction resources.<br>This trend is reshaping the tech and infrastructure landscape, with 2025 expected to be pivotal for data center development.</p><h4>4. Real-World AI Applications on the Rise</h4><p>In 2025, AI will move from experimentation to impactful deployment:<br>• Vertical Solutions: Generative AI is driving innovation in creative, legal, and medical fields.<br>• Product Differentiation: Companies are tailoring applications to enhance user experiences and gain a competitive edge.</p><h4>Looking Ahead</h4><p>2025 will mark a turning point for AI, with large-scale adoption, differentiated strategies, and real-world applications shaping the industry’s trajectory. Key themes include:<br>• LLM providers carving niche dominance.<br>• AI search redefining traditional paradigms.<br>• Data centers driving ripple effects across industries.</p><p>As AI applications continue to take root in 2025, the industry’s future landscape may depend on whether companies can leverage differentiated competitive advantages to capture market share.</p><h4>Check the original analysis</h4><p><a href="https://www.sequoiacap.com/article/ai-in-2025/">AI in 2025: Building Blocks Firmly in Place</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=57b7110d6439" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI’s Mathematical Myth Busted? FrontierMath Leaves LLMs Nearly Empty-Handed]]></title>
            <link>https://medium.com/research-highlights-by-winston-wang/ais-mathematical-myth-busted-frontiermath-leaves-llms-nearly-empty-handed-2b1f90833d0f?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b1f90833d0f</guid>
            <category><![CDATA[reasoning]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[benchmark]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[technews]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Wed, 01 Jan 2025 20:26:41 GMT</pubDate>
            <atom:updated>2025-01-01T20:26:41.345Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tz7vDoarl-MRiIj6f9Hpzg.png" /></figure><p>Although many large language models (LLMs) have recently demonstrated high scores in certain mathematical tests, growing research suggests that their reasoning abilities might not be as robust as initially expected. For instance, in a previous article shared in my <a href="https://medium.com/paper-skimming-by-winston-wang"><em>Paper Skimming by Winston Wang</em></a> column, Apple researchers tested mainstream models’ performance by creating variations of problems from the GSM8K dataset. Even simple modifications, such as changing names or numeric variables, caused a significant drop in accuracy across almost all models. While the o1-preview model exhibited strong robustness in these scenarios, it wasn’t immune to failure when test problems included statements that shouldn’t affect calculations. For more details, click <a href="https://medium.com/paper-skimming-by-winston-wang/evaluating-the-mathematical-reasoning-capabilities-of-large-language-models-limitations-and-1e504f047777">here</a>.</p><p>If simple problem variations can significantly degrade the performance of large language models (LLMs), critical questions about the current challenges in mathematical reasoning assessments for these models are raised. Here are the <strong>Key Issues</strong>:</p><p><strong>1. Difficulty Saturation</strong></p><p>Existing benchmarks, such as <strong>MATH</strong> and <strong>GSM8K</strong>, primarily assess high school and early undergraduate-level mathematical skills. As AI models advance, their performance on these benchmarks has approached near-perfect scores, resulting in <strong>reduced discriminative power</strong>. These benchmarks can no longer effectively evaluate AI’s abilities in higher-level, more complex mathematical domains. Consequently, we lack rigorous methods to assess models’ competencies in areas requiring <strong>deep theoretical understanding, creative insights, and expert knowledge</strong>.</p><p><strong>2. Data Contamination</strong></p><p>Problems in current benchmarks may unintentionally overlap with the models’ training data, causing <strong>data contamination</strong>. This contamination can <strong>artificially inflate model performance</strong>, masking their true reasoning abilities. While competitions like the International Mathematical Olympiad (IMO) provide fresh test problems, their <strong>limited quantity</strong> and reliance on <strong>manual scoring</strong> present significant challenges for scalability and objectivity.</p><p><strong>3. Limitations in Problem Type and Scope</strong></p><p>Many existing benchmarks focus on <strong>competition-style problems</strong> or <strong>basic mathematical reasoning</strong> rather than the intricate problems found in modern mathematical research. These benchmarks often fail to evaluate <strong>abstract concepts, creative thinking, and specialized expertise</strong>, which is crucial in advanced mathematics. Additionally, their limited coverage cannot encompass the breadth of modern mathematical disciplines, leaving many important areas unexplored.</p><p><strong>4. Lack of Automated Verification Mechanisms</strong></p><p>Some benchmarks rely on <strong>manual evaluation</strong>, introducing <strong>subjectivity and inconsistency</strong> into the assessment process. <strong>Automated validation mechanisms</strong> are essential to ensure objectivity and reproducibility. However, many current benchmarks lack such systems, undermining the reliability of their evaluations.</p><p><strong>5. Limitations in Assessment Methods</strong></p><p>Existing benchmarks may not capture the <strong>full problem-solving process</strong> of AI models, such as intermediate reasoning steps and experimental processes. This limitation hinders our understanding of how AI models approach and solve problems, restricting a comprehensive evaluation of their capabilities.</p><p><strong>6. Inability to Assess Long-Term Research Capabilities</strong></p><p>Current benchmarks typically restrict problems to those solvable within a few hours. However, <strong>real-world mathematical research often requires weeks, months, or even years</strong> of sustained exploration. These benchmarks fail to test AI models’ abilities in <strong>long-term research and exploratory problem-solving</strong>, leaving a significant gap in evaluating their potential for tackling real-world mathematical challenges.</p><p>Addressing these issues is crucial to developing benchmarks that more accurately reflect AI’s capabilities and limitations in advanced mathematics. Only then can we truly understand the scope of AI’s reasoning abilities and identify areas for improvement.</p><p><strong>FrontierMath</strong>, a benchmark dataset created collaboratively by over 60 mathematicians, has emerged in response to these existing issues. This dataset contains <strong>hundreds of highly challenging, original mathematical problems</strong> designed to push the boundaries of AI’s reasoning capabilities. These problems span all major branches of modern mathematics and are significantly more difficult than those found in existing benchmarks. FrontierMath provides a more precise evaluation of AI’s abilities in <strong>advanced mathematical reasoning</strong>, offering a critical tool to assess and advance AI’s potential in this field.</p><p>To start with the C<strong>onclusion</strong>: the six leading large language models on the market (including <strong>o1-preview, o1-mini, GPT-4o, Claude 3.5 Sonnet, Grok 2 Beta, and Gemini 1.5 Pro 002</strong>) performed <strong>universally poorly</strong> on the <strong>FrontierMath</strong> dataset. Their <strong>success rates were below 2%</strong>, contrasting their <strong>near-saturated performance on other mathematical benchmarks</strong> such as <strong>GSM8K</strong>, <strong>MATH</strong>, <strong>AIME</strong>, or <strong>Omni-MATH</strong>.</p><p>What does this indicate? Are today’s large language models completely inadequate at mathematical reasoning, or is the <strong>FrontierMath</strong> dataset simply too difficult? To answer this question, we need to examine how the <strong>FrontierMath</strong> dataset was designed, its level of difficulty, and its unique characteristics:</p><h4>Details of <strong>FrontierMath</strong> dataset designation</h4><p><strong>Problem Origins and Difficulty</strong></p><p>• <strong>Expert-Driven Challenges</strong>: All <strong>FrontierMath problems</strong> were designed and reviewed by <strong>expert mathematicians</strong>, ensuring both <strong>originality</strong> and <strong>great difficulty</strong>. These problems span major branches of modern mathematics, including number theory, algebraic geometry, group theory, probability theory, and topology.</p><p>• <strong>Beyond Existing Benchmarks</strong>: The difficulty level far exceeds existing benchmarks such as <strong>MATH</strong> and <strong>GSM8K</strong>, demanding deeper theoretical understanding, creative insights, and domain expertise. Experts may need hours of effort to solve a typical problem, while higher-level problems could take days.</p><p>• <strong>Wide Scope</strong>: Problems range from computationally intensive tasks to abstract conceptual challenges, reflecting the real complexities of modern mathematical research.</p><p><strong>Design Principles</strong></p><ul><li><strong>Originality</strong>: Problems must be novel and built on existing mathematical concepts in <strong>non-trivial and non-obvious ways</strong>. This requires clever adaptations or innovative combinations that significantly transform the original concepts, preventing models from relying on pattern matching.</li><li><strong>Automated Verifiability</strong>: Each problem must have a clear and computable solution, typically expressed as a numeric answer or as a mathematical object that can be represented in <strong>SymPy</strong>, an open-source Python symbolic computation library. These objects include symbolic expressions, matrices, and sets.</li><li><strong>Guessing Prevention</strong>: Problems are carefully designed to eliminate the possibility of guessing answers. Solutions require thorough problem-solving rather than speculation, and answers are often large, complex numbers that are difficult to predict.</li><li><strong>Computational Feasibility</strong>: For computationally intensive problems, solutions must include scripts demonstrating how the answer is derived. These scripts must run within <strong>one minute on standard hardware</strong>.</li></ul><p><strong>Automated Validation Process</strong></p><ul><li>Models must submit <strong>code</strong> that computes and stores their answers as Python objects.</li><li>A <strong>validation script</strong> loads the model’s submitted results and marks them as correct or incorrect.</li><li>The validation checks for exact matches for problems with unique integer solutions. For problems with unique symbolic real-number solutions, <strong>SymPy</strong> is used to verify whether the difference between the submitted answer and the actual solution simplifies to zero. In other cases, custom validation scripts are implemented.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JnxMj3O9ETdldAtojScQ6Q.png" /><figcaption>Sample Problems. FrontierMath features hundreds of advanced mathematics problems that require hours for expert mathematicians to solve.</figcaption></figure><p><strong>Multi-Dimensional Difficulty Assessment</strong></p><p>Each problem is evaluated on three key dimensions:</p><ol><li><strong>Background Knowledge</strong>:</li></ol><p>Rated on a scale of 1 to 5, corresponding to:</p><p>1: High school level, 2: Early undergraduate, 3: Late undergraduate, 4: Graduate, 5: Research-level</p><p><strong>2. Creativity</strong>:</p><p>Measures the time (in hours) required to develop the key insight to solve the problem.</p><p>3. <strong>Execution</strong>:</p><p>Measures the time (in hours) required to compute the final solution.</p><p>All difficulty assessments undergo <strong>peer review by experts</strong> to ensure accuracy and reliability.</p><p><strong>Quality Assurance</strong></p><ul><li><strong>Blind Reviews</strong>: Every problem is reviewed blindly by at least one expert mathematician to verify the correctness of the problem statement and solution and ensure it is resistant to guessing.</li><li><strong>Peer Review</strong>: The peer review process identifies issues such as incorrect answers, problems prone to guessing, or solutions that can be brute-forced. The benchmark only includes these problems after the authors successfully revise them.</li></ul><p><strong>Data Collection and Contamination Prevention</strong></p><ul><li>Problems are created by collaborating with over <strong>60 mathematicians</strong> from various countries and fields, ensuring expertise and diversity.</li><li>All submission processes are conducted via secure encrypted channels to prevent data leakage, and stored materials are encrypted for protection.</li><li>The core team of mathematicians conducts expert reviews to verify originality, supplemented by plagiarism detection software.</li></ul><p><strong>For Teams Interested in Evaluating Their Models</strong></p><p>Teams wishing to benchmark their models on <strong>FrontierMath</strong> can contact the organizers via email at <a href="/service/mailto:math_evals@epochai.org">math_evals@epochai.org</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XXBQ6BL2wWV7AABkP18fzQ.png" /><figcaption>Comparison of unsolved problems across five mathematics benchmarks.</figcaption></figure><h4><strong>Performance Details of Current LLMs on the FrontierMath Dataset</strong></h4><p><strong>Overall Poor Performance</strong></p><ul><li>None of the tested models <strong>succeeded above 2%</strong> during FrontierMath’s comprehensive evaluation.</li><li>This highlights a <strong>significant gap</strong> between the mathematical reasoning abilities of current AI systems and those of expert mathematicians. Even the top-performing models struggled significantly to solve the problems.</li></ul><p><strong>Differences Among Models</strong></p><p>Despite the overall poor performance, there were slight variations between models. For example:</p><ul><li><strong>o1-preview</strong> demonstrated <strong>relatively better performance</strong> in repeated trials.</li><li>However, researchers cautioned against overinterpreting model rankings due to the extremely low success rates and the fact that single successes could disproportionately impact rankings.</li></ul><p><strong>Behavioral Differences Across Models</strong></p><p>Models differed significantly in how they approached the problems:</p><ul><li><strong>o1-preview</strong> provided an average of <strong>1.29 responses per problem</strong>, whereas <strong>Grok 2 Beta</strong> averaged <strong>3.81 responses</strong>.</li><li><strong>o1-preview</strong> and <strong>Gemini 1.5 Pro 002</strong> often submitted final answers <strong>before conducting any experiments</strong>, despite explicit instructions allowing them to perform experiments and refine their answers.</li></ul><p>In contrast, other models demonstrated <strong>longer interaction processes</strong>, suggesting varied strategies in problem-solving.</p><p><strong>Token Usage</strong></p><ul><li><strong>Gemini 1.5 Pro 002</strong> reached the <strong>10,000-token limit</strong> the least frequently, at just <strong>16.8% of attempts</strong>.</li><li>In comparison, models like <strong>Claude 3.5 Sonnet</strong>, <strong>GPT-4o</strong>, and <strong>Grok 2 Beta</strong> exceeded the token limit in <strong>over 45% of attempts</strong>.</li><li><strong>Gemini 1.5 Pro 002</strong> also used the fewest tokens per problem on average (~6,000 tokens), while other models ranged from <strong>12,000 to 17,000 tokens per problem</strong>.</li></ul><p><strong>Analysis of Solved Problems</strong></p><p>Researchers selected four problems successfully solved by at least one model and conducted <strong>five repeated trials</strong> for each model on each problem. Results revealed <strong>high variability across trials</strong>, with no consistent performance. Only in one instance did a model solve the same problem in all five trials (<strong>o1-preview</strong> on Problem 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1WSpE-28fIxNA8mkrJXgGQ.png" /><figcaption>Performance of leading language models on FrontierMath based on a single evaluation.</figcaption></figure><p><strong>Key Insights from Researchers</strong></p><p>The mathematical topics in FrontierMath often draw from <strong>specific areas of expertise</strong> with <strong>limited representation in training data</strong>, such as specialized mathematical papers and niche research. Many problems demand familiarity with the “tricks” and intuition characteristic of specific mathematical fields, which are deeply rooted in <strong>domain knowledge</strong> rather than generic reasoning or pattern recognition.</p><h3><strong>Conclusion</strong></h3><p>The emergence of the <strong>FrontierMath</strong> dataset provides a rigorous testing platform for evaluating the mathematical reasoning abilities of AI systems, offering a means to <strong>quantify progress</strong> in this critical area. It also establishes a <strong>robust and practical benchmark</strong> that can serve as a foundation for testing AI’s mathematical capabilities for years to come, with the potential to evolve into a <strong>standardized test suite</strong>.</p><p>That said, the current performance of large language models on FrontierMath falls far behind that of human mathematical experts. This underscores that AI has <strong>significant room for improvement in advanced mathematical reasoning</strong>. However, creating the FrontierMath dataset represents a milestone, as it allows us to define <strong>a new baseline</strong> for evaluating AI systems in this domain.</p><h4>To check the benchmark:</h4><p><a href="https://epoch.ai/frontiermath/the-benchmark">FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI</a></p><h4>To view the website:</h4><p><a href="https://epoch.ai/frontiermath">FrontierMath</a></p><h4>To read the paper:</h4><p><a href="https://arxiv.org/abs/2411.04872">FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b1f90833d0f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/research-highlights-by-winston-wang/ais-mathematical-myth-busted-frontiermath-leaves-llms-nearly-empty-handed-2b1f90833d0f">AI’s Mathematical Myth Busted? FrontierMath Leaves LLMs Nearly Empty-Handed</a> was originally published in <a href="https://medium.com/research-highlights-by-winston-wang">Research Highlights by Winston Wang</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating the Mathematical Reasoning Capabilities of Large Language Models: Limitations and…]]></title>
            <link>https://medium.com/paper-skimming-by-winston-wang/evaluating-the-mathematical-reasoning-capabilities-of-large-language-models-limitations-and-1e504f047777?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e504f047777</guid>
            <category><![CDATA[technews]]></category>
            <category><![CDATA[reasoning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[chain-of-thought]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Tue, 15 Oct 2024 05:32:11 GMT</pubDate>
            <atom:updated>2024-10-15T05:41:30.665Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Evaluating the Mathematical Reasoning Capabilities of Large Language Models: </strong>Limitations <strong>and </strong>Challenges</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dm28VvnFvLgkigmcqJjOug.png" /></figure><p>LLMs have made remarkable progress in various fields, including natural language processing, question answering, and creative tasks, even demonstrating the ability to solve mathematical problems. Recently, OpenAI&#39;s o1 model, which uses CoT (Chain of Thought), has shown significant reasoning capabilities. However, for a long time, the commonly used <strong>GSM8K</strong> dataset has had a fixed set of questions and a single evaluation metric, which doesn&#39;t fully capture the models&#39; abilities. Moreover, due to the popularity of GSM8K, there may be data contamination issues, leading to overly optimistic evaluation results. At the same time, it is difficult to assess the impact of varying problem difficulty on model performance due to a lack of controllability.</p><p><strong>Apple researchers</strong> aim to address this by developing a more general, adaptive evaluation framework to resolve the reliability issues of current GSM8K evaluation results and explore the factors influencing performance variation. This framework will investigate whether LLMs genuinely understand mathematical concepts and their limitations in dealing with irrelevant information and solving more complex math problems.</p><h3>Let&#39;s first take a quick look at the article&#39;s conclusions:</h3><p>To conclude the mathematical reasoning capabilities of large language models, the researchers evaluated over 20 open-source models of various sizes, ranging from 2B to 27B parameters. They evaluated common series such as the <strong>Gemma series, Mistral series, Llama3 series, GPT-4o series, Phi series</strong>, and models like OpenAI&#39;s recently released <strong>o1-mini</strong> and <strong>o1-preview</strong>. Through extensive experiments on these models using the GSM8K and GSM-Symbolic datasets, the authors arrived at the following conclusions:</p><ol><li><strong>The reasoning process of LLMs may not be actual logical reasoning but rather complex pattern matching</strong>.<br>Reasons include a) <strong>LLM performance shows significant variance between different instances of the same problem</strong>, even when those instances differ only in name or numerical values, and b) <strong>LLM performance declines as problem difficulty increases, with performance gaps widening</strong>.</li><li><strong>LLMs struggle with handling irrelevant information</strong>.<br>LLM performance significantly drops when presented with seemingly related but irrelevant statements in a problem. Even when multiple examples of the same problem or similar irrelevant information are provided, LLMs still need help with these challenges. <strong>LLMs need a true understanding of mathematical concepts and the ability to identify relevant information for problem-solving</strong>.</li></ol><h3>Methodology Design</h3><p>New technical approaches must be developed to conduct a deeper evaluation of LLMs&#39; mathematical reasoning capabilities. For this reason, researchers at Apple have created a new benchmark:</p><h4>GSM-Symbolic</h4><p>GSM-Symbolic is generated from symbolic templates, creating various variants of GSM8K problems. This enables more detailed and reliable evaluations, surpassing the single-point accuracy metric.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T4ujAUZKBOtOyMNCAt9FAw.jpeg" /><figcaption>Illustration of the GSM-Symbolic template creation process.</figcaption></figure><h4><strong>GSM-Symbolic Template Creation Process</strong></h4><p>The design process for GSM-Symbolic is as follows:</p><ul><li><strong>Select a specific example from the GSM8K test set as the template base.</strong></li><li><strong>Parse the example problem into a parsable template</strong>:<br>· Identify the variables in the problem. For example, in the figure above, &quot;Sophie,&quot; &quot;nephew,&quot; &quot;31,&quot; &quot;8,&quot; &quot;9,&quot; and &quot;62&quot; are all variables.<br>· Define domains for each variable. For instance, &quot;name&quot; can be sampled from a list of common names, and &quot;family&quot; can be &quot;nephew,&quot; &quot;cousin,&quot; or &quot;brother.&quot;<br>· Add necessary conditions to ensure the correctness of the problem and solution. For example, ensure that all involved numbers are divisible so the solution yields an integer.</li><li><strong>Automate checks to ensure annotation correctness</strong>:<br>· Ensure the template does not contain original variable values.<br>· Ensure original values satisfy all conditions.<br>· Ensure the generated solution matches the original problem&#39;s solution.</li><li>Manually <strong>review generated templates</strong>: Randomly select ten samples for manual review to ensure template quality.</li><li><strong>The final automated check after all models are evaluated is</strong> to ensure that at least two models can correctly answer each question; otherwise, conduct another manual review.</li></ul><p>This process allows GSM-Symbolic to generate numerous problem instances and adjust the difficulty as needed. This allows researchers to evaluate LLMs&#39; mathematical reasoning capabilities more comprehensively and understand their performance in various situations. The resulting GSM-Symbolic dataset contains 100 templates, with 50 samples generated per template, totaling 5,000 examples.</p><p><strong>In addition to GSM-Symbolic, the research team also created the</strong></p><h4>GSM-NoOp dataset.</h4><p>Its purpose is to challenge the reasoning capabilities of large language models (LLMs) and reveal their limitations in truly understanding mathematical concepts and identifying relevant information for problem-solving.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EUE_IQjXIKKzTOfzV1uCsA.jpeg" /><figcaption>An example from the GSM-NoOp dataset</figcaption></figure><p>The creation method of GSM-NoOp is based on the GSM-Symbolic template, adding statements that seem relevant but have no impact on the reasoning process or final answer. These statements are called “No-Op” (no operation) sentences, which don’t involve any real mathematical operations. For example, in the figure above, the problem mentions, “but five of them were a bit smaller than average,” which is irrelevant to calculating the total number of kiwis Oliver has. However, many models incorrectly convert this irrelevant information into a subtraction operation, resulting in an incorrect final answer.</p><h3>Evaluation Method</h3><p><strong>For the GSM-Symbolic and GSM-NoOp datasets,</strong> the <strong>8-shot CoT (Chain-of-Thought) prompting</strong> and greedy decoding methods are used for evaluation.</p><ul><li>8-shot CoT prompting: Provide 8 example problems and solutions (including full reasoning steps) to guide the model in learning how to solve similar problems.</li><li>Greedy decoding: At each step, the most probable word is selected as the output to generate the final answer.</li></ul><p>The main focus for <strong>GSM-Symbolic</strong> is on model accuracy and distribution across different problem variants.</p><p>For <strong>GSM-NoOp,</strong> the focus is on model accuracy on problems with No-Op sentences, and these results are compared with those from the GSM-Symbolic dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TE1F-iqGDA4jKfNvN_yoaQ.jpeg" /><figcaption>Except for GPT-4o, the average performance on GSM-Symbolic is lower than on GSM8K (indicated by the dashed line)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wF6OFSvwGX85P7OMed48vQ.jpeg" /><figcaption>The performance of all state-of-the-art models on GSM-Symbolic drops compared to GSM8K.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b7Q56sj3cXUxanOVYShi4g.jpeg" /><figcaption>The model&#39;s performance shows significant variation even when only the names are changed. The variation becomes even more noticeable when they change the numbers or both names and numbers.</figcaption></figure><p><strong>GSM-Symbolic Dataset:</strong></p><p><strong>Performance variation:</strong> All models exhibit significant performance variation across different problem instances, even if these instances only differ in names or numerical values.</p><p><strong>Performance decline:</strong> As problem difficulty increases (e.g., by increasing the number of clauses in the problem), LLM performance declines, and performance variation also increases.</p><p><strong>GSM-NoOp Dataset:</strong></p><p><strong>Significant performance decline:</strong> All models&#39; performance declined significantly in the GSM-NoOp dataset, indicating that LLMs struggle with handling irrelevant information and identifying the necessary information to solve problems.</p><p>On GSM-NoOp, model performance declines significantly, with newer models showing greater drops than older models. See (a) in the picture below.</p><p>Models’ performance on GSM-Symbolic is close to GSM8K. However, on GSM-NoOp, the significant performance decline cannot be recovered. See (b) in the picture below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AbZkGSvPvA4MMz2fefYTMQ.jpeg" /></figure><p>GSM-NoOp adds seemingly related but irrelevant clauses to the GSM-Symbolic template. All models, including the o1 model, show significant performance declines. <strong>This indicates that even powerful models like o1 do not truly understand the logical structure of mathematical problems.</strong></p><p><strong>Difficult to resolve with few-shot learning or fine-tuning:</strong> Even when provided with multiple examples of the same problem or similar irrelevant information, LLMs still struggle to overcome the challenges posed by the GSM-NoOp dataset.</p><h4>Analysis of o1-preview and o1-mini</h4><p>While o1-preview and o1-mini outperform other open-source models in overall performance, they still share similar limitations with other models.</p><p>On the GSM-Symbolic dataset, o1-preview shows strong robustness, maintaining high accuracy and low-performance variation across different difficulty levels. However, as problem difficulty increases, o1-mini’s accuracy declines, and performance variation increases.</p><p>On the GSM-NoOp dataset, o1-mini also shows a significant performance drop, indicating it struggles with handling irrelevant information and identifying key information for solving problems. o1-preview also shows a significant decline, indicating it struggles with fully understanding mathematical concepts and distinguishing relevant information.</p><p>The authors provide related examples in the appendix. When dealing with irrelevant information, the o1 model tends to mechanically apply specific operations without truly understanding their meaning. For instance, in one GSM-NoOp problem, even though the donation amount mentioned in the problem is unrelated to the price difference to be solved, the model incorrectly applied a subtraction operation on the donation amount.</p><h3><strong>Conclusion</strong></h3><p><strong>In summary, while LLMs have made impressive progress in many areas, their mathematical reasoning capabilities still have significant limitations.</strong> Future research should explore new methods to help LLMs learn true mathematical concepts and logical reasoning abilities rather than relying solely on pattern matching. <strong>More research and exploration are needed to develop AI systems capable of human-level mathematical reasoning.</strong></p><p><em>Click </em><a href="https://machinelearning.apple.com/research/gsm-symbolic"><strong><em>here</em></strong></a><em> to see the research page.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e504f047777" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paper-skimming-by-winston-wang/evaluating-the-mathematical-reasoning-capabilities-of-large-language-models-limitations-and-1e504f047777">Evaluating the Mathematical Reasoning Capabilities of Large Language Models: Limitations and…</a> was originally published in <a href="https://medium.com/paper-skimming-by-winston-wang">Paper Skimming by Winston Wang</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unveiling AlphaFold 3: The Next Leap in Predicting Biomolecular Structures Across the Chemical…]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/research-highlights-by-winston-wang/unveiling-alphafold-3-the-next-leap-in-predicting-biomolecular-structures-across-the-chemical-fa8a2b754293?source=rss-9fdd9d6a1135------2"><img src="https://cdn-images-1.medium.com/max/1280/0*Hdpn3aOEgAT3k7Fs" width="1280"></a></p><p class="medium-feed-snippet">On October 9, 2024, the Royal Swedish Academy of Sciences decided to award half of the 2024 Nobel Prize in Chemistry to Demis Hassabis and&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/research-highlights-by-winston-wang/unveiling-alphafold-3-the-next-leap-in-predicting-biomolecular-structures-across-the-chemical-fa8a2b754293?source=rss-9fdd9d6a1135------2">Continue reading on Research Highlights by Winston Wang »</a></p></div>]]></description>
            <link>https://medium.com/research-highlights-by-winston-wang/unveiling-alphafold-3-the-next-leap-in-predicting-biomolecular-structures-across-the-chemical-fa8a2b754293?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa8a2b754293</guid>
            <category><![CDATA[alphafold]]></category>
            <category><![CDATA[computational-biology]]></category>
            <category><![CDATA[structural-biology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[protein-structure]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Sat, 12 Oct 2024 00:40:49 GMT</pubDate>
            <atom:updated>2024-10-12T00:55:23.843Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[在Apple Silicon上使用MLX对模型进行微调的新手教程]]></title>
            <link>https://medium.com/tutorial-by-winston-wang/%E5%9C%A8apple-silicon%E4%B8%8A%E4%BD%BF%E7%94%A8mlx%E5%AF%B9%E6%A8%A1%E5%9E%8B%E8%BF%9B%E8%A1%8C%E5%BE%AE%E8%B0%83%E7%9A%84%E6%96%B0%E6%89%8B%E6%95%99%E7%A8%8B-f5f9959d8961?source=rss-9fdd9d6a1135------2</link>
            <guid isPermaLink="false">https://medium.com/p/f5f9959d8961</guid>
            <category><![CDATA[model-finetuning]]></category>
            <category><![CDATA[apple-m-series-processor]]></category>
            <category><![CDATA[mlx]]></category>
            <category><![CDATA[apple-silicon]]></category>
            <category><![CDATA[tutorial]]></category>
            <dc:creator><![CDATA[Winston Wang]]></dc:creator>
            <pubDate>Wed, 02 Oct 2024 08:03:54 GMT</pubDate>
            <atom:updated>2025-04-26T17:41:32.884Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*PLZX3wBdnASgPkrm_K3puw.png" /></figure><p>目前市面上针对大语言模型微调的 python库，例如unsloth and lamini，在Apple M系列芯片上都不支持GPU加速。Apple Silicon芯片下的Mac使用MLX进行finetuning是一个很好的选择。</p><p>MLX是Apple公司开发的一个机器学习框架,专门为Apple Silicon芯片优化设计。除了用来训练大语言模型，MLX也提供了LoRA QLoRA finetuning的支持。美中不足的是，目前MLX的文档不是很清晰，给使用者造成了一些不便或困扰，这篇教程旨在解决这些问题，提供一个相对新手友好的Finetuning方案。</p><h4>安装必要的包</h4><p>安装mlx和它的依赖。</p><pre>!pip install -Uqq mlx mlx_lm transformers datasets</pre><ul><li>在pip前加入! 是告诉Jupyter Notebook要执行一个系统命令，而不是Python代码 ，本文所有代码都在Jupyter Notebook中执行</li><li>-U是升级指定的包到最新版本，-qq会使pip在安装过程中几乎不显示任何输出，除非发生错误</li></ul><p>本文还需要的包有: huggingface_hub pandas matplotlib.pyplot pathlib 和json。如果你的环境中还没有相应的包，请先安装上述所有包。</p><h4><strong>从HuggingFace上加载模型</strong></h4><p>直接下载原版的大语言模型需要有一个Hugging Face的账号。搜索模型并进入model card页面，如果看到下边这个图标，那就是要求获取访问权限。以meta的Llama3.1模型为例，You need to agree to share your contact information to access this model。按照提示输入自己的联系方式以获取权限。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/116/1*BGZlU8l0WHpbfa_cTHdBcg.png" /><figcaption>Gated Models</figcaption></figure><p>当你获取了模型的权限以后，进入到自己的设置中，找到access tokens选项。然后create new token，下边会出现很多的选项，可以根据实际情况进行勾选。为了下载模型，一定要在Repositories下边勾选 [ √] Read access to contents of all public gated repos you can access。保存token以后，即可获得 类似“hf_xxxxxxxx”的 token。一定要在这个时候复制并保存自己的token，一旦离开这个页面，token就不会再出现。当然你也可以对已经创建的token进行编辑，每次编辑后 value of token （也就是“hf_xxxxxxxx”）都会更改，之前的value就会失效，需要在代码里更新token。</p><pre>from huggingface_hub import login<br>access_token = &#39;hf_DB*************lR&#39; # paste your token here<br>login(access_token)</pre><p>载入模型时，我们可以将tonkenizer 一并加载，方便后续操作：</p><pre>from mlx_lm import generate, load<br>model, tokenizer = load(&quot;google/gemma-2-2b-it&quot;)</pre><blockquote>关于MLX包的一些问题，<a href="https://medium.com/tutorial-by-winston-wang/mlx%E6%A1%86%E6%9E%B6%E8%AF%A6%E8%A7%A3-%E6%A8%A1%E5%9E%8B%E6%94%AF%E6%8C%81-%E5%BE%AE%E8%B0%83%E6%96%B9%E6%B3%95%E4%B8%8Eapple-silicon%E4%BC%98%E5%8C%96-a1588f89fb8a"><strong>点击这里</strong></a>查看。</blockquote><p>接下来编辑一个prompt对模型进行提问，测试模型的载入。</p><pre>prompt = &quot;What is under-fitting and overfitting in machine learning?&quot;<br><br>messages = [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}]<br>prompt = tokenizer.apply_chat_template(<br>    messages, tokenize=False, add_generation_prompt=True<br>)<br><br>response = generate(model, tokenizer, prompt=prompt, verbose=True, max_tokens = 100)</pre><p>模型运行后可以看到给出的回复，说明载入正常。（该问题的回答并不完全，这是正常的。为了节约时间，这里max_tokens设置为了100（缺省值），限制了模型输出的长度。）</p><h4>载入数据集</h4><p>这里我们使用一个结合了 huggingface上7 个机器学习、数据科学和人工智能问答数据集中的问答而创建的机器学习问答数据集。这个数据集中的数据格式已经按照 Gemma 要求的方式格式化。如果使用的是其他的模型，应先查看模型对应的数据格式。 <a href="https://huggingface.co/datasets/win-wang/Machine_Learning_QA_Collection">点击这里</a>查看该数据集的卡片，也可以找到制作该数据集的jupyter notebook，利用其中的代码把数据格式更改为需要的格式。</p><pre>from datasets import load_dataset<br><br>ds = load_dataset(&quot;win-wang/Machine_Learning_QA_Collection&quot;)</pre><p>将数据集转化为DataFrame格式，并检查</p><pre>import pandas as pd<br>train_set = pd.DataFrame(ds[&quot;train&quot;])<br>dev_set = pd.DataFrame(ds[&quot;validation&quot;])<br>test_set = pd.DataFrame(ds[&quot;test&quot;])<br>train_set.head()</pre><p>MLX对数据的要求是放在可以进行随机读取的容器中，这里我们把它转化为python自带的list:</p><pre>def preprocess(dataset):<br>    return dataset[&quot;text&quot;].tolist()<br>    <br>train_set, dev_set, test_set = map(preprocess, (train_set, dev_set, test_set))</pre><p>此时我们就准备好了训练集、开发集和测试集。</p><h4>模型微调</h4><p>载入相关包</p><pre>import matplotlib.pyplot as plt<br>import mlx.optimizers as optim<br>from mlx.utils import tree_flatten<br>from mlx_lm import load, generate<br>from mlx_lm.tuner import train, TrainingArgs<br>from mlx_lm.tuner import linear_to_lora_layers<br>from pathlib import Path<br>import json</pre><p>创建一个目录来保存适配器配置和权重</p><pre>adapter_path = Path(&quot;adapters&quot;)<br>adapter_path.mkdir(parents=True, exist_ok=True)</pre><p>设置LoRA参数</p><pre>lora_config = {<br> &quot;lora_layers&quot;: 8,<br> &quot;lora_parameters&quot;: {<br>    &quot;rank&quot;: 8,<br>    &quot;scale&quot;: 20.0,<br>    &quot;dropout&quot;: 0.0,<br>}}</pre><p>将 LoRA 配置保存到适配器路径</p><pre>with open(adapter_path / &quot;adapter_config.json&quot;, &quot;w&quot;) as fid:<br>    json.dump(lora_config, fid, indent=4)</pre><p>设置训练的参数</p><pre>training_args = TrainingArgs(<br>    adapter_file=adapter_path / &quot;adapters.safetensors&quot;,<br>    iters=200,<br>    steps_per_eval=50<br>)</pre><p>先冻结基础模型</p><pre>model.freeze()</pre><p>将线性层转换为 LoRA 层</p><pre>linear_to_lora_layers(model, lora_config[&quot;lora_layers&quot;], lora_config[&quot;lora_parameters&quot;])</pre><p>查看可以训练的模型参数</p><pre>num_train_params = (<br>    sum(v.size for _, v in tree_flatten(model.trainable_parameters()))<br>)<br>print(f&quot;Number of trainable parameters: {num_train_params}&quot;)</pre><p>将模型设置为训练模式</p><pre>model.train()</pre><p>设置优化器</p><pre>opt = optim.Adam(learning_rate=1e-5)</pre><p>构建一个类用来记录训练过程中的数据</p><pre>class Metrics:<br>    train_losses = []<br>    val_losses = []<br>    def on_train_loss_report(self, info):<br>        self.train_losses.append((info[&quot;iteration&quot;], info[&quot;train_loss&quot;]))<br>    def on_val_loss_report(self, info):<br>        self.val_losses.append((info[&quot;iteration&quot;], info[&quot;val_loss&quot;]))<br>        <br>metrics = Metrics()</pre><p>将所有参数放入，开始微调</p><pre>train(<br>    model = model,<br>    tokenizer = tokenizer,<br>    args = training_args,<br>    optimizer = opt,<br>    train_dataset = train_set,<br>    val_dataset = dev_set,<br>    training_callback = metrics<br>)</pre><p>微调结束后画出损失值随着迭代的折线图</p><pre>train_its, train_losses = zip(*metrics.train_losses)<br>val_its, val_losses = zip(*metrics.val_losses)<br>plt.plot(train_its, train_losses, &#39;-o&#39;)<br>plt.plot(val_its, val_losses, &#39;-o&#39;)<br>plt.xlabel(&quot;Iteration&quot;)<br>plt.ylabel(&quot;Loss&quot;)<br>plt.legend([&#39;Train&#39;, &quot;Valid&quot;]);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*5i_HbfiEwTb0yfC34BmZYg.jpeg" /></figure><p>会得到类似这样的一张图。</p><p><strong>如果你清楚自己想要的迭代等 也可以使用shell command 直接进行微调</strong></p><p>例如：</p><pre>!mlx_lm.lora --model google/gemma-2-2b-it \<br>    --train \<br>    --iters 200 \<br>    --data data</pre><h4>使用原模型和适配器文件中的参数来构建LoRA后的模型</h4><pre>model_lora, _ = load(&quot;google/gemma-2-2b-it&quot;, adapter_path=&quot;adapters&quot;)</pre><p>让微调后的模型再次回答之前的问题</p><pre>generate(model_lora, tokenizer, prompt=prompt, verbose=True)</pre><h4>困惑度的计算</h4><p>这里用到的模型计算困惑度没有意义，如果你微调后的模型可以计算困惑度，可以使用如下代码:</p><p>其中 adapter-path 和 data 分别是adapter和data的储存路径</p><pre>!mlx_lm.lora \<br>    --model  &#39;google/gemma-2-2b-it&#39;\<br>    --adapter-path &quot;adapters&quot; \<br>    --data &quot;data&quot; \ <br>    --test</pre><h4>融合</h4><p>为了方便未来的使用，我们也可以将微调的参数和原始的模型融合，默认使用当前工作目录adaptes 中保存的文件</p><pre>!mlx_lm.fuse --model &quot;google/gemma-2-2b-it&quot;</pre><p>这时，融合后的模型会自动保存在当前工作目录的 lora_fused_model文件夹中</p><p>如果你需要自定义路径和保存可以使用如下参数：</p><pre>--model # 本地模型的路径 或 huggingface上的模型<br>--save-path # 融合后的模型保存的目录<br>--adapter-path # 训练后的适配器和权重保存的目录</pre><p>也可以使用 --upload-repo 和 --hf-path 直接将训练好的模型上传到huggingface上，例如:</p><pre>!mlx_lm.fuse --model &quot;google/gemma-2-2b-it&quot; \<br>	--upload-repo &quot;你自己的repo名称&quot; \<br>	--hf-path &quot;google/gemma-2-2b-it&quot;</pre><p>--hf-path 用来归因和模型版本控制。</p><blockquote><a href="https://github.com/wins-wang/tutorials/blob/9f2f158f16c8dbdfb2b18ed038273613efecf967/Fine%20Tuning%20With%20MLX%20on%20Apple%20Silicon.ipynb">点击这里</a>查看并下载完整的Jupyter Notebook。</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f5f9959d8961" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tutorial-by-winston-wang/%E5%9C%A8apple-silicon%E4%B8%8A%E4%BD%BF%E7%94%A8mlx%E5%AF%B9%E6%A8%A1%E5%9E%8B%E8%BF%9B%E8%A1%8C%E5%BE%AE%E8%B0%83%E7%9A%84%E6%96%B0%E6%89%8B%E6%95%99%E7%A8%8B-f5f9959d8961">在Apple Silicon上使用MLX对模型进行微调的新手教程</a> was originally published in <a href="https://medium.com/tutorial-by-winston-wang">Tutorial by Winston Wang</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>