<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Siddharth Patwardhan on Medium]]></title>
        <description><![CDATA[Stories by Siddharth Patwardhan on Medium]]></description>
        <link>https://medium.com/@sierra_india_delta?source=rss-36a701355647------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*1fINk9mVH4n8d4ru</url>
            <title>Stories by Siddharth Patwardhan on Medium</title>
            <link>https://medium.com/@sierra_india_delta?source=rss-36a701355647------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 08:53:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sierra_india_delta/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How Mathematics Won AI — A Story in Three Acts]]></title>
            <link>https://medium.com/@sierra_india_delta/how-mathematics-won-ai-a-story-in-three-acts-9215fec8fd05?source=rss-36a701355647------2</link>
            <guid isPermaLink="false">https://medium.com/p/9215fec8fd05</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Siddharth Patwardhan]]></dc:creator>
            <pubDate>Sat, 27 Jun 2026 09:08:58 GMT</pubDate>
            <atom:updated>2026-06-27T09:08:58.142Z</atom:updated>
            <content:encoded><![CDATA[<h4>… and what was left behind</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oaDsSgAwCQ1IgMuDM01LgQ.png" /><figcaption>Forty years of ACL, measured. The lines tell the story better than words can.</figcaption></figure><p>ACL — the <em>Annual Meeting of the Association for Computational Linguistics </em>— starts next week. A premier AI conference, it draws thousands of attendees each year. Its name would suggest that all are there for their interest in linguistics and language — building the best AI through their understanding of these topics. But, if you haven’t followed the trajectory of this field over the years, you would be surprised to learn that the conference is now largely <em>mathematics</em> and very little<em> linguistics</em>.</p><p>The story of how ACL lost its linguists is largely the story of the evolution of modern AI. In this post, I will briefly trace the path of how we got here — through the lens of this unique conference whose character has morphed over the years. Slowly at first, then all of a sudden. The chart at the top neatly captures the change over time. It plots percentage of “linguistic” vs. “non-linguistic” papers at the ACL conference over the years (based on keyword-based classification). Let me walk you through the eras of ACL.</p><h4>Act 1: In the beginning…</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g2V-SyFKOeYA0PRZd0pdBA.png" /><figcaption>The papers from ACL’s early decades. Aravind Joshi’s 1969 work — typewritten, formal, linguistic — is a window into that era.</figcaption></figure><p>The first ACL conference took place in 1963 in Denver, CO. At a time when the internet did not exist. At a time when a computer filled an entire room, and the idea of a machine that could read and understand language felt equal parts science and science fiction. We have been at this problem for a long, long time.</p><p>For many decades since that first ACL, our thinking about understanding<em> language understanding</em> (or NLP) was largely “functional” — linguistic, in a sense. Quite naturally, the instinct and scientific thought process was to observe language use in the wild, break down the key pieces and study these pieces in-depth. <em>What are the fundamental decomposable pieces of “language” — words, phrases, sentences? Are there different categories of these? Is there an implicit “structure” imposed on words through language? How does meaning emerge from the structures?</em> Such were the questions that drove the development of AI (such as it was) then.</p><p>Unsurprisingly, the ACL community and the leaders in the field were primarily linguists by training, with an inclination towards statistical and computational analysis. Some of the seminal research from this era reflects the character and paradigm of the community. The papers from this era read like a linguistics seminar — Chomsky on formal grammars, Hays on dependency systems, Bar-Hillel on phrase structure properties. Aravind Joshi (who later won the ACL Lifetime Achievement Award) is famous for creating <em>Tree-Adjoining Grammars (TAG) </em>around this time. His heavily cited 1969 paper began exploring “mildly context-sensitive” grammars.</p><p>Walk the halls of ACL in 1985 and you would find linguists, logicians, and cognitive scientists debating the structure of language. The proceedings were steeped in linguistics theory. The assumption underlying all of this work was profound and implicit: to build a machine that understands language, you first had to understand how language itself works. The pipeline that emerged reflects that belief — tokenization, part-of-speech tagging, syntactic parsing, semantic role labeling, discourse modeling, etc. Each component is distinct, reasoned about, and analyzed independently. It was linguistics as the engineering blueprint.</p><h4><strong>Act 2: The Shift</strong></h4><figure><img alt="Watson vs. Ken Jennings and Brad Rutter. Yorktown Heights, NY, 2011. I had a front-row seat." src="https://cdn-images-1.medium.com/max/1024/1*SurknhL5WfoBj7ysw6cKJg.png" /><figcaption>February 2011. Watson vs. Ken Jennings and Brad Rutter, Yorktown Heights, NY… I had a front-row seat.</figcaption></figure><p>A couple of decades since that first ACL conference, you start to see the initial indications of change. It didn’t happen overnight. And it didn’t start at ACL. The first tremors came from the speech community in the 1980s and 90s. Researchers at IBM, working on automatic speech recognition — a problem adjacent to NLP but deeply connected to it — were making a bet on mathematics and statistics. Instead of modeling the rules of language explicitly, <em>what if you just collected enough data and let the statistics do the work?</em> The result was groundbreaking research on statistical language modeling that demonstrated remarkably surprising outcomes.</p><p>Dr. Fred Jelinek, one of the architects of that statistical revolution at IBM, captured the mood of the era with characteristic bluntness: <em>“Every time I fire a linguist, the performance of our speech recognizer goes up.”</em> A quote often remembered by scientists who came up during those years, plainly reflected the power of mathematical modeling over linguistic analysis.</p><p>Through the 1990s and into the 2000s, statistical methods creep into NLP task by task. You can trace this shift in the ACL proceedings year by year. Statistical models start appearing in the late 1980s. By the 2000s they dominate individual tasks. The linguists are still there — but they are increasingly adopting statistical ML in everything they do.</p><p>Part-of-speech tagging. Named entity recognition. Machine translation. In each case, a carefully engineered linguistic system, built by experts who deeply understood the problem, would be enhanced and then outperformed by a statistical model trained on enough data. The model didn’t know what a noun was. It didn’t know what a sentence meant. But it knew what to do, just going on what it had seen before.</p><p>When I started my PhD in the early 2000s, this was the river we swam in — parse trees, treebanks, semantic role labels, all being now built with statistical approaches like regression methods, decision trees, and later, SVMs, CRFs and the like. We were still thinking about language understanding through a linguistic lens, but each of the “linguistic components” were built upon a solidly mathematical foundation — each being trained with machine learning at the core.</p><p>Jump forward to 2011. <em>IBM Watson</em> famously beat human grand champions — Ken Jennings and Brad Rutter — on a nationally televised event that captured the attention of a nation and the world. I was there! As a young scientist on IBM’s Watson team, I had a front row seat (literally and figuratively) to this momentous event in time. A massive system (for its time), we called the technology in Watson <em>DeepQA.</em></p><p>DeepQA was still using ML-based trained linguistic subsystems at its core, controlled by a single massive logistic regression model (but tiny by today’s standards). At the time, we published details of the system in the <em>IBM Journal of Research and Development. </em>A quick look at the abstract for the first paper — <em>How Watson Reads a Clue</em> — tells you all you need to know about the linguistic foundations of this pioneering work. A quote:</p><blockquote>“… question analysis uses Watson’s parsing and semantic analysis capabilities: a deep Slot Grammar parser, a named entity recognizer, a co-reference resolution component, and a relation extraction component. We apply numerous detection rules and classifiers using features from this analysis to detect critical elements of the question…”</blockquote><p>Then, in 2012, it all changed. Inspired by publication that year — from the vision community this time — a paradigm shift began.</p><p>With ImageNet, deep neural networks made a huge leap in image recognition. <em>If unstructured neural networks could learn to see, could they learn to read?</em> The answer, it turned out, was yes. <em>Word2Vec</em> (2013) showed that dense vector representations of words captured something real about meaning — relationships that no linguist had explicitly encoded. <em>Seq2Seq</em> showed you could map arbitrary sequences to arbitrary sequences with a neural network, making machine translation a solved-enough problem almost overnight.</p><p>There was deep skepticism that approaches so indifferent to linguistic theory could ever match systems built on decades of carefully reasoned structure. Prof. Ray Mooney captured the sentiment perfectly:<em> ‘You can’t cram the meaning of a whole %&amp;!$# sentence into a single $&amp;!# vector!’</em> He wasn’t wrong about the linguistics. But the mathematics didn’t care. With strong evidence of neural ML methods demonstrating superior language understanding, the field rapidly moved to exploring that space of math-based methods — GloVe, CNN, RNN, LSTM, each following the next. You slowly but surely saw the field transform — and the ACL proceedings with it.</p><h4>Act 3: Endgame</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cSZENIxgxbqjeMiwoaPmGA.png" /><figcaption>Vaswani et al., 2017. — the paper made the entire NLP pipeline feel unnecessary</figcaption></figure><p>By 2017, ACL had completed its transformation. The conference that began as a linguistics seminar was now, overwhelmingly, a math-first conference. The graph at the top of this post shows exactly when the lines crossed — announced by a single paper that changed everything that followed.</p><p>A team at Google published a paper called <em>“Attention is All You Need.” </em>The transformer architecture it introduced didn’t just improve on existing methods. It made the entire NLP pipeline — the tokenizers, the parsers, the semantic role labelers, the discourse models — feel unnecessary. You didn’t need to tell the model what a noun phrase was. You didn’t need to parse the sentence. You fed in the text, you trained on enough data, and <em>gradient descent </em>figured out the rest. No linguist required. No theory of language needed. Just mathematics, data, and compute. Going into 2017, mathematics had crossed over linguistics at ACL conferences. It has never looked back.</p><p>The transformer didn’t just win a benchmark. It displaced a research paradigm.</p><p>For decades, when an NLP system failed, you could ask why. You could open it up, look at the parse tree, examine the semantic role labels, trace the failure to a specific linguistic component and fix it. The system was interpretable not because we built interpretability in — but because the system <em>was</em> a theory of language, expressed in code. Failure was informative. Just the act of building the system taught you something about language.</p><p>With large language models, that changed. These systems work almost unsettlingly well. But when they fail, we struggle to explain why. When they succeed, we struggle to explain that too — though a growing interpretability research community is trying. We have traded a transparent theory of language for an opaque mathematical function that approximates it. The function is better, but is not a theory of language.</p><p>What did we lose, concretely? We lost the questions. The early ACL community asked: what is the functional role of syntax in understanding? How does coreference work across a discourse? What does it mean for a machine to <em>know</em> the meaning of a word? These questions about language, about cognition teach us something about what understanding actually is. These questions have become harder to find at the main conference, but often do show up in workshops and side discussions.</p><p>I don’t say this as criticism. Outcomes matter. Benchmarks matter. The systems we have built have changed the world.</p><p>But, somewhere along the way, the fundamental quest for understanding language understanding got left behind.</p><h4>Concluding Remarks</h4><p>Does it matter? I find myself genuinely uncertain.</p><p>The mathematical approach has delivered things the linguistic paradigm never could — systems that translate, summarize, reason, and converse at a level that would have seemed like magic to the ACL community of 1985, and almost unimaginable to the community of 1963. That is not nothing. That is, in fact, extraordinary.</p><p>And yet I find something missing. I spent 25 years in this field — through the linguistic pipeline era, through Watson, through the transformer revolution, and into the age of LLMs. I made the same transitions everyone else did. My current work runs on the very models I’ve been describing. But I find myself coming back to the question the early ACL community asked alongside <em>“can a machine do this?”</em> — namely, <em>“what does it tell us about language that a machine can do this?”</em></p><p>That second question feels harder to find at ACL in 2026. I’m not sure we’ve fully reckoned with what we traded away when we stopped asking it.</p><p>ACL starts next week. Thousands of researchers will gather to share work that will leave us in awe and wonder. It will inspire the next generation of scientists. Much of it will be brilliant. Some will change the world. But somewhere in that conference, in a workshop or in side conversations, there will be a small group of researchers still asking the old questions — about syntax, about meaning, about what it actually takes for a machine to understand.</p><p>I will be looking for them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*64pGh6ofe1SAbA8yOXVnjw.png" /><figcaption>ACL today — different in size, different in character from its early days. Two booths I know well.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9215fec8fd05" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>