<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alex Rosenrauch on Medium]]></title>
        <description><![CDATA[Stories by Alex Rosenrauch on Medium]]></description>
        <link>https://medium.com/@alex.r.rosenrauch?source=rss-e1ea093b53b5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*O6mmPGykhsO_OPKVmnYmgw.jpeg</url>
            <title>Stories by Alex Rosenrauch on Medium</title>
            <link>https://medium.com/@alex.r.rosenrauch?source=rss-e1ea093b53b5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 18:24:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alex.r.rosenrauch/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AI in Law: From Hype to Practice — A New Study’s Insights and the Road Ahead]]></title>
            <link>https://medium.com/@alex.r.rosenrauch/ai-in-law-from-hype-to-practice-a-new-studys-insights-and-the-road-ahead-0773434e7c8a?source=rss-e1ea093b53b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/0773434e7c8a</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[law]]></category>
            <category><![CDATA[legaltech]]></category>
            <dc:creator><![CDATA[Alex Rosenrauch]]></dc:creator>
            <pubDate>Wed, 05 Mar 2025 17:24:27 GMT</pubDate>
            <atom:updated>2025-03-05T17:24:27.301Z</atom:updated>
            <content:encoded><![CDATA[<h3>AI in Law: From Hype to Practice — A New Study’s Insights and the Road Ahead</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LGNuhwqnCwcWZJ29GNW9_A.png" /></figure><p>Artificial intelligence has stormed onto the legal scene, promising to revolutionise how lawyers work. Generative AI tools like OpenAI’s ChatGPT burst into public consciousness with astonishing feats — for example, GPT-4 even <strong>passed the bar exam</strong> in a <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4389233">recent study</a>​.</p><p>These advances fueled excitement that AI could automate legal drudgery and perhaps even outperform human lawyers at certain tasks. At the same time, early experiences tempered the hype. ChatGPT’s performance on law school exams hovered around a <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335905">mediocre C+ average</a>, and it has a <strong>well-documented tendency to “hallucinate”</strong> — in other words, to confidently make up fake facts or case citations​. One notorious incident involved lawyers <a href="https://www.reuters.com/legal/new-york-lawyers-sanctioned-using-fake-chatgpt-cases-legal-brief-2023-06-22/#:~:text=NEW%20YORK,%20June%2022%20,an%20artificial%20intelligence%20chatbot,%20ChatGPT">being sanctioned after their AI-written brief cited six non-existent cases</a>.</p><p>Thus, legal professionals have been torn between AI’s tantalising potential and its pitfalls. Some enthusiasts predict <strong>fundamental restructuring</strong> of legal work — envisioning AI handling countless tasks and even replacing certain types of lawyers​. Skeptics counter that while AI may streamline routine chores, it’s unlikely to <strong>alter the core of lawyering</strong> which relies on human judgment​.</p><p>Amid this debate, <strong>billions of dollars</strong> are flowing into AI-driven legal startups (<a href="https://www.harvey.ai/blog/harvey-raises-series-d">including Harvey</a>) and major legal tech companies are racing to integrate AI into their platforms​. From elite law firms to legal aid groups, professionals are experimenting with “AI copilots” in tasks like research and drafting. Even judges are cautiously exploring AI assistance in drafting opinions​. Law schools, in turn, face tough questions about how to prepare new lawyers for an AI-augmented future.</p><p>In this context, a new study — <em>“</em><a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5162111"><em>AI-Powered Lawyering: AI Reasoning Models, Retrieval Augmented Generation, and the Future of Legal Practice</em></a><em>”</em> (Schwarcz et al., 2024) — offers timely empirical insight. This is the most recent randomised controlled trial to evaluate AI <strong>tools specifically for legal work</strong>, shedding light on how generative AI might truly transform (or not transform) the practice of law. This blog reviews the study’s key findings and discusses what they mean for the legal profession, from daily practice to ethics and the future of lawyering.</p><h3><strong>Key findings from the new study</strong></h3><p>The study by Schwarcz and colleagues set out to test whether new AI innovations can overcome the limitations of earlier models. The authors focused on two emerging technologies: <strong>Retrieval Augmented Generation (RAG)</strong> and <strong>“AI reasoning” models.</strong> RAG systems (like <em>VLex’s Vincent AI</em>) combine a chatbot with a legal database — they <strong>retrieve relevant legal sources</strong> and ground the AI’s answers in those documents, which helps keep the output factual and supported by citations. In contrast, AI reasoning models (the study used OpenAI’s <em>“o1-preview”</em> model) employ more sophisticated problem-solving methods — they break down complex tasks into steps or follow structured prompts to improve logical reasoning. The researchers randomly assigned upper-level law students to complete six typical lawyering tasks under different conditions: one group had access to Vincent AI (the RAG tool), another used the o1-preview reasoning AI, and a control group worked with <strong>no AI assistance</strong>​.</p><p>The tasks ranged from drafting emails and persuasive letters to writing a legal memo, analysing a complaint, preparing a motion, and drafting a contract (NDA). All student work was graded blindly by legal instructors on various metrics (analysis quality, clarity, professionalism, etc.).</p><p>The study reported several notable findings:</p><ul><li><strong>Faster turnaround:</strong> Both AI-powered groups finished their assignments <strong>significantly faster</strong> than the control group. In 5 out of 6 tasks, having AI doubled (or more) the participants’ productivity. For instance, students using Vincent AI completed tasks <strong>38% to 115% faster</strong>, and those using o1-preview saw <strong>34% to 140% gains in speed</strong>​. These efficiency boosts confirm that generative AI can save substantial time on legal drafting and research, aligning with other research showing productivity gains from AI assistance in writing​. Importantly, this speedup did <em>not</em> come at the expense of quality — quite the opposite.</li><li><strong>Improved work quality:</strong> The quality of the students’ legal work <strong>improved markedly</strong> with AI help on most assignments. Participants using AI produced analyses and documents that earned higher scores on legal reasoning, organisation, and clarity compared to those working alone​. This is a striking result because it <strong>contrasts with earlier studies</strong> of GPT-4 in legal tasks, which found <strong>little to no improvement in quality</strong> despite faster completion times​. In this new experiment, AI assistance led to statistically significant quality boosts on <strong>four of the six tasks</strong> evaluated​. For example, the AI-assisted students wrote more coherent legal memos and more persuasive letters than their peers without AI. On average, quality scores rose by double-digit percentages — the reasoning AI’s help yielded about <strong>10%–28% improvements</strong> in quality metrics for certain complex tasks​. In short, modern legal AI tools aren’t just speeding up the grunt work; they’re also enhancing the end product’s substance and polish in many cases.</li><li><strong>Deeper analysis vs. factual accuracy:</strong> The two AI systems each had strengths and weaknesses, highlighting a trade-off between <strong>analytic depth and factual reliability</strong>. OpenAI’s o1-preview (the reasoning model) tended to significantly improve the <strong>analytical depth</strong> of students’ work — their arguments were more nuanced and better reasoned with this AI’s guidance​. However, the o1-preview model sometimes <strong>“hallucinated” content</strong> (for instance, citing a case or fact that doesn’t actually exist). In the study, assignments done with o1-preview contained a handful of such errors (a total of 11 hallucinated references across all users)​. On the other hand, the Vincent AI (RAG tool) excelled at factual accuracy. It almost never introduced fake legal citations or non-existent facts — the work produced with Vincent had <strong>no more hallucinations than the human-only control group</strong>​. In fact, <strong>students without AI made 4 factual errors, versus only 3 with Vincent, while the o1 group made 11​.</strong> The flip side is that Vincent’s contributions to deep legal analysis were a bit more limited; it improved overall quality but not as dramatically in the “analytical reasoning” dimension as the reasoning model did​. These findings make intuitive sense: a system tethered to real case texts will be trustworthy about the law but might not fundamentally reorganise a complex argument for you, whereas a free-form reasoning AI can brainstorm novel arguments (at risk of veering off into fiction).</li><li><strong>Limits of AI — The NDA surprise:</strong> Interestingly, there was one task in which neither AI tool provided a boost: drafting a basic non-disclosure agreement. For that transactional drafting assignment, students with and without AI performed about the same​. Why might that be? The researchers point out two factors​: First, unlike the other tasks which were litigation-oriented, the NDA was a <strong>transactional document</strong>, an area where Vincent AI (marketed more for litigators) may be less effective. Second, students were given a standard template for the NDA, so even those without AI had a big head start. In this scenario of a <strong>routine, form-based task</strong>, the AI had less room to demonstrate advantage. This highlights an important nuance — AI shines more when drafting or analysing complex, open-ended materials (e.g. a persuasive letter or legal memo) rather than when filling in blanks on a boilerplate form. In practice, many transactional lawyers already use templates and clauses libraries; AI might add less value there beyond what a template provides.</li><li><strong>Synergy of new approaches:</strong> Overall, both types of AI — retrieval-based and reasoning-based — showed strong benefits, and each mitigated different shortcomings of the other. The authors suggest that the <strong>next generation of legal AI tools may combine these approaches</strong> to get the best of both worlds​. In other words, a future system that integrates robust legal database retrieval with advanced reasoning could generate highly sophisticated legal work <em>and</em> stay grounded in real sources. Such hybrid AI could potentially produce work product approaching what a skilled lawyer might write, but in a fraction of the time. This synergy is exactly where many legal tech companies are headed, as they refine AI “copilots” that can reason through a user’s problem while double-checking the law as they go.</li></ul><p>In summary, the study delivers an optimistic message: when used properly, <strong>AI can significantly enhance a lawyer’s speed and output quality</strong> on many tasks. This marks a turning point. Early “first-wave” tools like base GPT-4 showed promise but also inconsistency in legal settings; now, with domain-specific grounding and reasoning enhancements, AI appears far more capable of handling nuanced legal work​. Of course, these findings come from a controlled experiment with law students — real-world practice will introduce additional complexities. Nonetheless, the research provides concrete evidence that the latest AI models can function as powerful assistants in legal work. The question becomes: <strong>what does this mean for practicing lawyers and the future of the profession?</strong></p><h3>What does this mean for legal practice?</h3><p>If AI tools can truly boost both the <strong>efficiency</strong> and <strong>effectiveness</strong> of legal work, the implications are far-reaching. For legal practitioners, one immediate impact is on <strong>productivity and workload management</strong>. Routine tasks that used to eat up hours — researching case law, drafting background sections of a brief, composing client emails or demand letters — could be completed in a fraction of the time with AI assistance. Lawyers might handle larger caseloads or spend more time on high-level strategy now that first drafts and research memos can be generated quickly. Clients, in turn, may come to expect faster turnaround and could benefit from lower bills if mundane work takes less lawyer time.</p><p>In fact, improvements in efficiency might help address the notorious <strong>access-to-justice gap</strong>: legal aid lawyers and clinics with strained resources might leverage AI to serve more clients with better quality work​. <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4733061">Early explorations in this vein are promising</a> — for example, experiments have shown generative AI can help draft simple pleadings and informational guides for self-represented litigants, suggesting technology may assist those who can’t afford full legal representation​.</p><p>However, along with these benefits come serious <strong>ethical and professional considerations</strong>. The legal profession is built on accuracy, judgment, and accountability, and introducing AI into the workflow doesn’t remove a lawyer’s duty to get things right. The hallucination problem, while reduced by new tools, is a cautionary reminder that <strong>AI can confidently produce incorrect or fictitious information</strong>. Lawyers must therefore adopt a mindset of <strong>trust but verify</strong> with AI outputs. As the <a href="https://www.reuters.com/legal/new-york-lawyers-sanctioned-using-fake-chatgpt-cases-legal-brief-2023-06-22/#:~:text=NEW%20YORK,%20June%2022%20,an%20artificial%20intelligence%20chatbot,%20ChatGPT">infamous New York case</a> demonstrated, blindly accepting an AI-generated brief can lead to embarrassing and sanctionable mistakes​. In that case, the judge noted there is nothing inherently wrong with using AI in law, <em>so long as lawyers fulfill their “gatekeeping” role of ensuring accuracy</em>​.</p><p>Going forward, we can expect legal professionals to emphasise that any AI-drafted content <strong>must be thoroughly reviewed and validated by a human lawyer</strong> before it goes out the door. Using AI in practice might require new quality-control workflows — for instance, cross-checking every AI-cited case and fact against primary sources (a step that RAG tools like Vincent AI make easier by providing the source documents alongside the answer).</p><p>Another implication is the need to safeguard <strong>client confidentiality and data security</strong>. Many AI tools operate in the cloud, raising concerns that sensitive case information typed into a chatbot could leak or be used to train someone else’s model​. Law firms are already grappling with these issues: some are opting for on-premises or private cloud AI solutions, and regulators are weighing in with guidance on protecting privileged data. We are already seeing the emergence of <strong>legal-specific AI platforms</strong> that explicitly guarantee no secondary use of the data (just like OpenAI for instance, which allows opt-outs so that API usage won’t feed into model training). In the meantime, prudent lawyers will avoid inputting any client-identifying details into public AI tools, or will scrub and anonymise data before doing so. Ethical duties of confidentiality and competence are evolving to include understanding these AI-related risks.</p><p>The infusion of AI into legal workflows also raises a big-picture question: <strong>How will this shift the roles and skills of lawyers?</strong> The study’s results hint that junior lawyers or law students can produce work closer to the level of experienced lawyers when guided by AI. Does that mean fewer junior associates will be needed for tasks like legal research, document review, or first-draft writing? It’s possible. If a partner can have an AI generate a decent first draft of a memo and then personally refine it, they might rely less on a first-year associate’s work. On the other hand, associates using AI themselves can work much faster and add more value than before. We might see young lawyers handling a higher volume of substantive work with AI’s help — but this could come at the cost of <strong>traditional training</strong> if not managed carefully. Much of a lawyer’s skill is historically built through slogging via research and drafting assignments; if AI does the heavy lifting, firms will need to ensure junior lawyers still learn the underlying craft and critical thinking, perhaps by focusing on reviewing AI outputs, spotting its mistakes, and sharpening the analysis further.</p><p>There’s also the human element. <strong>Lawyering has an irreducible aspect of judgment, creativity, and interpersonal skills</strong> that AI cannot (at least for now) replicate​. The best use of AI is likely as an augmenter — handling information processing and suggesting solutions — while the lawyer provides oversight, ethical judgment, and tailored advice to the client. Many experts foresee an “<strong>augmented lawyering</strong>” model​ where AI is a tireless research assistant or drafting tool, and the lawyer is the strategist, decision-maker, and communicator who guides the AI and evaluates its output. This dynamic could actually enhance the lawyer-client relationship: freed from some paperwork, lawyers can spend more time on client interaction, counseling, and advocacy. But it will require lawyers to develop <strong>new skills</strong>, such as crafting effective prompts for AI, understanding the tool’s limitations, and staying up-to-date on AI advancements. Law schools are already beginning to integrate AI literacy into the curriculum, recognising that tomorrow’s lawyers must know how to responsibly leverage AI tools as part of their toolkit.</p><h3>The future of “AI-Powered Lawyering”</h3><p>What does the road ahead look like as AI continues to evolve in the legal domain? This recent study provides strong evidence that we’re entering a <strong>“second wave” of legal AI</strong> tools that are far more capable than the first generation that emerged just a couple years ago​. If the pace of improvement continues, I think we can anticipate several trends and transformations in legal practice:</p><ul><li><strong>Mainstream adoption of “AI Copilots”:</strong> In the near future, using an AI assistant for legal work may become as common as using online legal research services. Major players like Westlaw and <a href="https://www.lexisnexis.com/community/pressroom/b/news/posts/lexisnexis-and-openai-announce-plan-to-deliver-custom-ai-technology-for-legal-professionals?srsltid=AfmBOooUGNf60EvKnt6WJEB5PNXXcaRSq-OT1uTQlmFLjmZip_ud2OMc">LexisNexis</a> have already integrated generative AI into their platforms​, meaning lawyers can query an AI that has access to the full universe of legal databases and get draft answers with citations. As these integrated tools prove their worth, legal professionals will incorporate them into standard practice. We can expect that within a few years (at most!), tasks like drafting a litigation brief, composing contract clauses (if not already), or writing a client memo will routinely start with an AI-generated draft that the lawyer edits — flipping the current paradigm of a human first draft that might later be reviewed by an AI. Lawyers who embrace these tools early will have a competitive edge in efficiency.</li><li><strong>Enhanced legal research and reasoning:</strong> The combination of <strong>reasoning models with legal databases</strong> is likely to keep improving. Future AI systems might be able to not only find relevant cases but also analogise and distinguish them, construct legal arguments, and suggest counterarguments — essentially performing a substantial portion of the legal reasoning process. OpenAI’s experimental o1-preview model demonstrated that AI can engage in more complex reasoning when guided properly​ (and its successor, o3, is expected to perform even better). It’s plausible that next-gen models will incorporate a form of legal “chain-of-thought,” where the AI shows its step-by-step legal analysis, making it easier for lawyers to follow and trust the AI’s reasoning.</li><li><strong>Improved reliability and explainability:</strong> One barrier to trusting AI in law has been the reliability of its outputs. The future will bring systems that are <strong>more transparent about their sources and reasoning</strong>. RAG-based AI already cites its sources; going forward, AI assistants might also flag their level of confidence on specific points, or explicitly note when a question falls into a gray area of law. This could help lawyers assess when an AI’s suggestion is solid versus when it needs extra scrutiny. We also anticipate lower hallucination rates as techniques to fact-check the AI’s output are integrated. For example, an AI might automatically cross-verify any case citation against a database to ensure it’s real (some products are building this in). With guardrails like these, the nightmare scenario of fictitious cases slipping into court filings should fade. In short, future AI will be a more <strong>accountable partner</strong>, designed to support the lawyer’s duty of candor and accuracy.</li><li><strong>Shifting professional landscape:</strong> As AI takes on more of the routine drafting and research, the <strong>role of the human lawyer will continue to shift</strong> toward higher-level functions. Skills like empathetic client counseling, courtroom advocacy, negotiation, creative problem-solving, and ethical decision-making will only become more important — these are the areas where humans excel and machines can’t substitute for genuine human engagement. At the same time, completely new roles may emerge in the legal field. We will see more <strong>legal technologists or AI specialists</strong> within law firms (the leaders already have this function), who manage and curate the AI tools, similar to how e-discovery lawyers developed in response to electronic data explosion. There may also be increased collaboration between lawyers and data scientists to refine AI outputs or develop custom AI models for a firm’s unique needs (for instance, training an AI on a firm’s briefs and templates to maintain a consistent style and approach).</li><li><strong>Access to justice and new services:</strong> Looking further ahead, AI has the potential to democratise legal services in profound ways. As AI tools become cheaper and widely available, they could empower non-lawyers to get basic legal help. We are already seeing AI-driven chatbot services that guide individuals in drafting simple contracts, contesting parking tickets, or navigating small claims courts. While these tools currently operate with caution (and sometimes controversy), their capability will grow. It’s conceivable that <strong>AI-driven legal clinics</strong> or apps will emerge that can handle common legal issues for people who can’t afford a lawyer, under lawyer supervision. This could ease the burden on pro bono and legal aid organisations by automating preliminary work and freeing human lawyers to focus on complex cases or court representation. Judges and court systems might also use AI to assist self-represented litigants — for example, generating a draft order or finding relevant precedents for a judge — thereby speeding up proceedings. In the long run, we might find that AI doesn’t so much replace lawyers as it <strong>expands legal access</strong> to those historically underserved, changing the public’s relationship to the law. Of course, along with these positive transformations, there will be challenges to navigate. Regulation will need to catch up — ensuring there are standards for AI in legal practice (much like there are for lawyer competence and ethics). Courts may develop rules on citing AI or even require disclosure if a filing was AI-assisted. And the profession will have to wrestle with tough questions about responsibility: if an AI-assisted decision goes awry, who is accountable? Likely it will remain the lawyer or firm, which reinforces why maintaining human oversight is critical. In the foreseeable future, <strong>AI will not replace lawyers, but lawyers who use AI may well replace those who don’t.</strong> Embracing technology while upholding professional standards will be the balancing act.</li></ul><h3>Conclusion</h3><p>The advent of AI in law is reminiscent of past technological shifts — from law libraries to online databases, from typewriters to word processors — but on a far grander scale. This latest study on <em>AI-powered lawyering</em> underscores that we have entered a new era where AI is not just a gimmick but a genuine asset in legal work. By intelligently leveraging tools like legal-focused retrieval systems and advanced reasoning models, lawyers can deliver results faster and, in many cases, with equal or better quality than before​. The <strong>future of lawyering is poised to be a partnership between human expertise and artificial intelligence</strong>. Such a partnership holds great promise: imagine more time for lawyers to strategise and connect with clients, while AI handles the heavy lifting of research and initial drafting. It also presents an opportunity for the profession to evolve, freeing lawyers from some mundane tasks and challenging them to hone the uniquely human skills that clients and justice truly demand.</p><p>That said, the path forward requires a thoughtful approach. Lawyers must be vigilant against the shortcomings of AI — verifying its outputs, protecting client data, and remembering that technology is a tool, not a substitute for judgment. As the saying goes in tech circles, “with great power comes great responsibility.” The legal community will need to establish norms and training so that every lawyer knows how to use AI responsibly and effectively. Those who do will likely find that AI is an invaluable colleague for the routine and even the complex aspects of practice. Those who resist may find themselves at a competitive disadvantage as clients gravitate toward more efficient service providers.</p><p>In closing, the rise of AI in law is <strong>not the end of lawyers — it’s the beginning of a new kind of lawyering</strong>. It’s an evolution where mastery of AI tools will be as important as mastery of legal precedent. The evidence so far suggests that when man and machine collaborate in the law, the result can be better, faster, and more accessible legal services. The legal profession stands at a crossroads: by embracing innovation and coupling it with the timeless principles of ethics and justice, we can usher in a future where AI amplifies what lawyers do best. The next few years will be critical in shaping this future. One thing is certain: AI is here to stay in the legal world, and the smartest path is to shape it, harness it, and <strong>use it to elevate the practice of law</strong> for the benefit of clients and society.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0773434e7c8a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>