<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anna Kiefer on Medium]]></title>
        <description><![CDATA[Stories by Anna Kiefer on Medium]]></description>
        <link>https://medium.com/@annakiefer12?source=rss-41da5e70b5a1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*c_UjunOgRjvwK63HLSO5qg.jpeg</url>
            <title>Stories by Anna Kiefer on Medium</title>
            <link>https://medium.com/@annakiefer12?source=rss-41da5e70b5a1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 07:19:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@annakiefer12/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Named Person Matching]]></title>
            <link>https://medium.com/@annakiefer12/the-importance-of-diverse-test-data-in-token-based-matching-2ea4eb8450b2?source=rss-41da5e70b5a1------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ea4eb8450b2</guid>
            <category><![CDATA[named-entity-recognition]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[nlp-training]]></category>
            <dc:creator><![CDATA[Anna Kiefer]]></dc:creator>
            <pubDate>Fri, 20 Oct 2023 17:46:09 GMT</pubDate>
            <atom:updated>2023-10-22T16:48:06.090Z</atom:updated>
            <content:encoded><![CDATA[<h3>What’s in a name?</h3><h4>The Importance of Diverse Test Data in Token-Based Matching</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/1*JWnKhMIVT4bLw5eT6UVehg.png" /></figure><p>In the natural language processing context, text matchers are tools that help identify words or phrases in unstructured text. Like a sophisticated regular expression, matchers typically recognize these words or phrases using a set of pre-defined patterns.</p><p>spaCy’s <a href="https://spacy.io/usage/rule-based-matching#matcher">Token-Based Matcher</a> is one popular text matcher. It allows for pattern matching using verbatim text, digits, punctuation, and entity types, among others (see <a href="https://spacy.io/usage/rule-based-matching#matcher">here</a> for a full list of the token attributes that can be used).</p><p>Recently, I used spaCy’s Token-Based Matcher to extract speaker names in legislative proceedings. This was part of my work on the Digital Democracy project, a collaboration between the media and news organization CalMatters and California Polytechnic State University.</p><p>One component of the Digital Democracy project is known as the Transcription Tool. The Transcription Tool is responsible for obtaining accurate, timely transcriptions of California state legislative proceedings.</p><p>During Committee Hearings, lobbyists, regulators, and members of the general public will often introduce themselves before speaking. Automatically extracting the full names and affiliations of speakers using a token-based matcher may help speed up the transcription editing, or up-leveling, process (at least that’s the hope).</p><p>For the speaker self-identification name matching, I included this list of 10 patterns as a first pass into my matcher:</p><pre>from spacy.matcher import Matcher<br>import en_core_web_lg<br><br><br>nlp = en_core_web_lg.load()<br><br><br>matcher = Matcher(nlp.vocab)<br>patterns = [<br>    [{&quot;LOWER&quot;: &quot;my&quot;}, {&quot;LOWER&quot;: &quot;name&quot;}, {&quot;LOWER&quot;: &quot;is&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>    [{&quot;LOWER&quot;: &quot;i&quot;}, {&quot;LOWER&quot;: &quot;am&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>    [{&quot;LOWER&quot;: &quot;i&quot;}, {&quot;LOWER&quot;: &quot;&#39;m&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>    [{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;on&quot;}, {&quot;LOWER&quot;: &quot;behalf&quot;}, {&quot;LOWER&quot;: &quot;of&quot;}],<br>    [{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;IS_PUNCT&quot;: True}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>    [{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;with&quot;}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>    [{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;from&quot;}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>    [{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>    [{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;FAC&quot;}],<br>    [{&quot;LOWER&quot;: &quot;i&quot;}, {&quot;LOWER&quot;: &quot;am&quot;}, {&quot;IS_ALPHA&quot;: True}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>    [{&quot;LOWER&quot;: &quot;i&quot;}, {&quot;LOWER&quot;: &quot;am&quot;}, {&quot;IS_ALPHA&quot;: True}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>]<br><br>matcher.add(match_name, patterns)</pre><p>Then, I ran it on a test dataset of 10,000 distinct instances of speaker introductions from California state legislative hearings from a six year period starting in 2016. All of the utterances contained the full name of the speaker. These utterances had first been automatically transcribed using a speech-to-text service, then hand corrected for accuracy.</p><p>I ran my first-pass matcher on the utterances, using spaCy’s <a href="https://spacy.io/models/en#en_core_web_lg">en_core_web_lg</a>, its large English pipeline optimized for CPU, for the actual tokenization and Named Entity Recognition. Here is a histogram of the number of matches for each pattern.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/444/1*IAeL7fcREP9p0BfyEhjUIw.png" /><figcaption>Speaker matches using spaCy’s Matcher for 10K rows</figcaption></figure><p>And here are the patterns by pattern index:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5dmjWpbP5_qpPV8J6oxzmg.jpeg" /></figure><p>Out of 10,000 utterances, 6,724 rows contained at least one match, or roughly 67%. Not awful, but not great.</p><p>I took a look at several dozen of the missing rows to determine where I might be going wrong.</p><p>For starters, I had grossly underestimated the number of ways a speaker might choose to introduce his/her/themselves. It turns out “My name is &lt;NAME&gt;” only occurs 242 times in the dataset.</p><p>Two, there were often small grammatical differences in the transcript that I failed to account for, such as “My name’s &lt;NAME&gt;” or “&lt;NAME&gt; here”.</p><p>Third and perhaps most importantly, I had assumed that, like my own name, a speaker’s name would consist of a single token first name and a single token last name.</p><p>Looking at the missing rows revealed that the matcher was missing a lot of names that didn’t conform to my initial preconceived notion of a name. I had coded for mono-token first and last names, conforming to my own Western/Anglophone name. Therefore, names containing more than two tokens, like Jose de Santos or even Mary Rose Shelton, were being omitted entirely.</p><p>After this discovery, I added over 20 additional patterns to improve my matcher:</p><pre>[{&quot;LOWER&quot;: &quot;i&quot;}, {&quot;LOWER&quot;: &quot;am&quot;}, {&quot;IS_ALPHA&quot;: True}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;LOWER&quot;: &quot;i&quot;}, {&quot;LOWER&quot;: &quot;am&quot;}, {&quot;IS_ALPHA&quot;: True}, {&quot;IS_ALPHA&quot;: True}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;LOWER&quot;: &quot;my&quot;}, {&quot;LOWER&quot;: &quot;name&quot;}, {&quot;LOWER&quot;: &quot;is&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;LOWER&quot;: &quot;my&quot;}, {&quot;LOWER&quot;: &quot;name&quot;}, {&quot;LOWER&quot;: &quot;&#39;s&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;LOWER&quot;: &quot;i&quot;}, {&quot;LOWER&quot;: &quot;am&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;LOWER&quot;: &quot;this&quot;}, {&quot;LOWER&quot;: &quot;is&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;LOWER&quot;: &quot;this&quot;}, {&quot;LOWER&quot;: &quot;is&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;IS_PUNCT&quot;: True}, {&quot;LOWER&quot;: &quot;with&quot;}, {&quot;LOWER&quot;: &quot;the&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;IS_PUNCT&quot;: True}, {&quot;LOWER&quot;: &quot;with&quot;}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;for&quot;}, {&quot;LOWER&quot;: &quot;the&quot;}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;here&quot;}, {&quot;LOWER&quot;: &quot;on&quot;}, {&quot;LOWER&quot;: &quot;behalf&quot;}, {&quot;LOWER&quot;: &quot;of&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;here&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;of&quot;}, {&quot;LOWER&quot;: &quot;the&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;IS_ALPHA&quot;: True}, {&quot;LOWER&quot;: &quot;representing&quot;}],<br>[{&quot;TEXT&quot;: &quot;I&quot;}, {&quot;LOWER&quot;: &quot;&#39;m&quot;}, {&quot;IS_ALPHA&quot;: True}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;IS_PUNCT&quot;: True}, {&quot;IS_ALPHA&quot;: True}, {&quot;LOWER&quot;: &quot;representing&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;IS_PUNCT&quot;: True}, {&quot;IS_ALPHA&quot;: True}, {&quot;IS_ALPHA&quot;: True}, {&quot;LOWER&quot;: &quot;representing&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;IS_PUNCT&quot;: True}, {&quot;LOWER&quot;: &quot;the&quot;}, {&quot;IS_ALPHA&quot;: True}, {&quot;LOWER&quot;: &quot;of&quot;}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;representing&quot;}, {&quot;LOWER&quot;: &quot;the&quot;}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>[{&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;LOWER&quot;: &quot;representing&quot;}, {&quot;ENT_TYPE&quot;: &quot;ORG&quot;}],<br>[{&quot;LOWER&quot;: &quot;assemblymember&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],<br>[{&quot;LOWER&quot;: &quot;assembly&quot;}, {&quot;LOWER&quot;: &quot;member&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}, {&quot;ENT_TYPE&quot;: &quot;PERSON&quot;}],</pre><p>Note that the last two patterns are context specific. It is common for legislators to introduce themselves as “Assembly Member” (or “Assemblymember”).</p><p>With these added patterns, my new matcher was able to match 7,796 of the 10K utterances, or 78%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/1*xMKdx0qLdmrUbA_H4Ql0xQ.png" /></figure><p>There are still many improvements to be done, like adding more multi-name and title patterns (Mr., Ms., Dr., etc), but it’s a definite improvement.</p><p>I’m glad I caught this blunder early. If my dataset had only contained single-token names, my matcher performance might have been satisfactory, but would have missed many speakers when used in the real world. I had even considered creating synthetic data using a list of common first and last names with which to use to test my matcher, which would have also omitted many names.</p><p>Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ea4eb8450b2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>