<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Omri Mendels on Medium]]></title>
        <description><![CDATA[Stories by Omri Mendels on Medium]]></description>
        <link>https://medium.com/@omri-mendels?source=rss-2b84200044b9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*Fe3kyUISTk-n9c6482iyHg.jpeg</url>
            <title>Stories by Omri Mendels on Medium</title>
            <link>https://medium.com/@omri-mendels?source=rss-2b84200044b9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:48:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@omri-mendels/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Evaluating PII Detection Models]]></title>
            <link>https://omri-mendels.medium.com/evaluating-pii-detection-models-fa0c745d7a4c?source=rss-2b84200044b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa0c745d7a4c</guid>
            <category><![CDATA[pii]]></category>
            <category><![CDATA[de-identification]]></category>
            <category><![CDATA[privacy]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-privacy]]></category>
            <dc:creator><![CDATA[Omri Mendels]]></dc:creator>
            <pubDate>Fri, 26 Jun 2026 17:06:00 GMT</pubDate>
            <atom:updated>2026-06-26T17:06:00.901Z</atom:updated>
            <content:encoded><![CDATA[<h3>Evaluating PII Detection Models: A Field Guide</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cXV2heZns3Zcormunavvhw.jpeg" /></figure><h2>A Tale of Two Numbers</h2><p>Suppose you’re choosing a PII detection model for your organization or platform. You run two candidates (call them Model A and Model B) on the same annotated test set. Model A reports an accuracy (e.g., F2, which penalizes false negatives more than false positives) of 0.91. Model B reports 0.78. You pick Model A.</p><p>Then you deploy it, and privacy incidents keep happening.</p><p>What went wrong? Possibly nothing with the model itself. The numbers were real; they just weren’t measuring what you thought they were measuring.</p><h3>PII evaluation is not one question</h3><p>When evaluating a model for general Named Entity Recognition (NER), the task of locating and classifying named things in text, you’re typically asking a single question: <em>how well does this model detect entity type X?</em> PII detection is different because it serves several distinct purposes, each of which requires a different answer:</p><ul><li><strong>Did the model identify all the private data?</strong> This is the redaction question. You care about false negatives above everything else: a missed name is a privacy leak regardless of what the model called it. In Machine Learning terms, this is a binary classification problem.</li><li><strong>Did the model correctly identify what kind of PII this is</strong>? This is the auditing question. When gathering statistics on what PII exists in a corpus, you need to know whether a detected entity is a name, a location, or a government ID. In ML terms, this is a multi-class classification problem.</li><li><strong>How specifically does the model classify PII types?</strong> This is the synthetic substitution question. Replacing a person’s initials with a fake full name produces unusable synthetic data. If you want to replace only the year in a date of birth but the model detects a generic DATE_TIME, you lose the ability to make that substitution precisely. In other words, this is a hierarchical classification problem.</li></ul><p>These three objectives map directly to three levels of evaluation granularity:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7eo0M5JyCWABcNkyqEn_PA.png" /></figure><p>A single aggregate F2 score conflates all three. A model can have excellent binary recall (finds almost all PII) while being mediocre at category-level classification, which is fine for a redaction use case but unacceptable for auditing or synthesis. Reporting only one number forces you to pick which objective you’re optimizing for implicitly, and makes it impossible to compare models across objectives.</p><p>This is the first structural challenge in PII evaluation: <strong>what are you actually measuring?</strong></p><p>Three further problems compound to make the answer harder than it looks:</p><ol><li><strong>Models don’t share a label vocabulary.</strong> Two models detecting the same “thing” will call it different names, and naive string comparison treats them as detecting different things.</li><li><strong>Token-level scoring has structural biases.</strong> While being the simplest approach for evaluation, in token-level scoring, the <em>format</em> of a PII value can weight some misses more heavily than others, independent of whether the model actually detected the entity.</li><li><strong>Evaluation datasets are rarely representative.</strong> Real PII datasets are almost never available due to privacy constraints, so practitioners fall back on open-source benchmarks (which models are often trained on) or LLM-generated synthetic data (which is unnaturally clean), both of which make models look better than they perform in production.</li></ol><p>This post walks through each problem in detail, using concrete examples, and describes the evaluation framework in <a href="https://github.com/microsoft/presidio-research">presidio-research</a> that addresses them.</p><h3>The Label Vocabulary Problem</h3><h4>Why labels don’t line up</h4><p>Every model and every dataset brings its own entity taxonomy. Your annotated dataset might label personal names as PERSON. One model calls it PER. Another uses BIO tags: B-PERSON, I-PERSON. A third, trained on clinical data, emits PATIENT_NAME. A compliance-focused model outputs FULL_NAME and FIRST_NAME as separate types. A regulation-specific model appends country codes: GERMANY_PASSPORT_NUMBER, US_DRIVER_LICENSE, AU_MEDICARE.</p><p>If you compare these labels directly, every correct detection looks like a mismatch. A model that perfectly redacts every name in your corpus scores zero recall, because its label is PER and your dataset says PERSON.</p><p>The standard workaround is a hand-crafted alias file: a flat dictionary mapping each model’s labels to your dataset’s labels. This works at small scale. It breaks down as soon as you add a second or third model, because every new model means manually auditing dozens of new label names and adding them to the dictionary. The dictionary becomes a maintenance liability, its mappings are invisible to anyone reviewing evaluation results, and there’s no guarantee different practitioners map the same label the same way.</p><h4>The canonical entity hierarchy</h4><p>A better approach is a shared, model-agnostic vocabulary that both models and datasets map <em>to</em>, rather than mapping to each other. This is what the CanonicalMapper in presidio-research implements.</p><p>The canonical vocabulary is organized as a three-level hierarchy:</p><pre>Root<br>├── PERSON          → NAME, USERNAME, ALIAS<br>├── CONTACT         → EMAIL_ADDRESS, PHONE_NUMBER, FAX, SOCIAL_HANDLE<br>├── LOCATION        → STREET_ADDRESS, CITY, COUNTRY, POSTAL_CODE, GEO_COORDINATES<br>├── ORGANIZATION    → COMPANY, EDUCATIONAL_INSTITUTION, GOVERNMENT_AGENCY<br>├── GOVERNMENT_ID   → SSN, PASSPORT, DRIVER_LICENSE, TAX_ID, NATIONAL_ID<br>├── FINANCIAL_PII   → CREDIT_CARD, IBAN, BANK_ACCOUNT, CRYPTO_WALLET<br>├── PHI             → PATIENT_ID, MRN, HEALTH_INSURANCE, DIAGNOSIS, MEDICATION<br>├── DATE_TIME       → DATE, TIME, DURATION<br>└── ...             (DEMOGRAPHIC, EMPLOYMENT, DEVICE_IDENTIFIER, BIOMETRIC, ...)</pre><p>Every label (from any model, from any dataset) gets resolved to a canonical entity through five tiers applied in order:</p><ol><li><strong>Exact match</strong> in the alias map (EMAIL → EMAIL_ADDRESS, PER → NAME)</li><li><strong>BIO tag stripping</strong> (B-PERSON, I-PERSON → NAME)</li><li><strong>Country prefix</strong> (GERMANY_PASSPORT_NUMBER → PASSPORT, US_DRIVER_LICENSE → DRIVER_LICENSE)</li><li><strong>Country prefix fallback</strong> (IN_PAN → strips IN, resolves PAN → TAX_ID)</li><li><strong>Fuzzy string match</strong> at ≥ 0.80 similarity for near-misses like CREDITCARD → FINANCIAL</li></ol><p>If none of these tiers resolve the label, it’s flagged as UNRESOLVED, a hard error that blocks evaluation until the user manually maps it. This is intentional: silent mismatches are worse than loud failures.</p><h4>The three-level evaluation surface</h4><p>The hierarchy serves a second purpose: it defines three natural levels at which to measure performance.</p><ul><li><strong>Binary:</strong> did the model detect any PII at all?</li><li><strong>Category:</strong> did the model detect the right <em>category</em> of PII? (PERSON vs. LOCATION vs. GOVERNMENT_ID)</li><li><strong>Detailed:</strong> did the model detect the right specific entity type? (NAME vs. PASSPORT vs. EMAIL_ADDRESS)</li></ul><p>The evaluation depth is inferred automatically from your dataset. If your annotations use fine-grained labels like EMAIL_ADDRESS and SSN, the mapper computes a canonical surface at depth 3 (detailed). If your annotations use coarse labels like PERSON and LOCATION, it computes depth 2 (category). The canonical surface reflects what your data can actually distinguish, with no manual tuning required, and two different users comparing the same model against their respective datasets get consistent, meaningful scores at the right granularity.</p><p>A single evaluation run produces all three levels simultaneously:</p><ul><li><strong>Binary</strong>: Did the model catch the PII and nothing else?</li><li><strong>Category</strong>: Did it know what <em>kind</em> of PII?</li><li><strong>Detailed</strong>: Is the entity type <em>specific</em> enough?</li></ul><p>A model can have high binary recall (finds most PII) but low detailed recall (mislabels types frequently). These tell you different things about deployment risk.</p><h4>When mapping requires human judgment</h4><p>Canonical mapping resolves most label mismatches automatically, but some cases are genuinely ambiguous and require a decision. Two examples:</p><p>A model that outputs NRP (nationality/religion/political affiliation) might map to DEMOGRAPHIC, while the dataset annotated the same tokens as LOCATION. Both are defensible; the mapper flags the conflict and asks you to pick one. Silently choosing either would distort the evaluation in ways that are hard to trace.</p><p>A subtler problem runs in the opposite direction: a model may correctly detect entity types that your dataset simply never annotated. Every such prediction looks like a false positive, silently deflating precision, even though the model is doing exactly what it was designed to do. If your test set has no ORGANIZATION annotations but the model flags company names throughout, those detections count against it. The mapper surfaces this so you can decide whether to add the missing annotations, suppress that entity type from evaluation, or explicitly acknowledge it as out of scope.</p><h2>The Boundary Problem</h2><h4>What token-level evaluation actually measures</h4><p>Token-level evaluation is the default in NER benchmarks: split text into tokens, assign each a label, compare predicted label vs. gold label token by token, aggregate into precision/recall/F2. Simple, transparent, and widely implemented.</p><p>The problem isn’t boundary sensitivity per se; it’s that token-level scoring has two structural biases that quietly distort comparisons.</p><p><strong>Multi-token entities are penalized proportionally more than single-token ones.</strong></p><p>Consider the same phone number in two formats:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N-RSKvjX-QGzCwhLgTK8qQ.png" /></figure><p>Both represent identical PII, and the model missed both entirely. But the hyphenated version contributes five times as many false negatives to the confusion matrix. The formatting of the value, something the model has no control over and the annotator may not have standardized, determines how heavily a miss is counted.</p><p><strong>Skip words inside entity spans generate phantom errors.</strong></p><p>Many multi-token entities contain function words that some taggers annotate as non-entities. “University of Washington” is a canonical example: the gold annotation might be [ORG, O, ORG] if the annotator tagged &quot;of&quot; as non-PII, while the model correctly outputs [ORG, ORG, ORG] treating it as a single organization span. Token-level evaluation sees one false positive (the model&#39;s &quot;of&quot; prediction) even though the model got the entity right.</p><p>The reverse happens too: a model trained to emit clean spans will output [ORG, O, ORG], fragmenting &quot;University of Washington&quot; into two separate organization predictions. Token-level evaluation sees two true positives but the redaction pipeline produces two separate redacted spans with a gap, which may or may not be the intended behavior, depending on whether &quot;of&quot; carries identifying information in context.</p><p>These aren’t edge cases. They occur wherever PII entities contain prepositions, articles, or punctuation that sits ambiguously between “part of the entity” and “surrounding text”: “Bank of America”, “Dr. Jane Doe”, “42 Main St, Apt. 3B”.</p><h4>IoU-based span evaluation</h4><p>Presidio Evaluator’s SpanEvaluator operates at the entity level rather than the token level. It treats each entity as a single span with character boundaries, and uses Intersection over Union (IoU) to determine whether a predicted span and an annotated span match:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*4S6srBrN-7w1PpO3rGPueQ.png" /><figcaption>Intersection Over Union. A — Annotation, P — Prediction</figcaption></figure><p>A match is declared when IoU≥θ (default <em>θ</em>=0.75), configurable per use case. This resolves both structural problems above:</p><ul><li><strong>Multi-token penalty is gone.</strong> Whether the entity is one token or ten, it counts as one annotated span and one predicted span. Missing 555-121-5553 contributes exactly one FN, same as missing 5551215553.</li><li><strong>Skip word artifacts are eliminated explicitly.</strong> The SpanEvaluator accepts a configurable skip_words list. Words on this list (prepositions, articles, punctuation marks) are ignored when merging adjacent same-type tokens into a single span before IoU is computed. &quot;University of Washington&quot; with of as a skip word collapses into one span regardless of how the model or annotator tagged &quot;of&quot; individually.</li></ul><p>The IoU threshold also provides proportional credit for near-misses. A model that detects only “John” from “John Michael Smith” scores an IoU of 0.33, below the default threshold, counted as a miss. A model that detects “Dr. John Smith” when the annotation is “John Smith” scores an IoU of ~0.67, a near-miss rather than a clean hit.</p><p>The evaluator also handles the common case where a model emits multiple overlapping predictions for the same annotated span. If a model predicts [ORG, O, ORG] for &quot;University of Washington&quot;, the two ORG spans are combined before IoU is computed against the gold annotation. Multiple predictions of the same type over the same region count as one prediction, not two TPs, which would overcount correct detections.</p><p>Both mechanisms, label mapping and span merging, are often needed together. Consider a clinical record:</p><pre>Input:      Patient name: Kenobi, Obi-Wan<br>Model:      Patient name: [LAST_NAME], [FIRST_NAME]<br>Dataset:    Patient name: [PERSON]</pre><p>Label mapping resolves LAST_NAME and FIRST_NAME to the same canonical entity as PERSON. The model now has two predicted spans where the dataset has one. The comma between them is ignored, so the evaluator merges the two predictions into a single span before computing IoU. The merged span aligns with the annotated [PERSON] span, and the model receives full credit, which is the correct outcome, since it identified both components of the name precisely.</p><h2>The Dataset Problem</h2><p>The dataset problem has three layers that compound each other.</p><h4>Real PII data is effectively unreachable</h4><p>The most accurate evaluation data would be real documents containing real PII, annotated by domain experts. While this data exists inside healthcare systems, financial institutions, HR platforms, and the customer service queues of large organizations, it is almost never accessible for evaluation purposes.</p><p>Even teams building de-identification tools <em>within</em> those organizations frequently can’t use production data for benchmarking. Legal constraints, privacy regulations, and data governance policies mean that the data most relevant to the problem is the data you’re least able to touch. The result is that virtually all published PII evaluation is done on proxy datasets that differ from the deployment context in ways that matter.</p><h4>Open-source datasets are often in the training set</h4><p>The obvious fallback is open-source annotated corpora. The problem is that the most widely used ones have already been incorporated into the training pipelines of the models you’re trying to evaluate.</p><p>The most widely used corpora span real annotated data — CoNLL 2003 (Reuters newswire) and i2b2 (clinical notes) — as well as explicitly synthetic datasets like ai4privacy/pii-masking-300k on Hugging Face and the training data for Nvidia&#39;s Nemotron models. In each case, these datasets have been incorporated into popular fine-tuning pipelines, meaning models evaluated on samples from those distributions have effectively seen the test data.</p><p>Evaluations of these models using the training datasets reflect memorization of a distribution as much as genuine detection capability. When you deploy such a model on customer service chat logs or multilingual intake forms, it encounters text that looks nothing like the training data, and performance drops sharply.</p><h4>LLM-generated datasets are unnaturally clean</h4><p>Generating fresh evaluation data with GPT-5 or Claude avoids the contamination problem. But it introduces a different one: <strong>LLM-generated text is too clean</strong>.</p><p>Real PII-bearing text is noisy in ways that matter for detection:</p><ul><li><strong>Casing errors</strong>: “john doe” and “JOHN DOE” are common in real transcripts; LLMs default to standard casing.</li><li><strong>Typos and OCR artifacts</strong>: “Jahn Smth”, “<a href="/service/mailto:johnn@examp1e.com">johnn@examp1e.com</a>”. LLMs rarely produce these.</li><li><strong>Speech-to-text artifacts</strong>: “my name is john smith my phone number is five five five”, with no punctuation, numbers spelled out, entity boundaries unclear.</li><li><strong>Multi-turn fragmentation</strong>: PII is often spread across multiple turns in a conversation, with no single span containing the full value. A credit card number dictated digit-by-digit while an operator reads it back looks nothing like a contiguous 16-digit string:</li></ul><pre>Operator: What&#39;s your credit card number?<br>Customer: 1664<br>Operator: Yes<br>Customer: 5122<br>Operator: Aha<br>Customer: 3371<br>…</pre><p>The practical consequence is that models evaluated on LLM-generated data appear to perform well because the test text matches the clean, well-formed input those models were trained on. The cleanliness gap only becomes visible after deployment.</p><p>None of this means synthetic data is useless. It’s valuable for controlled experiments and for covering rare entity types. The mistake is treating it as a proxy for real-world performance without accounting for what it doesn’t capture.</p><h3>What good evaluation data looks like</h3><p>Good PII evaluation data has a short checklist:</p><ol><li><strong>Held-out from training.</strong> Models being evaluated should have no access, direct or indirect, to the test data.</li><li><strong>Realistic noise distribution.</strong> Typos, casing inconsistencies, punctuation variation, and speech artifacts should appear at rates consistent with the deployment domain.</li><li><strong>Representative PII coverage.</strong> The entity types in the test set should match the types the model will encounter in production, not just the types that are easy to annotate.</li><li><strong>Clear annotation guidelines.</strong> Annotators should agree on boundary rules, especially for titles, nested entities, and implied PII.</li><li><strong>Multi-domain coverage.</strong> A single-domain test set (news, clinical, social media) will not generalize.</li></ol><p>The <a href="https://github.com/microsoft/presidio-research">presidio-research</a> data generator produces synthetic datasets from templates, allowing controlled noise injection: typos, casing shifts, date format variation. Combining templates generated by LLMs and manually created ones can bring more realistic results. Moreover, using tools like Faker and real PII dictionaries often creates better distributions than asking the LLM to come up with a value. It’s a partial solution to the cleanliness problem, not a complete one.</p><h2>Putting It Together</h2><p>The three problems described above (label mismatch, token-level scoring bias, and dataset contamination) are independent but compound. A clean evaluation pipeline needs to address all three, in sequence:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dDa4uycPGlPs7YKdNTkBbQ.png" /></figure><p>The pipeline makes the comparison claim explicit: two models evaluated through this pipeline on the same dataset are measuring the same thing, using the same canonical vocabulary, the same boundary tolerance, and the same hierarchical decomposition of performance.</p><p>In practice, this means:</p><ul><li>Model A calls names PER, Model B calls them FULL_NAME, your dataset says PERSON. All three resolve to NAME in the canonical hierarchy. The comparison is fair.</li><li>A phone number written as 555-121-5553 and one written as 5551215553 both contribute one annotated span to the denominator, regardless of how many tokens the tokenizer produces.</li><li>At the end, you get three F2 numbers per model: binary (found any PII?), category (right category?), detailed (right type?). These tell a richer story than a single aggregate score.</li></ul><p>The motivating scenario from the introduction (Model A reporting 0.91 and Model B 0.78 on the same dataset) often inverts when you run through this pipeline. Model A may have been benefiting from label leakage (its label vocabulary matched the dataset’s exactly, while Model B’s didn’t), or from token-level scoring that rewarded its tokenizer’s coincidental alignment with the test corpus, or both. Canonical mapping and span evaluation remove both of those advantages and leave the comparison grounded in what actually matters:</p><p>Does the model find PII, with the right type, within a reasonable boundary tolerance? <strong>That’s the question the evaluation pipeline is designed to answer honestly.</strong></p><p>The evaluation framework described here is part of <a href="https://github.com/microsoft/presidio-research">microsoft/presidio-research</a>.</p><p><strong>This work was done jointly with the amazing Presidio team: </strong>Sharon Hart, Coby Peled, Nava Vaisman Levy, Noa Gruber Uziely, Ron Shakutai, and many other contributors.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa0c745d7a4c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is it with NLP models and biblical names?]]></title>
            <link>https://medium.com/data-science/what-is-it-with-nlp-models-and-biblical-names-c389cf7a24c9?source=rss-2b84200044b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/c389cf7a24c9</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-privacy]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[named-entity-recognition]]></category>
            <dc:creator><![CDATA[Omri Mendels]]></dc:creator>
            <pubDate>Sat, 08 May 2021 18:27:35 GMT</pubDate>
            <atom:updated>2021-10-16T11:35:04.441Z</atom:updated>
            <content:encoded><![CDATA[<h4>A comparison of different NER models and their ability to detect different named entities.</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F3ohc10GA6j4XrLWzZK%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F3ohc10GA6j4XrLWzZK%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2F3ohc10GA6j4XrLWzZK%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/6767b50bbfd0ffa86789393af474c55f/href">https://medium.com/media/6767b50bbfd0ffa86789393af474c55f/href</a></iframe><p>This blog post highlights an issue with spaCy and other Named Entity Recognition models not being able to accurately detect person names, especially if they are biblical names. The detection differences between regular names and biblical names are quite overwhelming.</p><p>You’ll see that for the simplest example we can think of, “My name is X”, biblical names are almost never detected as names of persons.</p><p>I tried to get to the bottom of this and believe I have an answer. But first, let&#39;s do a short experiment with two spaCy models (using spaCy version 3.0.5).</p><h4>Compare detection rates of biblical vs. other names</h4><p>Why is there a difference in the first place? The reason for the different detection rates could arise from:</p><ol><li>The fact that biblical names are sometimes older and less common (therefore might be less frequent in the dataset the model was trained on).</li><li>That the surrounding sentence is less likely to co-occur with the specific name on the original dataset.</li><li>Issue with the dataset itself (such as wrong annotations by humans).</li></ol><p>To (simplistically) test these hypotheses, we compare biblical names with both old and new names, and three templates, two of which are from the bible:</p><ul><li>“My name is X”</li><li>“And X said, Why hast thou troubled us?”</li><li>“And she conceived again, a bare a son; and she called his name X.”</li></ul><p>Let’s start by creating name lists and templates:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5018f44eb98a7778d972516c54e437c7/href">https://medium.com/media/5018f44eb98a7778d972516c54e437c7/href</a></iframe><p>Method for running the spaCy model and checking if “PERSON” was detected:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3bdf567637bb932a629e417d77410fba/href">https://medium.com/media/3bdf567637bb932a629e417d77410fba/href</a></iframe><h4>Model 1: spaCy’s en_core_web_lg model</h4><ul><li>This model uses the original (non-transformers based) spaCy architecture.</li><li>It was trained on the <a href="https://catalog.ldc.upenn.edu/LDC2013T19">OntoNotes 5.0</a> dataset and features <a href="https://spacy.io/models/en#en_core_web_lg">0.86 F-measure on named entities</a>.</li></ul><p>Loading the model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/65e5610770091b2836d5caff47c78f8a/href">https://medium.com/media/65e5610770091b2836d5caff47c78f8a/href</a></iframe><p>Let’s run it:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/29456aeef47d19328285de21541aad0c/href">https://medium.com/media/29456aeef47d19328285de21541aad0c/href</a></iframe><p>Here are the results we get:</p><pre>Model name: en_core_web_lg<br>Name set: Biblical, Template: &quot;My name is {}&quot;<br>Recall: 0.25</pre><pre>Name set: Other, Template: &quot;My name is {}&quot;<br>Recall: 0.94</pre><pre>Name set: Biblical, Template: &quot;And {} said, Why hast thou troubled us?&quot;<br>Recall: 0.67</pre><pre>Name set: Other, Template: &quot;And {} said, Why hast thou troubled us?&quot;<br>Recall: 0.94</pre><pre>Name set: Biblical, Template: &quot;And she conceived again, a bare a son; and she called his name {}.&quot;<br>Recall: 0.58</pre><pre>Name set: Other, Template: &quot;And she conceived again, a bare a son; and she called his name {}.&quot;<br>Recall: 0.94</pre><pre>Detailed results:<br>{(&#39;And she conceived again, a bare a son; and she called his name {}.&#39;, &#39;Biblical&#39;): {<br>&#39;Abraham&#39;: True,<br>&#39;David&#39;: True,<br>&#39;Isaac&#39;: False,<br>&#39;Jacob&#39;: False,<br>&#39;Jesus&#39;: False,<br>&#39;John&#39;: True,<br>&#39;Judas&#39;: False,<br>&#39;Mary&#39;: True,<br>&#39;Matthew&#39;: True,<br>&#39;Moses&#39;: False,<br>&#39;Samuel&#39;: True,<br>&#39;Simon&#39;: True},</pre><pre>(&#39;And she conceived again, a bare a son; and she called his name {}.&#39;, &#39;Other&#39;): {<br>&#39;Ariana&#39;: True,<br>&#39;Barack&#39;: True,<br>&#39;Beyonce&#39;: True,<br>&#39;Bill&#39;: True,<br>&#39;Charles&#39;: True,<br>&#39;Coby&#39;: False,<br>&#39;Donald&#39;: True,<br>&#39;Frank&#39;: True,<br>&#39;George&#39;: True,<br>&#39;Helen&#39;: True,<br>&#39;Joe&#39;: True,<br>&#39;Katy&#39;: True,<br>&#39;Lebron&#39;: True,<br>&#39;Margaret&#39;: True,<br>&#39;Robert&#39;: True,<br>&#39;Ronald&#39;: True,<br>&#39;William&#39;: True},</pre><pre>(&#39;And {} said, Why hast thou troubled us?&#39;, &#39;Biblical&#39;): {<br>&#39;Abraham&#39;: True,<br>&#39;David&#39;: True,<br>&#39;Isaac&#39;: True,<br>&#39;Jacob&#39;: False,<br>&#39;Jesus&#39;: False,<br>&#39;John&#39;: True,<br>&#39;Judas&#39;: False,<br>&#39;Mary&#39;: True,<br>&#39;Matthew&#39;: True,<br>&#39;Moses&#39;: False,<br>&#39;Samuel&#39;: True,<br>&#39;Simon&#39;: True},</pre><pre>(&#39;And {} said, Why hast thou troubled us?&#39;, &#39;Other&#39;): {<br>&#39;Ariana&#39;: True,<br>&#39;Barack&#39;: True,<br>&#39;Beyonce&#39;: True,<br>&#39;Bill&#39;: True,<br>&#39;Charles&#39;: True,<br>&#39;Coby&#39;: False,<br>&#39;Donald&#39;: True,<br>&#39;Frank&#39;: True,<br>&#39;George&#39;: True,<br>&#39;Helen&#39;: True,<br>&#39;Joe&#39;: True,<br>&#39;Katy&#39;: True,<br>&#39;Lebron&#39;: True,<br>&#39;Margaret&#39;: True,<br>&#39;Robert&#39;: True,<br>&#39;Ronald&#39;: True,<br>&#39;William&#39;: True},</pre><pre>(&#39;My name is {}&#39;, &#39;Biblical&#39;): {<br>&#39;Abraham&#39;: True,<br>&#39;David&#39;: False,<br>&#39;Isaac&#39;: False,<br>&#39;Jacob&#39;: False,<br>&#39;Jesus&#39;: False,<br>&#39;John&#39;: False,<br>&#39;Judas&#39;: False,<br>&#39;Mary&#39;: True,<br>&#39;Matthew&#39;: True,<br>&#39;Moses&#39;: False,<br>&#39;Samuel&#39;: False,<br>&#39;Simon&#39;: False},</pre><pre>(&#39;My name is {}&#39;, &#39;Other&#39;): {<br>&#39;Ariana&#39;: True,<br>&#39;Barack&#39;: True,<br>&#39;Beyonce&#39;: True,<br>&#39;Bill&#39;: True,<br>&#39;Charles&#39;: True,<br>&#39;Coby&#39;: False,<br>&#39;Donald&#39;: True,<br>&#39;Frank&#39;: True,<br>&#39;George&#39;: True,<br>&#39;Helen&#39;: True,<br>&#39;Joe&#39;: True,<br>&#39;Katy&#39;: True,<br>&#39;Lebron&#39;: True,<br>&#39;Margaret&#39;: True,<br>&#39;Robert&#39;: True,<br>&#39;Ronald&#39;: True,<br>&#39;William&#39;: True}}</pre><p>So there’s a pretty big difference between biblical names detection and other names.</p><h4>Model 2: spaCy’s en_core_web_trf model</h4><p>spaCy recently released a new model, en_core_web_trf, based on the huggingface transformers library, and also trained on OntoNotes 5.</p><p>Let’s try this model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/19beb15433818247a82106264abf4d44/href">https://medium.com/media/19beb15433818247a82106264abf4d44/href</a></iframe><p>This time we get:</p><pre>Model name: en_core_web_trf<br>Name set: Biblical, Template: &quot;My name is {}&quot;<br>Recall: 0.50</pre><pre>Name set: Other, Template: &quot;My name is {}&quot;<br>Recall: 1.00</pre><pre>Name set: Biblical, Template: &quot;And {} said, Why hast thou troubled us?&quot;<br>Recall: 0.00</pre><pre>Name set: Other, Template: &quot;And {} said, Why hast thou troubled us?&quot;<br>Recall: 0.11</pre><pre>Name set: Biblical, Template: &quot;And she conceived again, a bare a son; and she called his name {}.&quot;<br>Recall: 0.00</pre><pre>Name set: Other, Template: &quot;And she conceived again, a bare a son; and she called his name {}.&quot;<br>Recall: 0.50</pre><pre>Detailed results:<br>{(&#39;And she conceived again, a bare a son; and she called his name {}.&#39;, &#39;Biblical&#39;): {<br>&#39;Abraham&#39;: False,<br>&#39;David&#39;: False,<br>&#39;Isaac&#39;: False,<br>&#39;Jacob&#39;: False,<br>&#39;Jesus&#39;: False,<br>&#39;John&#39;: False,<br>&#39;Judas&#39;: False,<br>&#39;Mary&#39;: False,<br>&#39;Matthew&#39;: False,<br>&#39;Moses&#39;: False,<br>&#39;Samuel&#39;: False,<br>&#39;Simon&#39;: False},</pre><pre>(&#39;And she conceived again, a bare a son; and she called his name {}.&#39;, &#39;Other&#39;): {<br>&#39;Ariana&#39;: True,<br>&#39;Barack&#39;: True,<br>&#39;Beyonce&#39;: True,<br>&#39;Bill&#39;: False,<br>&#39;Charles&#39;: False,<br>&#39;Coby&#39;: False,<br>&#39;Donald&#39;: True,<br>&#39;Frank&#39;: True,<br>&#39;George&#39;: False,<br>&#39;Helen&#39;: False,<br>&#39;Joe&#39;: True,<br>&#39;Katy&#39;: True,<br>&#39;Lebron&#39;: False,<br>&#39;Margaret&#39;: False,<br>&#39;Robert&#39;: False,<br>&#39;Ronald&#39;: True,<br>&#39;William&#39;: False},</pre><pre>(&#39;And {} said, Why hast thou troubled us?&#39;, &#39;Biblical&#39;): {<br>&#39;Abraham&#39;: False,<br>&#39;David&#39;: False,<br>&#39;Isaac&#39;: False,<br>&#39;Jacob&#39;: False,<br>&#39;Jesus&#39;: False,<br>&#39;John&#39;: False,<br>&#39;Judas&#39;: False,<br>&#39;Mary&#39;: False,<br>&#39;Matthew&#39;: False,<br>&#39;Moses&#39;: False,<br>&#39;Samuel&#39;: False,<br>&#39;Simon&#39;: False},</pre><pre>(&#39;And {} said, Why hast thou troubled us?&#39;, &#39;Other&#39;): {<br>&#39;Ariana&#39;: False,<br>&#39;Barack&#39;: True,<br>&#39;Beyonce&#39;: True,<br>&#39;Bill&#39;: False,<br>&#39;Charles&#39;: False,<br>&#39;Coby&#39;: False,<br>&#39;Donald&#39;: False,<br>&#39;Frank&#39;: False,<br>&#39;George&#39;: False,<br>&#39;Helen&#39;: False,<br>&#39;Joe&#39;: False,<br>&#39;Katy&#39;: False,<br>&#39;Lebron&#39;: False,<br>&#39;Margaret&#39;: False,<br>&#39;Michael&#39;: False,<br>&#39;Robert&#39;: False,<br>&#39;Ronald&#39;: False,<br>&#39;William&#39;: False},</pre><pre>(&#39;My name is {}&#39;, &#39;Biblical&#39;): {<br>&#39;Abraham&#39;: False,<br>&#39;David&#39;: True,<br>&#39;Isaac&#39;: True,<br>&#39;Jacob&#39;: False,<br>&#39;Jesus&#39;: False,<br>&#39;John&#39;: True,<br>&#39;Judas&#39;: False,<br>&#39;Mary&#39;: True,<br>&#39;Matthew&#39;: True,<br>&#39;Moses&#39;: False,<br>&#39;Samuel&#39;: True,<br>&#39;Simon&#39;: False},<br> <br>(&#39;My name is {}&#39;, &#39;Other&#39;): {<br>&#39;Ariana&#39;: True,<br>&#39;Barack&#39;: True,<br>&#39;Beyonce&#39;: True,<br>&#39;Bill&#39;: True,<br>&#39;Charles&#39;: True,<br>&#39;Coby&#39;: True,<br>&#39;Donald&#39;: True,<br>&#39;Frank&#39;: True,<br>&#39;George&#39;: True,<br>&#39;Helen&#39;: True,<br>&#39;Joe&#39;: True,<br>&#39;Katy&#39;: True,<br>&#39;Lebron&#39;: True,<br>&#39;Margaret&#39;: True,<br>&#39;Robert&#39;: True,<br>&#39;Ronald&#39;: True,<br>&#39;William&#39;: True}}</pre><p>Although the numbers are different, we still see a difference between the two sets. However, this time it seems that old names (like Helen, William or Charles) are something the model is also struggling with.</p><h3>So what’s going on here?</h3><p>As part of our work on <a href="https://aka.ms/presidio">Presidio</a> (a tool for data de-identification), we develop models to detect PII entities. For that purpose, <a href="https://aka.ms/presidio-research">we extract template sentences</a> out of existing NER datasets, including CONLL-03 and OntoNotes 5. The idea is to augment these datasets with additional entity values, for better coverage of names, cultures and ethnicities. In other words, every time we see a sentence with a tagged person name on a dataset, we extract a template sentence (e.g. The name is [LAST_NAME], [FIRST_NAME] [LAST_NAME]) and later replace it with multiple samples each containing different first and last names.</p><p>When we manually went over the templating results, we figured out that there are still many names in our new templates dataset which didn’t turn into templates. A majority of these names came from the biblical sentences that OntoNotes 5 contains. So many of the samples in the OntoNotes 5 did not contain any PERSON labels, even though they did contain names, an entity type the OntoNotes dataset claims to support. It seems like these models actually learn the errors in the dataset, in this case to ignore names if they are biblical.</p><p>Obviously, these errors are found in both the train and test set, so a model that would learn that biblical names are not really names would also succeed on a similar test set. This is yet another example why <a href="https://paperswithcode.com/sota/named-entity-recognition-ner-on-ontonotes-v5">SOTA</a> results are not necessarily the best way to show progress in science.</p><h3>Is it only spaCy?</h3><p>A similar evaluation on two <a href="https://github.com/flairNLP/flair">Flair</a> models show that the a model trained on OntoNotes achieves significantly lower results on this test. The CONLL based model actually does pretty well!</p><p>CONLL-03 based model results:</p><pre>Model name: ner-english (CONLL)<br>Name set: Biblical, Template: &quot;My name is {}&quot;<br>Recall: 1.00<br><br>Name set: Other, Template: &quot;My name is {}&quot;<br>Recall: 1.00<br><br>Name set: Biblical, Template: &quot;And {} said, Why hast thou troubled us?&quot;<br>Recall: 1.00<br><br>Name set: Other, Template: &quot;And {} said, Why hast thou troubled us?&quot;<br>Recall: 1.00<br><br>Name set: Biblical, Template: &quot;And she conceived again, a bare a son; and she called his name {}.&quot;<br>Recall: 1.00<br><br>Name set: Other, Template: &quot;And she conceived again, a bare a son; and she called his name {}.&quot;<br>Recall: 0.94</pre><p>OntoNotes based model results:</p><pre>Model name: ner-english-ontonotes<br>Name set: Biblical, Template: &quot;My name is {}&quot;<br>Recall: 0.50<br><br>Name set: Other, Template: &quot;My name is {}&quot;<br>Recall: 1.00<br><br>Name set: Biblical, Template: &quot;And {} said, Why hast thou troubled us?&quot;<br>Recall: 0.00<br><br>Name set: Other, Template: &quot;And {} said, Why hast thou troubled us?&quot;<br>Recall: 0.83<br><br>Name set: Biblical, Template: &quot;And she conceived again, a bare a son; and she called his name {}.&quot;<br>Recall: 0.00<br><br>Name set: Other, Template: &quot;And she conceived again, a bare a son; and she called his name {}.&quot;<br>Recall: 0.00</pre><h3>Conclusion</h3><p>spaCy is one of the most exciting things happening in NLP today and it’s considered one of the most mature, accurate, fast and well documented NLP libraries in the world. As shown with the Flair example, this is an inherent problem in ML models and especially ML datasets.</p><p>Three relevant pointers to conclude:</p><ol><li>Andrew Ng recently argued that <a href="https://analyticsindiamag.com/big-data-to-good-data-andrew-ng-urges-ml-community-to-be-more-data-centric-and-less-model-centric/">the ML community should be more data-centric and less model-centric</a>. This post is another example of why this is true.</li><li>This is another example of an <a href="https://www.csail.mit.edu/news/major-ml-datasets-have-tens-thousands-errors">issue with a major ML dataset</a>.</li><li>A tool like <a href="https://github.com/marcotcr/checklist">Checklist</a> is really helpful to validate that your model or data doesn’t suffer from similar issues. Make sure you check it out.</li></ol><p><em>A Jupyter notebook for this blog post can be found </em><a href="https://gist.github.com/omri374/95087c4b5bbae959a82a0887769cbfd9"><em>here</em></a><em>.</em></p><p><a href="https://www.linkedin.com/in/omrimendels/"><strong>About the author: </strong>Omri Mendels</a> is a Principal Data Scientist at Microsoft.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c389cf7a24c9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/what-is-it-with-nlp-models-and-biblical-names-c389cf7a24c9">What is it with NLP models and biblical names?</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How we reduced our text similarity runtime by 99.96%]]></title>
            <link>https://medium.com/data-science-at-microsoft/how-we-reduced-our-text-similarity-runtime-by-99-96-e8e4b4426b35?source=rss-2b84200044b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/e8e4b4426b35</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[text-similarity]]></category>
            <category><![CDATA[machine-learning-python]]></category>
            <dc:creator><![CDATA[Omri Mendels]]></dc:creator>
            <pubDate>Tue, 23 Mar 2021 07:02:51 GMT</pubDate>
            <atom:updated>2021-05-25T00:02:42.525Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8JJbf2Xte2KG6-ULDQMzeg.png" /></figure><p>In many NLP pipelines, we wish to compare a query to a set of text documents. The process usually involves some encoding or embedding of the query, and a similarity measurement (maybe using cosine similarity) between the query and each document.</p><p>This article describes one aspect of a full system developed by the Commercial Software Engineering (CSE) team at Microsoft, together with developers and researchers from University College London’s <a href="https://eppi.ioe.ac.uk/cms/">EPPI Centre</a>. UCL EPPI Centre develops intelligent tools for <a href="https://en.wikipedia.org/wiki/Systematic_review">Systematic Literature Reviews</a>, used by researchers who conduct analyses in many public-health related domains, to come up with evidence-based suggestions for policymakers. Our main objective in this project was to keep reviews up to date by proposing new scientific papers to review owners. For more information on the project, <a href="https://devblogs.microsoft.com/cse/2021/02/10/developing-and-deploying-a-recommender-model-for-continuous-systematic-literature-reviews/">see this blog post</a>.</p><h3>Problem formulation</h3><p>In our setting, each review holds a set of papers already collected, and we wish to propose relevant new papers that the researchers might be interested in reviewing and including in their reviews. More formally, the problem is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jb-K1lUF4PVzSS8J1kk6EA.png" /></figure><p>Let’s say we have three reviews and a stream of new papers. We want to compare each new paper to each review to come up with a similarity measure and a recommendation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zv7QXdiwHNM7zhDKc310bw.png" /></figure><p>In our production setting, we are likely to get between 100,000 to 1.5 million new papers to score. The requirement is to make this process run for no more than a couple hours.</p><h3>Modeling approach</h3><p>Our dataset was comprised of title and abstract per paper. One of the most accurate approaches we looked into was TF/IDF on the paper level in each review. In this setting, we compare the new paper’s TF/IDF vector to TF/IDF vectors of all or some of the papers in each review. We then aggregate the paper-to-paper similarities to get a similarity metric between the new paper and an entire review. For example, we could look at mean similarity to the 10 closest neighbors. This approach turned out to be more accurate than methods like <a href="https://tfhub.dev/google/universal-sentence-encoder/4">Universal Sentence Encoder</a>, <a href="https://github.com/allenai/scibert">sciBERT</a>, <a href="https://openreview.net/forum?id=SyK00v5xx">SIF</a>, <a href="https://fasttext.cc/docs/en/supervised-tutorial.html#multi-label-classification">FastText </a>and others.</p><p>For pre-processing, we used spaCy to perform actions like tokenization, removal of stop words and special characters, and lemmatization. <a href="https://gist.github.com/omri374/ec1c243a5a94a657dae40078d47977b6">See this gist for more details</a>.</p><h3>Inference runtime improvements</h3><p>The pseudo-code for inference is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tCEd-gB4KfXkIC_xOPeBZA.png" /><figcaption>Inference phase pseudo-code</figcaption></figure><p>When the number of reviews is large, and the number of papers in each review is also large, this process can take a lot of time. In fact, the first time we ran the model as it was developed during research, it took 16 seconds on average to compare one test sample to all reviews, which translates to two weeks of processing for the number of test papers we have.</p><p>We had to come up with a way to improve it. We first highlighted our options:</p><ol><li>Faster pre-processing</li><li>Improvements to TF/IDF vectorization</li><li>Sampling papers from each review</li><li>Reducing the number of candidate test papers by using a rule-based logic that detects irrelevant papers (for instance, using other paper attributes like field of study)</li><li>Cosine distance calculation improvements (implementation and input representation)</li><li>Distributed inference (using Spark)</li></ol><p>As it turns out, significant improvement in the runtime process could be realized. After evaluating and benchmarking our system, we concluded that improvements with the most impact would come from representing the input matrices differently and selecting the right cosine distance implementation (point 5). Here’s how!</p><h4>Cosine distance implementation</h4><p>We looked at two main implementations: The <a href="https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.cosine_similarity.html">scikit-learn <em>cosine-similarity</em></a> and the <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cdist.html">scipy <em>cdist</em></a>. There are more, but these two are interesting from two main perspectives:</p><ol><li><strong>Data representation</strong>: In sklearn, the cosine-similarity method can accept a sparse matrix (<a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html">csr matrix</a>) that is comprised of a set of input vectors. Sparsity should be an advantage if the dataset is large. <em>cdist, </em>on the other hand, accepts ndarrays only.</li><li><strong>Implementation</strong>: The implementation in both packages is different. The <em>cdist </em>method <a href="https://github.com/scipy/scipy/issues/9172">is implemented entirely in C</a> and <em>cosine-similarity</em> uses numpy operators.</li></ol><p>Which one is better? The answer isn’t that obvious and relies on input size and representation.</p><h4>Input representation</h4><p>Cosine similarity is essentially a normalized dot product. So one question is how each input matrix is represented. In our setting, there are three main options:</p><ol><li>Compare each input vector (test paper TF/IDF) to each vector of each paper in each review.</li><li>Compare each input vector to a matrix comprised of one review’s papers vectors.</li><li>Compare each input vector to a global matrix comprised of all papers in all reviews.</li></ol><p>This figure outlines the difference among the three configurations:</p><figure><img alt="Three different ways to calculate cosine distance" src="https://cdn-images-1.medium.com/max/873/1*nj4UY7GGo8XojAPuUqF4Zg.png" /><figcaption>A new test paper could be compared to each paper from each review (1), to all papers in each review (2), or to all papers in all reviews (3). In each configuration, the cosine distance calculations takes place a different number of times on different matrix sizes.</figcaption></figure><figure><img alt="cosine distance illustration" src="https://cdn-images-1.medium.com/max/596/1*KZYwB3krFaGlg0juPG0D1w.png" /></figure><p>In order to examine which configuration would be the best, we created a short python script that fits the TF/IDF vectorizer on the full training set and performs inference on a relatively small number of test samples. We then ran it on about 20 different configurations and calculated the time it took on average for one test sample to be processed.</p><p>The following graph shows the average runtime for different configurations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HAB-IAdxo9i-hpngwUFRhQ.png" /><figcaption>Runtime for different configurations</figcaption></figure><p>It turns out that searching for the optimal configuration is not a linear process. Here’s a detailed description per development iteration:</p><ol><li>We started with the <em>sklearn </em>implementation with preprocessing happening for each class, as this is how the model was initially developed.</li><li>In iteration 2 we made pre-processing run once per sample.</li><li>Iteration 3: We changed the cosine calculation to <em>scipy </em>(about two seconds instead of about 12 between iterations 2 and 3).</li><li>Iteration 4: By creating a review matrix, we improved the runtime to about 1.5 seconds, but that wasn’t good enough.</li><li>The same configuration with the <em>sklearn</em> implementation (iteration 5) provided worse results.</li><li>In iteration 6 we implemented a global matrix (all training samples in one big matrix) using <em>cdist </em>as the similarity function.</li><li>In iteration 7 we returned to sklearn’s implementation, even though it provided worse results on all previous configurations. This time it turned out to be extremely efficient compared to previous iterations.</li><li>In iteration 8 we introduced some rule-based pre-filtering using field-of-study values to reduce the number of candidates.</li></ol><p>In conclusion, from step 1 to 8 we reduced 99.96% of the run-time per sample, even before leveraging parallel processing. Utilizing a global matrix was more or less the same for <em>scipy </em>(iterations 4 and 6), but much faster for <em>sklearn </em>(0.059 seconds per request, iteration 7). In this global matrix configuration (iterations 6, 7, 8), there is only one vector X matrix multiplication for each test sample.</p><p>Our analysis shows that the sparse representation, when the matrix is large, was responsible for the major improvement in runtime on iteration 7 when calculating cosine similarity between a test vector and a matrix comprised of all samples in all classes. When the matrix was smaller (due to a smaller data or the matrix-per-review configuration), the overhead of handling sparsity caused this implementation to be slower than the <em>scipy </em>option.</p><p>Additional changes we looked into were parameters to the TfidfVectorizer (like num_features to reduce the vector size), the sample size from each class, and whether to apply rule based pre-filtering.</p><h3>Tools</h3><h4>Profiling</h4><p>In addition to multiple runs of each configuration, profiling is an important aspect of runtime improvements. We have used <a href="https://jiffyclub.github.io/snakeviz/">SnakeViz</a>, an awesome Python profiling visualizer.</p><h4>Measuring time</h4><p>We found this simple context manager as a neat way of measuring time:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dcdfa3077aadda6f4ee20f54044743bd/href">https://medium.com/media/dcdfa3077aadda6f4ee20f54044743bd/href</a></iframe><h3>Conclusions</h3><p>Performance improvements are always tricky. The recipe of creating a small representative data example and evaluating multiple configurations worked for us. As these improvement steps are usually non-linear, changing one thing at a time might make you miss the optimal configuration.</p><p>Because we’re dealing with probabilistic methods, the accuracy of the model in hand should also be taken into account as there are obvious trade-offs between runtime and classification accuracy. In our experiments we always verified that the model’s accuracy was not reduced due to the changes in the model’s logic.</p><p>These performance improvements, in addition to leveraging Spark for parallel processing, reduced the runtime of the model on 500,000 test samples from days to less than two hours. Goal reached!</p><p>Special thanks to <strong>James Thomas, Paolo Tenti, Katya Mustafina, Nava Vaisman Levy and Ehsan Zare Borzeshi.</strong></p><p><em>Want to know more about how you can set up your Machine Learning projects for success? Read our recent article:</em></p><p><a href="https://medium.com/data-science-at-microsoft/setting-up-machine-learning-projects-for-success-4cba7840d24a">Setting up Machine Learning projects for success</a></p><p><a href="https://www.linkedin.com/in/omrimendels/"><em>Omri Mendels</em></a><em> is on LinkedIn.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e8e4b4426b35" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-at-microsoft/how-we-reduced-our-text-similarity-runtime-by-99-96-e8e4b4426b35">How we reduced our text similarity runtime by 99.96%</a> was originally published in <a href="https://medium.com/data-science-at-microsoft">Data Science + AI at Microsoft</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NLP approaches to data anonymization]]></title>
            <link>https://medium.com/data-science/nlp-approaches-to-data-anonymization-1fb5bde6b929?source=rss-2b84200044b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/1fb5bde6b929</guid>
            <category><![CDATA[pii]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[privacy]]></category>
            <dc:creator><![CDATA[Omri Mendels]]></dc:creator>
            <pubDate>Wed, 08 Jan 2020 03:34:52 GMT</pubDate>
            <atom:updated>2023-06-04T19:03:19.568Z</atom:updated>
            <content:encoded><![CDATA[<h3>Custom NLP Approaches to Data Anonymization</h3><h4>Practical ways for de-identifying real-world private data</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZFJTLvxqvquk6Y74cd7Xhg.png" /></figure><p>As internet services become ubiquitous, the aspiration for internet privacy continues to grow. In recent years, different laws such like GDPR which standardize the way services collect private information came into play. This brought the privacy aspects to the attention of every company and increased the investment in handling and anonymizing private data.</p><p>My job at the Commercial Software Engineering (CSE) team in Microsoft is to collaborate with Microsoft’s most strategic customers. Together, we develop new workloads in the fields of AI, data at scale, IoT and more. While engaging with these customers, we’ve realized that the PII (Personally Identifiable Information) issue is a recurring topic and blocker for many companies who wish to extend their set of solutions, either on-premises or in the cloud.</p><p>Therefore, we’ve decided to create <a href="https://aka.ms/presidio">Presidio</a>, a production ready open-source service, available for free, for anyone who wishes to address the data privacy problem.</p><p>Presidio allows any user to create standard and transparent processes for anonymizing PII entities on structured and unstructured data. To do so, it exposes a set of predefined PII recognizers (for common entities like names, credit card numbers and phone numbers), and tools for extending it with new logic for identifying more specific PII entities. In this blog post, we’ll focus on how Natural Language Processing could be leveraged to identify the different types of private entities.</p><h3>The PII detection process</h3><p>Presidio leverages a set of recognizers, each capable of detecting one or more PII entities in one or more languages. The process, described in figure 1, is generally comprised of 8 different steps :</p><ol><li>Get a request for anonymization from the user</li><li>Pass request to Presidio-Analyzer for PII entities identification</li><li>Extract NLP features (lemmas, named entities, keywords, part-of-speech etc.), to be used by the various recognizers</li><li>Fetch all PII recognizers (predefined + custom from the Recognizer Store service)</li><li>Run all recognizers</li><li>Aggregate results</li><li>Pass to Presidio-Anonymizer for de-dentification</li><li>Return de-identified text to caller</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j1U_1OYeD9Q8i_crg4E9sQ.gif" /><figcaption><em>Figure 1 — Animation of the identification process in Presidio</em></figcaption></figure><p>The following animation in figure 2 illustrates the same process on one specific example. Notice that when we change the context from “phone number” to “pool card number” the confidence of the phone number recognizer decreases. This is a screenshot of the Presidio Demo. <a href="https://huggingface.co/spaces/presidio/presidio_demo">Check it out</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OQNkIqeAu-Zq1Kkz9R742Q.gif" /><figcaption><em>Figure 2 — Example input and output</em></figcaption></figure><h3>NLP for data anonymization</h3><p>PII recognizers are required to detect different types of entities in free text. Different NLP approaches come to mind for such task:</p><ul><li>For entities which share a pattern, we could leverage <strong>Regular Expressions</strong>, <strong>validation</strong> (e.g. <a href="https://en.wikipedia.org/wiki/Checksum">checksum</a>) and <strong>context </strong>of surrounding words. This logic could be used, for example, to detect a credit card number or phone number.</li><li>For a finite list of options, we could use a <strong>blacklist. </strong>It could either be a static black-list (e.g. all the titles: <em>Mr., Ms., Mrs. Miss, Dr., Prof., …</em>) or dynamic (i.e. connected to a database and queries all possible options).</li><li>For entities which can be identified using specific logic, we can write <strong>rule-based</strong> recognizers.</li><li>For entities that require natural language understanding of the input, we could train <strong>Machine Learning</strong> models, specifically for <strong>Named Entity Recognition</strong> (NER), or use pre-trained models.</li></ul><p>In the next section, we will focus on the work we did around improving the Named Entity Recognition rates for person names, locations and organizations.</p><h3>NER for person names, locations and organizations</h3><p>To improve our detection rates on these three entities, we experimented with different models. The following section describes the datasets used, different models evaluated and results. The code for running this process can be found on our <a href="https://github.com/microsoft/presidio-research">GitHub repo for research</a>.</p><h4>Datasets</h4><p>While some labeled datasets are available, we wanted to increase the coverage of names, organizations, and locations. Thus we started with a labeled dataset (e.g., <a href="https://catalog.ldc.upenn.edu/LDC2013T19">OntoNotes </a>or <a href="https://www.aclweb.org/anthology/W03-0419.pdf">CoNLL-2003</a>), and processed it to extract templates. These examples were later used to generate new sentences with a wider variety of entity values (names, organizations and locations) than the original dataset.</p><p>For example, from the sentence “Thank you <strong><em>George</em></strong>!”, where George was manually labeled as person, we extracted the following template: “Thank you [PERSON]!”. Figure 3 provides an additional example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1nYOPSHiu7cDJv9t5JXDGA.png" /><figcaption><em>Figure 3 — Data augmentation example</em></figcaption></figure><p>We used a fake PII dataset and multiple fake PII generators to sample entities and create new sentences. These sentences were automatically labeled during the generation process, so training new NER models was easily applicable.</p><p>However, this process required us to do some pre-processing of the dataset and come up with creative solutions to different problems. To name a few:</p><ul><li>How to treat <em>country </em>vs. <em>nationality</em>? In many cases, a “location” entity refers to a nationality (or nation man or nation woman). For example, In the sentence “<em>The El-Salvadorian [LOC] boxer won the world championship</em>”, we can’t replace “<em>El-Salvadorian</em>” with “<em>Ethiopia</em>” as this would make the sentence incorrect. Therefore, we have created new intermediate entities for Country, Nationality, Nation-man and Nation-woman.</li><li>How should <em>gender </em>be handled? Some sentences originally talked about a male or female, but during the data generation process we might replace a name with a name from the opposite gender.</li><li>Should real-life knowledge matter? If the original sentence describes a conflict between two countries, should we just replace these countries with arbitrary country names?</li><li>Some person names in the dataset are the name of an institution or an organization, like the “Allen AI Institute” or “the Trump administration”. In this case, should we replace “Trump” with an arbitrary name?</li></ul><p>From 8,000 unique templates, we extracted 80,000 different labeled sentences. We also obtained a cleaner dataset than the previous one, as many unlabeled entities were handled during this process. The new dataset was split to train/test/validation sets, with no samples from the same template appearing in more than one set. In addition, we made 10% of the samples lower-case, as this generally represents the ratio of lowercase texts we might meet when the model is deployed. <strong>To the best of our knowledge, this the biggest PII dataset available to this day</strong>.</p><h4>Models</h4><p>Different approaches for modeling were evaluated. Specifically, we looked at Conditional Random Fields, spaCy based models and Flair based models.</p><p><strong>spaCy: </strong>spaCy is a production grade NLP library for tokenization, part-of-speech tagging, entity extraction, text classification and more. It contains a convolutional neural network model <a href="https://spacy.io/usage/facts-figures#benchmarks">which is considered to be the fastest deep learning NLP model</a>. While other models have higher accuracy on public datasets, it’s likely that they suffer from longer training and inference times. spaCy also provides fast tokenization and lemmatization, which is used in the context analysis module in Presidio. We evaluated different flavors of spaCy: First, we looked at results from the pretrained spaCy models (both 2.1.0 and 2.2.0), Then, we looked at fine-tuning a pretrained spaCy model, and finally, at training a spaCy model from scratch while leveraging pretrained word embeddings (FastText).</p><p><strong>Flair: </strong>Flair is a deep learning NLP toolkit with promising results on public datasets. It is built on top of PyTorch, and features both special embedding techniques (called <a href="https://drive.google.com/file/d/17yVpFA7MmXaQFTe-HDpZuqw9fJlmzg56/view">Flair embeddings</a>) and prediction models. Furthermore, it provides easy integration with other embedding models (like BERT, ELMo) and stacking of embeddings from different models and sources. We evaluated two different Flair based models: A Flair model with BERT embeddings, and a Flair model with a stacking of Flair embeddings and GloVe embeddings.</p><p><strong>Conditional Random Fields (CRF): </strong>CRFs are a class of methods for sequence tagging. These discriminative graphical models learn the dependencies between predictions and are a natural fit for Named Entity Recognition tasks. Prior to the introduction of recurrent and convolutional neural networks, CRFs achieved state of the art performance on NER tasks on public datasets. They are much faster in training and prediction than neural-network-based models and provide relatively interpretable results. We evaluated only the vanilla CRF model using L-BGFS optimization, using the <a href="https://sklearn-crfsuite.readthedocs.io/en/latest/">sklearn-crfsuite Python package</a>.</p><p>To conclude, here are the models we experimented with:</p><ol><li>Evaluate using default pre-trained models:</li></ol><ul><li><strong>spaCy</strong> 2.1.0</li><li><strong>spaCy</strong> 2.2.0 (which has <a href="https://explosion.ai/blog/spacy-v2-2#models">better support for lower-case entities</a>)</li></ul><p>2. Train from scratch:</p><ul><li><strong>Conditional Random Fields</strong> (CRF)</li></ul><p>3. Train using pretrained embeddings:</p><ul><li><strong>spaCy</strong> with <strong>FastText</strong> embeddings</li><li><strong>Flair</strong> with <strong>BERT</strong> embeddings</li><li><strong>Flair</strong> with <strong>GloVe</strong> and <strong>Flair</strong> embeddings</li></ul><p>4. Fine-tune an existing trained model:</p><ul><li><strong>spaCy</strong> version 2.2.0</li></ul><h4>Metrics</h4><p>We focused on F2 (recall is more important than precision) of the PII/no-PII binary decision. We also looked at specific classes F2 and computational performance. Lastly, interpretability was an additional factor to consider.</p><h4>Results</h4><p>As can be seen in figure 4, all models achieved decent results, but Flair based models were superior.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qxYa9fnVkT5pRpzuro3yNw.png" /><figcaption>Figure 4 — F2 results on various models and entities</figcaption></figure><p>Some insights from these results:</p><ul><li>Flair based models achieved higher results than spaCy based models, which achieved higher results on all entities than CRF.</li><li>We could see that spaCy 2.2.0 provides better results than 2.1.0 as expected, but not by a large amount.</li><li>spaCy and CRF models struggled with the Org entity, which is often confused with person names. Flair models achieve much higher results on Org which might hint on overfitting to the generated fake PII entities for Org.</li><li>Training spaCy models or fine-tuning them fail to improve the model’s F2 score. This might be due to the nature of the training set, which was derived from the same dataset spaCy was originally trained on (OntoNotes). It might be that other sources of data would benefit from fine-tuning or training these models.</li></ul><h4>Computational performance</h4><p>Although Flair models achieve higher F2 scores, they are also much slower to train and predict. Table 1 shows approximated training and inference time using the various methods evaluated. The analysis was made on one GPU machine<a href="#_ftn1">[1]</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VeeIi4-EKxZRd-QfDcFuFQ.png" /><figcaption><em>Table 1 — Inference and training time for the different models</em></figcaption></figure><p><a href="#_ftnref1">[1]</a> NC6 instance on Azure: 6 vCPU, 56GiB Memory, one half NVIDIA Tesla K80, GPU memory 12GiB</p><h3>Conclusion</h3><p>In this work, we evaluated various models for better detection rates in Presidio. We considered the trade-off between detection rates and computational performance, which is crucial in many use cases. For the datasets we evaluated, we see that there isn’t a practical reason to replace the current spaCy model we use in Presidio. It is, however, possible to apply the data augmentation framework and the different models evaluated to new data, and tailor Presidio towards more domain-specific datasets. We also see that by having a CRF model instead of spaCy, we can potentially improve Presidio’s run time. If performance is not an issue, for example for offline jobs, we should consider using a Flair based approach, potentially with Flair embeddings + GloVe, to improve the detection rates in Presidio.</p><p>Presidio is fully open source and available for free, for anyone who wishes to address the data privacy problem. We also welcome contributors, pull requests or any kind of feedback. <a href="https://github.com/microsoft/presidio">Click here to get started</a>.</p><h3>For more information</h3><ol><li><a href="https://aka.ms/presidio">Presidio’s GitHub repo</a></li><li><a href="https://microsoft.github.io/presidio/">General Presidio Documentation</a></li><li><a href="https://github.com/microsoft/presidio-research">Research, dataset creation and modeling code</a></li></ol><h3>Other references</h3><ol><li><a href="https://spacy.io/">spaCy</a></li><li><a href="https://github.com/flairnlp/flair">Flair</a></li><li><a href="https://www.fakenamegenerator.com/">Fake Name Generator</a></li><li><a href="https://github.com/joke2k/faker">Faker</a></li></ol><h3>About the author</h3><p>Omri Mendels is a Principal Data Scientist @ Microsoft. You can connect with him on <a href="https://www.linkedin.com/in/omrimendels/">LinkedIn</a>, <a href="https://github.com/omri374/">GitHub </a>or <a href="https://www.quora.com/profile/Omri-Mendels-1">Quora</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1fb5bde6b929" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/nlp-approaches-to-data-anonymization-1fb5bde6b929">NLP approaches to data anonymization</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>