<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anhduy Nguyen on Medium]]></title>
        <description><![CDATA[Stories by Anhduy Nguyen on Medium]]></description>
        <link>https://medium.com/@anguyen61191?source=rss-6a22ca3bcb5f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*p0m8PNr0vvRA9R7w</url>
            <title>Stories by Anhduy Nguyen on Medium</title>
            <link>https://medium.com/@anguyen61191?source=rss-6a22ca3bcb5f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 08:55:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anguyen61191/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Harmful Text]]></title>
            <link>https://medium.com/@anguyen61191/harmful-text-68785e7a1ec9?source=rss-6a22ca3bcb5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/68785e7a1ec9</guid>
            <category><![CDATA[harmful-speech]]></category>
            <dc:creator><![CDATA[Anhduy Nguyen]]></dc:creator>
            <pubDate>Mon, 27 Jun 2022 11:17:44 GMT</pubDate>
            <atom:updated>2022-06-27T11:17:44.134Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Purpose</strong></h3><p>I was reading a paper on <a href="https://arxiv.org/abs/2204.14256">Handling and Presenting Harmful Text</a> and I thought I should share it with the community. The purpose of the paper was to expound on harmful text and categorized it into three axes: the harm type, whether it is elicited as a feature of the research design from directly studying harmful content versus spuriously invoked from working on unrelated problems, but with datasets that contain harmful content, and who it affects. This is known as an unsolved problem in NLP(Natural Language Processing) with how these texts should be handled, presented, and discussed, but because there are these problems shouldn’t mean it shouldn’t be utilized.</p><h3>Summary</h3><p>The paper breaks down harm into three axes: what type of harm is contained in the content itself, why the appear during the research process, and who it effects and when. The authors than introduce a framework known as HarmCheck, a checklist for transparent, responsible, and reflexive reporting of textual harms. Due to the nature of NLP a need for large datasets that more than likely contain a great amount of harmful content. This content than needs to be cleaned before causing unnecessary problems during the modeling process. The paper identifies three major groups harmed by text data, which are those represented in the data, those working with the data, and those consuming research about the data. The author than proposes the checklist:</p><ol><li>Risk of Harm Protocol : Summarize the steps taken during the research progress to identify and mitigate harm to at-risk groups</li><li>Preview : Summarize any warning of harmful content and presentation of examples</li><li>Distance: Summarize distancing statements and views of authors</li><li>Disclaim: Summarize documentation of sources and origins of harmful content</li><li>Replace : Summarize any corrections, displacements or counter-claims to harmful content</li><li>Respect : Summarize any steps taken to protect the dignity and personal privacy of data subjects.</li></ol><h3>Conclusion</h3><p>In our growing world where data is gathered from every corner of the internet it more likely than not that text containing derogatory terms, harmful language, slurs, etc.. make way into data sets gathered by data scientists. This HarmCheck checklist should be utilized to reduce the amount of harm that these text could possibly inflict on not only the individuals mentioned in the text, but the scientist that work on the dataset and there readers that that learn about the data sets and models. This helps reduce the amount of trauma that might be caused by these data sets and also help scientist gain an insight on why the specific words were chosen and how it is utilized. This allows for the original intention of the words to come through, but give individuals a fair warning of what is contained in the text so they can better prepare themselves.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68785e7a1ec9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Choropleth Maps in Python]]></title>
            <link>https://medium.com/@anguyen61191/choropleth-maps-in-python-ce6a63f3b749?source=rss-6a22ca3bcb5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce6a63f3b749</guid>
            <category><![CDATA[choropleth-map]]></category>
            <dc:creator><![CDATA[Anhduy Nguyen]]></dc:creator>
            <pubDate>Mon, 30 May 2022 23:46:10 GMT</pubDate>
            <atom:updated>2022-05-30T23:46:10.932Z</atom:updated>
            <content:encoded><![CDATA[<p>Similar to my most recent post about fuzzy/percentage matching the inspiration for this post comes from my most recent project on housing data in the Seattle, Washington area. My collaborator and I though it would be cool to show a heat/choropleth map of the zip codes based on pricing and it ended up looking very nice. The struggle was finding all the documentation and the little blogs/guides/walkthroughs of making the map/graph the way we wanted it. I hope for those that are reading this post find it helpful.</p><p>This was the final product of our choropleth map.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*3BrShSHR04f6vasgVP1_HA.png" /></figure><h3><strong>Install Plotly</strong></h3><p>First and foremost you will need to install Plotly for Python. I won’t be going over that section. Plotly does that very well themselves and I’ll link to there installation page here <a href="https://plotly.com/python/getting-started/">https://plotly.com/python/getting-started/</a>.</p><h3><strong>Importing the appropriate files</strong></h3><p>I will first be importing the appropriate libraries of Pandas for work with dataframes, json to read the geojson files that go into the map making, and plotly.express the tool used to make the choropleth maps. My data set had contained zip codes so I specifically looked for geojson data with zip codes. Here is a <a href="https://github.com/OpenDataDE/State-zip-code-GeoJSON">link</a> to the repository of geojson that I used, but if your data has coordinates such as longitude and latitude that works also. I than read the geojson data into Washinton variable and my dataframe into df.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/05cf41020c79dbe033d02c3c51fe36ae/href">https://medium.com/media/05cf41020c79dbe033d02c3c51fe36ae/href</a></iframe><h3><strong>Choropleth Map Making</strong></h3><p>I than average out my dataframe data on zip codes for easier mapping. The code segment below shows you how to specifically create the choropleth map.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/110632547b354fd8c73662f8bf93e575/href">https://medium.com/media/110632547b354fd8c73662f8bf93e575/href</a></iframe><p>I will go over line by line what every part means.</p><p><strong>fig</strong> : is used to create the chart/map</p><p><strong>px.choropleth</strong> : is the module express from the plotly library, and choropleth is a method of express</p><p><strong>data_frame</strong> : is the data frame we are using to plot data onto the map</p><p><strong>locations</strong> : is the name of the column from the dataframe we are specifically tying to the geojson data for me this would be <em>zipcode</em></p><p><strong>featureidkey </strong>: is the location or the index we are tying the<strong> locations</strong> parameter to. In my case this would be <em>properties.ZCTA5CE10 </em>which refers to zip codes in my geojson data</p><p><strong>geojson</strong> : is the geojson data we retrieved earlier to make the background of the choropleth map</p><p><strong>color </strong>: is the measure from the dataframe we are using to compare to the location for me this is <em>price</em></p><p><strong>color_continuous_scale</strong> : is the color schema being used</p><p><strong>labels</strong> : Shows the labels of the graph</p><p><strong>fig.update_geos</strong> : is used to zoom in on the locations of the dataframe</p><h3><strong>Final Product</strong></h3><p>This is what the final product of the plotly map looks like. Because this is a png you can’t hover over the zip codes to show the pricing amount.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*5mx_McEayC_VLtScLR_LgQ.png" /></figure><p>This was a fun tool I used for my project and I hope this blog helps out others too. The difference in the looks of the final product using plotly and the original choropleth map was down to the difference in geojson data used.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce6a63f3b749" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fuzzy/percentage matching of strings in Python]]></title>
            <link>https://medium.com/@anguyen61191/fuzzy-percentage-matching-of-strings-in-python-7b766036b5aa?source=rss-6a22ca3bcb5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b766036b5aa</guid>
            <dc:creator><![CDATA[Anhduy Nguyen]]></dc:creator>
            <pubDate>Mon, 02 May 2022 17:56:00 GMT</pubDate>
            <atom:updated>2022-05-02T21:16:49.228Z</atom:updated>
            <content:encoded><![CDATA[<p>I recently finished a project dealing with movies and one of the main complications of the project was connecting the different data sources together using movie titles. You would think just using the <strong>in</strong> operator or the pandas <strong>.str.contains()</strong> would help solve my problems. This would’ve have been a quick solution if I was dealing with a dataset of only about 20 or so strings and I could hard code the movie titles together to get them matching with the <strong>.replace()</strong>, but the data set contained over 10,000 movies and this wasn’t a viable solution. There were also cases in the different data sources where the difference in the movie titles were if a semi colon was used or if the number 7 was written out or it was roman numerals. (ex. (Star Wars: Episode VII - The Force Awakens),(Star Wars : The Force Awakens)) I did what every decent programmer does I googled to see if anyone has already solved my problem. I ran into a couple of user made libraries/modules, but I actually found a Module in the Python Standard Library that solved my problems. It was the <strong>.get_close_matches()</strong> in the <a href="https://docs.python.org/3/library/difflib.html"><strong>difflib</strong></a> library.</p><h3><strong>.get_close_matches(word, possibilities, n, cutoff)</strong></h3><p><strong><em>word</em></strong><em>: The string used to get close matches</em></p><p><strong><em>possibilities</em></strong><em>: List of strings to match with the chosen word</em></p><p><strong><em>n:</em></strong><em> Optional parameter, default of 3, n &gt; 0,number of maximum closes matches</em></p><p><strong><em>cutoff:</em></strong><em> Optional parameter, default of .6, range of [0,1], percentage wise how close the strings in the list are to the chosen string</em></p><p>The .get_close_matches method returns a list of best matched strings that satisfy the percentage cutoff.</p><p><strong>Code Example:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*b0kEwh2S7ahyxTs9AyIHmA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*Ffb6NizQdjywt6bvDOKMMw.png" /></figure><p><strong>Link below to test code:</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ef1c73e1cfb025016c31c80ba0c96fc4/href">https://medium.com/media/ef1c73e1cfb025016c31c80ba0c96fc4/href</a></iframe><p>Test different parameters with link to browser ide below.</p><p><a href="https://www.online-python.com/">Online Python IDE</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b766036b5aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[So Why Data Science?]]></title>
            <link>https://medium.com/@anguyen61191/so-why-data-science-d2ec194a2cc4?source=rss-6a22ca3bcb5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2ec194a2cc4</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[jobs]]></category>
            <category><![CDATA[careers]]></category>
            <dc:creator><![CDATA[Anhduy Nguyen]]></dc:creator>
            <pubDate>Mon, 11 Apr 2022 15:54:30 GMT</pubDate>
            <atom:updated>2022-04-11T15:54:30.413Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FMvBDoSKPl55TBG1e_G6PA.jpeg" /></figure><p>I’ve been asked this question a few times in the past couple of weeks and I’ve been asking myself the same question. Before we answer that question of why, we should ask ourselves what is data science? Well the Wikipedia article states that “Data science is an interdisciplinary field that uses scientific methods , processes, algorithms and systems to extract knowledge and insights from noisy, structured and unstructured data, and apply knowledge and actionable insights from data across a broad range of application domain. Data science is related to data mining, machine learning, and big data.” So what does that mean to me? It means that data science is using massive amounts of data to answer question such as: Does a $5 coupon or a 25% off coupon result in more return customers, Is this an image of a cat or dog, Which aircraft is causing this radar signature, Is this pressure reading unusual, and What temperature will it be this weekend?</p><h4><strong>The Pay:</strong></h4><p>Than why did I get into Data Science? I had to think about this question for a good moment before being able to come up with an answer. Well one of the main reasons that any adult looking for a new career thinks about is the pay good? I can answer without a doubt and say the pay is very good. A quick google search shows that the average data scientist salary is $103,226 according to <a href="https://www.indeed.com/career/data-scientist/salaries">Indeed</a>. The second question is are their jobs available and is there room for growth in the field. The outlook of the job is very healthy. According to the <a href="https://www.bls.gov/ooh/computer-and-information-technology/computer-and-information-research-scientists.htm">Bureau of Labor Statistics</a> there is a growth of 22% between the period of 2020 to 2030. Which means the field is growing and new jobs are being made to accommodate for the demand. What is not to like about Data Science, great pay and great future prospects.</p><h4>The Interest:</h4><p>Do I enjoy making these graphs and charts that answer the questions that are asked? I would say yes. At my previous job as a database administrator/developer I was assigned to make charts and graphs in a BI tool called Tableau with data from the my previous companies database. I absolutely loved doing this. Being able to find changes and anomalies in the data by visualizing it in an easy to see format. I was able to catch errors in the database by being able to see the data visually instead of looking through tens of thousands of rows. This was one of the highlights of my day working there. I also was interested in the machine learning aspect of data science. I took a course in university that gave a brief overview of what machine learning is. A really cool implementation of machine learning is AlphaGo. AlphaGo is a software program that used machine learning by taking in data from human and computer play in a board game called Go to learn to identify the best moves that increases the percentage chance of winning the game. The AI program was than able to go and beat grandmasters in the game. The AlphaGo program has come to evolve into AlphaZero which added the games Shogi and Chess, and than into MuZero which learning without being taught rules. Through the subject of data science I’ll be able to learn more about the subject and feed my curiosity.</p><h4><strong>The Conclusion:</strong></h4><p>The two items that mattered most to me about switching careers and learning about data science was the job viability and the interest I had about the subject at hand. I had learned through some research that not only did jobs related to the data science field end up being financially rewarding, but the data science skill set is an in demand skill in the job market and there is growth in the data science field for the foreseeable future. The second is the interest I had in the subject. I was already interested in manipulating data shown by my previous job as a database administrator/developer. I felt that it would be a natural progression in career choice going from a database administrator to a data scientist because of my interest in machine learning and data visualization.</p><p><a href="https://en.wikipedia.org/wiki/Data_science#:~:text=Data%20science%20is%20an%20interdisciplinary,broad%20range%20of%20application%20domains">https://en.wikipedia.org/wiki/Data_science#:~:text=Data%20science%20is%20an%20interdisciplinary,broad%20range%20of%20application%20domains</a>.</p><p><a href="https://en.wikipedia.org/wiki/AlphaGo">https://en.wikipedia.org/wiki/AlphaGo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2ec194a2cc4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>