<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yu Huang, M.D., M.S. in CS on Medium]]></title>
        <description><![CDATA[Stories by Yu Huang, M.D., M.S. in CS on Medium]]></description>
        <link>https://medium.com/@jadezhang244?source=rss-759013c23ad5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*kKt3uVaMZnHTfcOSIFoF2A.jpeg</url>
            <title>Stories by Yu Huang, M.D., M.S. in CS on Medium</title>
            <link>https://medium.com/@jadezhang244?source=rss-759013c23ad5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 20:04:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jadezhang244/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Creating Insightful Dashboards with Spark and Tableau Desktop]]></title>
            <link>https://medium.com/data-science/creating-insightful-dashboards-with-spark-and-tableau-desktop-1fd91837b8b5?source=rss-759013c23ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/1fd91837b8b5</guid>
            <category><![CDATA[tableau]]></category>
            <category><![CDATA[spark-sql]]></category>
            <category><![CDATA[dashboard]]></category>
            <category><![CDATA[hands-on-tutorials]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Yu Huang, M.D., M.S. in CS]]></dc:creator>
            <pubDate>Thu, 29 Jun 2023 03:58:13 GMT</pubDate>
            <atom:updated>2023-06-29T13:35:22.919Z</atom:updated>
            <content:encoded><![CDATA[<h4>Large-scale data visualization automation with Tableau Desktop Tool</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*424lMbrFqa4eMjESS5pYhg.jpeg" /><figcaption>Photo by Author</figcaption></figure><h3>1. Introduction</h3><p>As a visual representation of data, data visualization is a widely adopted method in data analytics to gain useful business insights (e.g., trends, patterns, outliers, correlations, etc.) from large-scale datasets. Recently, I presented a software development method for using Spark, Plotly, and Dash to develop interactive and insightful data visualization dashboards for Web applications in Python [1].</p><p>Similarly to [1], this paper uses the same open source dataset as used in [2] to show how to use Spark and Tableau Desktop [3] to create insightful dashboards from large-scale datasets in Cloud data lake without programming.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*r8v0h-LYwGfKJhC4WkYf5Q.png" /><figcaption><strong>Figure 1:</strong> High-level overview of work flow.</figcaption></figure><p>Figure 1 shows the high-level overview of the work flow. It consists of the following major steps:</p><ul><li>connecting the Tableau Desktop for dashboard authoring to Spark</li><li>querying dataset from Cloud Data Lake</li><li>creating data visualization graphs from loaded dataset</li><li>creating dashboards from created individual graphs</li><li>publishing dashboards to Tableau server for sharing</li></ul><h3>2. Connecting Tableau Desktop to Spark</h3><p>As described in [4], the following steps can be followed to use Spark SQL as a distributed query engine using its JDBC/ODBC [5] and connect Tableau Desktop to the distributed Spark SQL Engine [4]:</p><ul><li>install Hadoop</li><li>setup Hive</li><li>setup MySQL</li><li>setup Spark</li><li>setup Tableau Desktop</li></ul><h3>3. Querying Dataset from Cloud Data Lake</h3><p>Once Tableau Desktop has been connected to the distributed Spark SQL engine successfully, we should be able to browse to the default schema and see the Hive Hadoop cluster tables [4].</p><p>From the perspective of creating dashboards with Tableau Desktop, there is no difference between a table loaded from a Hive Hadoop cluster and a table loaded from a local Microsoft Excel file. For convenience, the free version of Tableau Desktop Public with a local Excel file that is converted from the dataset <em>csv</em> file in [2] will be used for demonstration purpose in this paper.</p><h3>4. Creating Data Visualization Graphs</h3><p>We need to create individual visualization graphs first before visualization dashboards can be created.</p><p>We can use Tableau Desktop to create many different types of graphs. As described in [1], some of the graphs are suitable for visualizing continuous numeric features, while others are suitable for visualizing discrete categorical features.</p><p>Similarly to [1], this paper uses Tableau Desktop to create the following common diagrams for demonstration purpose.</p><ul><li><strong>Graphs for numeric features:</strong> scatter plot, histogram chart, and line chart</li><li><strong>Graphs for categorical features:</strong> bar chart, line chart, and pie chart</li></ul><h4>4.1 Graphs for Numeric Features</h4><p>Tableau Desktop uses the symbol<em> #</em> to indicate numeric features. This subsection shows how to use Tableau Desktop to create the following three common graphs for numeric features:</p><ul><li>scatter plot</li><li>histogram chart</li><li>line chart</li></ul><p><strong>4.1.1 Scatter Plot</strong></p><p>For a pair of numeric features, scatter plot uses each pair of feature values as coordinates to draw a point on a 2D plane. As example, as in [1], Figure 2 shows a scatter plot of two numeric features <em>Patient ID</em> and <em>Admission Deposit </em>for people from 21 to 30 years old. The feature <em>Type of Admission</em> is used for color coding.</p><p>The following steps can be followed to create the scatter plot:</p><ul><li>drag the feature <em>Patientid</em> and drop it into the <em>Columns</em> shelf</li><li>drag the feature <em>Admission Deposit</em> and drop it into the <em>Rows</em> shelf</li><li>drag the feature <em>Type of Admission</em> and drop it on to the <em>Color</em> property of the Marks Card</li><li>click the dropdown on Marks Card and select <em>Circle</em></li><li>right-click on the feature <em>Age</em>, choose <em>show filter</em>, and select <em>21–30</em> only</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NxV7BCYrX-lz15uq_lc6pQ.png" /><figcaption><strong>Figure 2:</strong> Sample scatter plot for a pair of numeric features.</figcaption></figure><p>This scatter plot reveals the business insight that the majority of emergency and trauma patients between 21–30 years old had a deposit in the range of $3,000 to $6,000.</p><p><strong>4.1.2 Histogram Chart</strong></p><p>Tableau Desktop treats a histogram as a bar chart. Because of this, the numeric feature for columns needs to be converted to categorical by binning, and the numeric feature for rows needs to be aggregated such as summation.</p><p>The following steps can be followed to create the histogram in Figure 3:</p><ul><li>right-click on the numeric feature <em>Patientid</em> and select “create bins” to create a new feature <em>Patientid (bin)</em>.</li><li>drag the new feature <em>Patientid (bin) </em>and drop it into the <em>Columns</em> shelf</li><li>drag the numeric feature <em>Admission Deposit</em> and drop it into the <em>Rows</em> shelf, and then click on the dropped feature and select aggregation <em>SUM</em></li><li>drag the categorical feature <em>Type of Admission</em> and drop it on to the <em>Color</em> property of the Marks Card</li><li>click the dropdown on Marks Card and select <em>Bar</em></li><li>right-click on the feature <em>Age</em>, choose <em>show filter</em>, and select <em>21–30</em> only</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jr0kUlIO3ZUvm2_5q5V4Hg.png" /><figcaption><strong>Figure 3:</strong> Sample histogram chart for a pair of numeric features.</figcaption></figure><p>This histogram shows us the business insight that urgent patients between 21–30 years old had the smallest total amount of admission deposit, while the emergency patients had the largest total amount of admission deposit.</p><p><strong>4.1.3 Line Chart</strong></p><p>Similar to the creation of scatter plot, the following steps can be followed to create the line chart in Figure 4:</p><ul><li>drag the feature <em>Patientid</em> and drop it into the <em>Columns</em> shelf</li><li>drag the feature <em>Admission Deposit</em> and drop it into the <em>Rows</em> shelf</li><li>drag the feature <em>Type of Admission</em> and drop it on to the <em>Color</em> property of the Marks Card</li><li>click the dropdown on Marks Card and select <em>Line</em></li><li>right-click on the feature <em>Age</em>, choose <em>show filter</em>, and select <em>21–30</em> only</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5GilfU4mNxTeX0STv1NXvg.png" /><figcaption><strong>Figure 4:</strong> Sample line chart for a pair of numeric features.</figcaption></figure><p>Similar to scatter plot, this line chart shows the same business insight that majority of emergency and trauma patients between 21–30 years old had a deposit in the range of $3,000 to $6,000.</p><h4>4.2 Graphs for Categorical Features</h4><p>Tableau Desktop uses the symbol <em>Abc</em> to indicate categorical features. This subsection shows how to use Tableau Desktop to create the following three of the common graphs for categorical feature value counts:</p><ul><li>bar chart</li><li>line chart</li><li>pie chart</li></ul><p><strong>4.2.1 Bar Chart</strong></p><p>As an example, the following steps can be followed to create the bar chart in Figure 5 for the value counts of the categorical feature <em>Stay</em>:</p><ul><li>drag the feature <em>Stay</em> and drop it into the <em>Columns</em> shelf</li><li>drag the same feature <em>Stay</em> and drop it into the <em>Rows</em> shelf, and then click on the dropped feature and select aggregation <em>Count</em></li><li>click the dropdown on Marks Card and select <em>Bar</em></li><li>right-click on the feature <em>Age</em>, choose <em>show filter</em>, and select <em>21–30</em> only</li><li>Click on the <em>Color</em> property of Marks Card and select a <em>purple</em> color</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uyEIF6ux2hBwIxDQxyB2JA.png" /><figcaption><strong>Figure 5:</strong> Sample bar chart for categorical feature value counts.</figcaption></figure><p>This bar chart shows the business insight that for patients between 21–30 years old, more patients stayed in hospital for 21–30 days compared with other hospitalization intervals.</p><p><strong>4.2.2 Line Chart</strong></p><p>Similar to bar chart, the following steps can be followed to create the line chart in Figure 6 for the value counts of the categorical feature <em>Stay</em>:</p><ul><li>drag the feature <em>Stay</em> and drop it into the <em>Columns</em> shelf</li><li>drag the same feature <em>Stay</em> and drop it into the <em>Rows</em> shelf, and then click on the dropped feature and select aggregation <em>Count</em></li><li>click the dropdown on Marks Card and select <em>Line</em></li><li>right-click on the feature <em>Age</em>, choose <em>show filter</em>, and select <em>21–30</em> only</li><li>Click on the <em>Color</em> property of Marks Card and select a <em>purple</em> color</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zTVM2nxephJC4lS_Tas4_w.png" /><figcaption><strong>Figure 6:</strong> Sample line chart for categorical feature value counts.</figcaption></figure><p>Similar to bar chart, this line chart reals the same business insight that for patients between 21–30 years old, more patients stayed in hospital for 21–30 days compared with other hospitalization intervals.</p><p><strong>4.2.3 Pie Chart</strong></p><p>The creation of pie chart is not as straight forward as the creation of line and bar charts.</p><p>we can follow the following steps to create the pie chart in Figure 8 for the value counts of the categorical feature <em>Stay</em>:</p><ul><li>drag the feature <em>Stay</em> and drop it into the <em>Columns</em> shelf</li><li>drag the same feature <em>Stay</em> and drop it into the <em>Rows</em> shelf, and then click on the dropped feature and select aggregation <em>Count</em></li><li>right-click on the feature <em>Age</em>, choose <em>show filter</em>, and select <em>21–30</em> only</li><li>click <em>Show Me</em> on the upper-right corner on the graphing sheet and select the <em>pie</em> icon. A small <em>pie</em> chart will show up as shown in Figure 7.</li><li>increase the size of the pie chart by selecting the <em>Size</em> property of the Marks Card and then selecting and dragging the bounding box of the pie chart</li><li>drag the aggregation feature <em>CNT(Stay)</em> and drop on to the <em>Label</em> property of Marks Card</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oiFF7e4FYQQMPplC3jHNNA.png" /><figcaption><strong>Figure 7:</strong> Converting bar chart to pie chart.</figcaption></figure><p>Figure 8 shows the final pie chart after following the above steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MW1WQLK4LqH2Lnrz2xsjKA.png" /><figcaption><strong>Figure 8:</strong> Sample pie chart for categorical feature value counts.</figcaption></figure><p>Similar to bar chart and line chat, this pie chart confirms the same business insight that for patients between 21–30 years old, more patients (2,197 in total) stayed in hospital for 21–30 days compared with other hospitalization intervals.</p><h3>5. Creating Dashboards</h3><p>Once individual graphs have been created, we can select and combine individual graphs into a dashboard.</p><h4>5.1 Dashboard for Visualizing Pair of Numeric Features</h4><p>The following procedure can be followed to create a dashboard for the visualization of the pair of numeric features <em>Patientid</em> and <em>Admission Deposit</em>.</p><p><strong>Step 1:</strong> select the <em>Dashboard</em> menu and then select <em>New Dashboard</em></p><p><strong>Step 2:</strong> drag the following created graph sheets one by one and drop on to the new dashboard sheet:</p><ul><li>Sample Scatter Plot of Pair of Numeric Features</li><li>Sample Histogram of Pair of Numeric Features</li><li>Sample Line chart of Pair of Numeric Feature</li></ul><p>Figure 9 shows the new created dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qNGp5EjrPdRg6F6jzJrtFA.png" /><figcaption><strong>Figure 9:</strong> Sample dashboard for visualizing a pair of numeric features.</figcaption></figure><h4>5.2 Dashboard for Visualizing Categorical Value Counts</h4><p>The following procedure can be followed to create a dashboard for the visualization of the value counts of the categorical feature <em>Stay</em>.</p><p><strong>Step 1:</strong> select the <em>Dashboard</em> menu and then select <em>New Dashboard</em></p><p><strong>Step 2:</strong> drag the following created graph sheets one by one and drop on to the new dashboard sheet:</p><ul><li>categorical_feature_bar</li><li>categorical_feature_line</li><li>categorical_feature_pie</li></ul><p>Figure 10 shows the new created dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mfL8rpGyceK_OSh8zBMMLA.png" /><figcaption><strong>Figure 10:</strong> Sample dashboard for visualizing categorical feature value counts.</figcaption></figure><h3>6. Publishing Dashboards</h3><p>Once a dashboard is created, it can be published to Tableau Server for sharing.</p><p>There are three types of servers:</p><ul><li>Tableau Public</li><li>Tableau Server</li><li>Tableau Cloud</li></ul><p>The dashboards created in this paper can only be published to Tableau Public because the free Tableau Desktop Public has been used for creating those dashboards.</p><p>The publishing steps in [6] can be followed to publish the dashboards with related dataset to Tableau Public server for public view.</p><h3>7. Conclusion</h3><p>Tableau is a multi-tier visual data analytics platform with complicated architecture. This paper presented a method to integrate Spark with Tableau for query data from large-scale data lake (e.g., Hadoop Hive) in Cloud first and then demonstrated how to use Tableau Desktop to create insightful dashboards from loaded datasets without programming.</p><p>As a dashboard authoring and sharing tool, Tableau Desktop supports many different ways of creating visualization graphs. It can be confusing to understand where to start and how to create insightful visualization graphs and dashboards. This paper can help to learn Tableau Desktop quickly by focusing on only two simple visualization scenarios: one for visualizing a pair of numeric features and the other for visualizing the value counts of a categorical feature.</p><h3>References</h3><p>[1] Yu Huang, <a href="https://towardsdatascience.com/developing-interactive-and-insightful-dashboards-with-spark-and-plotly-dash-5c0805341922">Developing Interactive and Insightful Dashboards with Spark and Plotly Dash</a></p><p>[2] Yu Huang, <a href="https://towardsdatascience.com/predicting-hospitalized-time-of-covid-19-patients-f4e70456db9b">Predicting Hospitalized Time of Covid-19 Patients</a></p><p>[3] <a href="https://help.tableau.com/current/guides/get-started-tutorial/en-us/get-started-tutorial-home.htm">Tutorial: Get Started with Tableau Desktop</a></p><p>[4] <a href="https://community.tableau.com/s/news/a0A4T000002O4RxUAK/a-guide-to-setting-up-tableau-with-apache-spark">A Guide to Setting up Tableau with Apache Spark</a></p><p>[5] <a href="https://spark.apache.org/docs/latest/sql-distributed-sql-engine.html">Distributed SQL Engine</a></p><p>[6] <a href="https://help.tableau.com/current/guides/get-started-tutorial/en-us/get-started-tutorial-share.htm#Use_Tableau_Public">Share your findings</a></p><p><strong>NOTE:</strong> All photos/pictures/graphs in this paper are by author.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1fd91837b8b5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/creating-insightful-dashboards-with-spark-and-tableau-desktop-1fd91837b8b5">Creating Insightful Dashboards with Spark and Tableau Desktop</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Developing Interactive and Insightful Dashboards with Spark and Plotly Dash]]></title>
            <link>https://medium.com/data-science/developing-interactive-and-insightful-dashboards-with-spark-and-plotly-dash-5c0805341922?source=rss-759013c23ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c0805341922</guid>
            <category><![CDATA[spark-sql]]></category>
            <category><![CDATA[dashboard]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[dash-plotly]]></category>
            <category><![CDATA[insights]]></category>
            <dc:creator><![CDATA[Yu Huang, M.D., M.S. in CS]]></dc:creator>
            <pubDate>Wed, 21 Jun 2023 14:18:49 GMT</pubDate>
            <atom:updated>2023-06-21T14:18:49.731Z</atom:updated>
            <content:encoded><![CDATA[<h4>Interactive Large-Scale Data Visualization for Web applications in Python</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lhK0GHV81RNr1MYLjDqLFg.jpeg" /><figcaption>Photo by Author</figcaption></figure><h3>1. Introduction</h3><p>Cloud data lake is widely adopted by enterprise organizations as a scalable and low cost repository of all types (structured and unstructured) of data. There are many challenges in analyzing a large-scale dataset from data lake efficiently to gain meaningful insights for data-driven decision-making. One challenge is that the dataset size tends to be too big to fit in one single machine. A cluster of servers are typically required for handling a big dataset. Another challenge is how to share the results of data analysis on servers with related customers/share holders anywhere easily and cost effectively.</p><p>This paper uses the same open source dataset as used in [1] to present an open-source based Web application framework for the development of interactive and insightful dashboards using Spark [2][3] and Plotly Dash[4]. This framework allows us to analyze and visualize large-scale dataset on servers and share the results of data analysis and visualization as dashboards anywhere.</p><p>As shown in the Figure 1, the new Web application framework consists of three major components:</p><ul><li>Spark SQL service (e.g., DataFrame) for distributed data processing (see Section 2)</li><li>Plotly graphing service for creating data visualization graphs as dashboards (see Section 3)</li><li>Dash Web service for interaction between server-side Plotly graphing service and the dashboard clients (see Section 4)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*mK3hwyb0oby6MI_DjbxVcw.png" /><figcaption><strong>Figure 1:</strong> High-level application framework architecture.</figcaption></figure><h3>2. Spark SQL Service for Distributed Data Processing</h3><p>As described in [2], PySpark (Python API for Spark) can be easily used to read <em>csv</em> files from Cloud data lake such as AWS S3. For simplicity, it’s assumed in this paper that a dataset <em>csv</em> file <em>train_data.csv</em> [1] is available on local machine without loss of generality.</p><p>The following code is to load the <em>csv</em> file into memory as a Spark SQL DataFrame:</p><pre>import pyspark<br>from pyspark.sql import SparkSession<br><br>spark = SparkSession.builder.appName(&#39;hospital-stay&#39;).getOrCreate()<br>spk_df = spark.read.csv(&#39;./data/train_data.csv&#39;, <br>                        header = True, inferSchema = True)</pre><p>After loading data, a global temporary view can be created as follows for the convenience of dynamic data query.</p><pre>spk_df.createOrReplaceTempView(&quot;dataset_view&quot;)</pre><p>Once the dataset view is created, we can use Spark SQL to query data like common data query from a database. As an example, the following code queries all of the rows from <em>dataset_view</em> where the age of people falls into the range of [21, 30].</p><pre>age = &quot;21-30&quot;<br>sdf = spark.sql(f&quot;SELECT * FROM dataset_view WHERE Age==&#39;{age}&#39;&quot;)</pre><p>In order to use Plotly to create data visualization graphs from a Spark DataFrame <em>sdf</em>, we must convert it into a Pandas DataFrame <em>pdf</em> because Plotly does not support Spark DataFrame directly.</p><pre>pdf = sdf.toPandas()</pre><h3>3. Plotly for Creating Data Visualization Graphs</h3><p>Plotly supports generation of many different types of graphs. Some of them are suitable for creating graphs from continuous numeric features, while others are suitable for creating graphs from discrete categorical features.</p><p>This paper uses Plotly Express library to create the following common diagrams for demonstration purpose.</p><ul><li><strong>Graphs for numeric features:</strong> scatter plot, histogram chart, and line chart</li><li><strong>Graphs for categorical features:</strong> bar chart, histogram chart, line chart, and pie chart</li></ul><h4><strong>3.1 Graphs for Numeric Features</strong></h4><p>As described before, three of common graphs for numeric features are:</p><ul><li>scatter plot</li><li>histogram chart</li><li>line chart</li></ul><p>Given a pair of numeric features, scatter plot uses each pair of feature values as coordinates to draw a point on a 2D plane. As example, the following figure shows a scatter plot of two numeric features <em>Patient ID</em> and <em>Admission Deposit </em>for people from 21 to 30 years old. The feature <em>Type of Admission</em> is used for color coding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MYYkNEtRVXLNYN1kDmZBvA.png" /><figcaption><strong>Figure 2:</strong> Sample scatter plot for a pair of numeric features.</figcaption></figure><p>Assuming that a dashboard user has selected age range [21, 30], a pair of numeric features <em>x</em> = <em>patientid</em> and <em>y</em> = <em>Admission_Deposit</em>, and <em>color</em> coding feature = <em>Type of Admission</em>, the following statement creates the above scatter plot.</p><pre>fig = px.scatter(dff, x = x, y = y, color = color_feature)</pre><p>Similarly, the following statement is to create a histogram of the same data:</p><pre>fig = px.histogram(dff, x = x, y = y, color = color_feature)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aNQkHxu5PKkDQylFHuVq3w.png" /><figcaption><strong>Figure 3: </strong>Sample histogram for a pair of numeric features.</figcaption></figure><p>To be complete, the following statement is used to create a line chart.</p><pre>fig = px.ine(dff, x = x, y = y, color = color_feature)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FtXfKdZ25QUbbuzY9rHuWA.png" /><figcaption><strong>Figure 4:</strong> Sample line chart for a pair of numeric features.</figcaption></figure><p>Even though we can easily create a line chart, but a line chart like the one above does not reveal useful insights. A good use of line chart is to apply it to a dataset that is sorted in a meaningful way such as a time-ordered sequence of data or a list of feature values sorted by counts as shown in Section 3.2.</p><h4><strong>3.2 Graphs for Categorical Features</strong></h4><p>This subsection shows four of the common graphs for categorical features:</p><ul><li>bar chart</li><li>histogram chart</li><li>line chart</li><li>pie chart</li></ul><p>Assuming that a dashboard user has selected <em>age</em> = [<em>21–30</em>], categorical <em>feature </em>= <em>Stay</em>, <em>color</em> = <em>purple</em>, <em>graph style</em> = <em>bar</em>, the following code can be used to generate the bar chart below.</p><pre>dff = spark.sql(f&quot;SELECT * FROM dataset_view WHERE Age==&#39;{age}&#39;&quot;).toPandas()<br>vc = dff[feature].value_counts()<br>fig = px.bar(vc, x = vc.index, y = vc.values)</pre><p>I noticed that a histogram chart has the same result as a bar chart for categorical feature value counts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7KKr46hG9iKP2stY4s6sjQ.png" /><figcaption><strong>Figure 5:</strong> Sample bar chart for value counts of a categorical feature.</figcaption></figure><p>We can create a line chart as follows by selecting <em>graph style</em> = <em>line</em>:</p><pre>fig = px.line(vc, x = vc.index, y = vc.values)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YHbZh16qB6GLXvHHqrWpXQ.png" /><figcaption><strong>Figure 6:</strong> Sample line chart for value counts of a categorical feature.</figcaption></figure><p>As mentioned before, line chart is suitable for visualizing value counts of a categorical feature.</p><p>The code below is to create a pie chart for the value counts of the same categorical feature of <em>Stay</em>.</p><pre>fig = px.pie(vc, x = vc.index, y = vc.values)</pre><p>This pie chart uses automatic color coding rather than the selected color <em>purple</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qt1nZ0r830hch30nPf3-3A.png" /><figcaption><strong>Figure 7:</strong> Sample pie chart for value counts of a categorical feature.</figcaption></figure><h3>4. Dash for Interactive Data Visualization</h3><p>The previous section describes how to create dashboards with different types of graphs using Plotly Express library on a cluster of Spark servers. This section shows how to use Dash to share dashboards with Web application clients and allow clients to use the dashboards to visualize data in various ways interactively.</p><p>The following procedure can be followed to develop a one page dashboard of a Web application:</p><ul><li>Step 1: import Dash library modules</li><li>Step 2: create a Dash application object</li><li>Step 3: define a dashboard layout of HTML page</li><li>Step 4: define callback functions (Web service end points)</li><li>Step 5: start server</li></ul><h4>4.1 Import Dash Library Modules</h4><p>As the first step, the Plotly Dash library modules are imported as follows for the purpose of demonstration in this paper.</p><pre>import plotly.express as px<br>from dash import Dash, dcc, html, callback, Input, Output</pre><h4>4.2 Create Dash Application Object</h4><p>After importing library modules, the next step is to create a Dash application object:</p><pre>app = Dash(__name__)</pre><h4>4.3 Define Dashboard Layout</h4><p>Once a Dash application object is created, we need to define a dashboard layout as a HTML page.</p><p>The dashboard HTML page is divided into two parts in this paper:</p><ul><li>Part 1: visualization of numeric features</li><li>Part 2: visualization of categorical features</li></ul><p>Part 1 of the dashboard layout is defined as follows:</p><pre>app.layout = html.Div([ # dashboard layout<br>    html.Div([ # Part 1<br>        html.Div([<br>            html.Label([&#39;Age:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown(<br>                [&#39;0-10&#39;, &#39;11-20&#39;, &#39;21-30&#39;, &#39;31-40&#39;, &#39;41-50&#39;, <br>                &#39;51-60&#39;, &#39;61-70&#39;, &#39;71-80&#39;, &#39;81-90&#39;, &#39;91-100&#39;,<br>                &#39;More than 100 Days&#39;],<br>                value=&#39;21-30&#39;,<br>                id=&#39;numerical_age&#39;<br>            ),<br>        ],<br>        style={&#39;width&#39;: &#39;20%&#39;, &#39;display&#39;: &#39;inline-block&#39;}),<br><br>        html.Div([<br>            html.Label([&#39;Numerical Feature x:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown(<br>                [&#39;patientid&#39;, <br>                 &#39;Hospital_code&#39;,<br>                 &#39;City_Code_Hospital&#39;],<br>                value=&#39;patientid&#39;,<br>                id=&#39;axis_x&#39;,<br>            )<br>        ], style={&#39;width&#39;: &#39;20%&#39;, &#39;display&#39;: &#39;inline-block&#39;}),<br><br>        html.Div([<br>            html.Label([&#39;Numerical Feature y:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown(<br>                [&#39;Hospital_code&#39;,<br>                 &#39;Admission_Deposit&#39;,<br>                 &#39;Bed Grade&#39;,<br>                 &#39;Available Extra Rooms in Hospital&#39;,<br>                 &#39;Visitors with Patient&#39;,<br>                 &#39;Bed Grade&#39;,<br>                 &#39;City_Code_Patient&#39;],<br>                value=&#39;Admission_Deposit&#39;,<br>                id=&#39;axis_y&#39;<br>            ),<br>        ], style={&#39;width&#39;: &#39;20%&#39;, &#39;display&#39;: &#39;inline-block&#39;}),<br><br>        html.Div([<br>            html.Label([&#39;Color Feature:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown(<br>                [&#39;Severity of Illness&#39;,<br>                 &#39;Stay&#39;,<br>                 &#39;Department&#39;,<br>                 &#39;Ward_Type&#39;,<br>                 &#39;Ward_Facility_Code&#39;,<br>                 &#39;Type of Admission&#39;,<br>                 &#39;Hospital_region_code&#39;],<br>                value=&#39;Type of Admission&#39;,<br>                id=&#39;color_feature&#39;<br>            ),<br>        ], style={&#39;width&#39;: &#39;20%&#39;, &#39;display&#39;: &#39;inline-block&#39;}),<br><br>        html.Div([<br>            html.Label([&#39;Graph Style:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown(<br>                [&#39;scatter&#39;,<br>                 &#39;histogram&#39;,<br>                 &#39;line&#39;],<br>                value=&#39;histogram&#39;,<br>                id=&#39;numerical_graph_style&#39;<br>            ),<br>        ], style={&#39;width&#39;: &#39;20%&#39;, &#39;float&#39;: &#39;right&#39;, &#39;display&#39;: &#39;inline-block&#39;})<br>    ], <br>    style={<br>        &#39;padding&#39;: &#39;10px 5px&#39;<br>    }),<br><br>    html.Div([<br>        dcc.Graph(id=&#39;numerical-graph-content&#39;)<br>    ]),<br>......</pre><p>Figures 2, 3, and 4 are created by using Part 1 of the dashboard layout.</p><p>Part 2 of the dashboard layout is defined as follows:</p><pre>......<br>html.Div([ # Part 2<br>        html.Div([<br>            html.Label([&#39;Age:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown(<br>                [&#39;0-10&#39;, &#39;11-20&#39;, &#39;21-30&#39;, &#39;31-40&#39;, &#39;41-50&#39;, <br>                &#39;51-60&#39;, &#39;61-70&#39;, &#39;71-80&#39;, &#39;81-90&#39;, &#39;91-100&#39;,<br>                &#39;More than 100 Days&#39;],<br>                value=&#39;21-30&#39;,<br>                id=&#39;categorical_age&#39;<br>            ),<br>        ],<br>        style={&#39;width&#39;: &#39;25%&#39;, &#39;display&#39;: &#39;inline-block&#39;}),<br><br>        html.Div([<br>            html.Label([&#39;Categorical Feature:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown(<br>                [&#39;Severity of Illness&#39;,<br>                 &#39;Stay&#39;,<br>                 &#39;Department&#39;,<br>                 &#39;Ward_Type&#39;,<br>                 &#39;Ward_Facility_Code&#39;,<br>                 &#39;Type of Admission&#39;,<br>                 &#39;Hospital_region_code&#39;],<br>                value=&#39;Stay&#39;,<br>                id=&#39;categorical_feature&#39;<br>            ),<br>        ], <br>        style={&#39;width&#39;: &#39;25%&#39;, &#39;display&#39;: &#39;inline-block&#39;}),<br><br>        html.Div([<br>            html.Label([&#39;Color:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown(<br>                [&#39;red&#39;,<br>                 &#39;green&#39;,<br>                 &#39;blue&#39;,<br>                 &#39;orange&#39;,<br>                 &#39;purple&#39;,<br>                 &#39;black&#39;,<br>                 &#39;yellow&#39;],<br>                value=&#39;blue&#39;,<br>                id=&#39;categorical_color&#39;<br>            ),<br>        ], <br>        style={&#39;width&#39;: &#39;25%&#39;, &#39;display&#39;: &#39;inline-block&#39;}),<br><br>        html.Div([<br>            html.Label([&#39;Graph Style:&#39;], style={&#39;font-weight&#39;: &#39;bold&#39;, &quot;text-align&quot;: &quot;center&quot;}),<br>            dcc.Dropdown([<br>                 &#39;histogram&#39;,<br>                 &#39;bar&#39;,<br>                 &#39;line&#39;,<br>                 &#39;pie&#39;],<br>                value=&#39;bar&#39;,<br>                id=&#39;categorical_graph_style&#39;<br>            ),<br>        ], <br>        style={&#39;width&#39;: &#39;25%&#39;, &#39;float&#39;: &#39;right&#39;, &#39;display&#39;: &#39;inline-block&#39;})<br>    ], <br>    style={<br>        &#39;padding&#39;: &#39;10px 5px&#39;<br>    }),<br><br>    html.Div([<br>        dcc.Graph(id=&#39;categorical-graph-content&#39;)<br>    ])<br>]) # end of dashboard layout</pre><p>Figures 5, 6, and 7 are created by using Part 2 of the dashboard layout.</p><h4>4.4 Define Callback Functions</h4><p>The dashboard layout only creates a static HTML page of a dashboard. Callback functions (i.e., Web service end points) must be defined so that a dashboard user’s action can be sent to a server-side callback function as a Web service request. In other words, callback functions enable interaction between dashboard users and server-side dashboard Web services such as creating a new graph upon user request (e.g., select a dropdown choice).</p><p>There are two callback functions defined in this paper for the two parts of the dashboard layout.</p><p>The callback function for Part 1 of the dashboard layout is defined as follows:</p><pre>@callback(<br>    Output(&#39;numerical-graph-content&#39;, &#39;figure&#39;),<br>    Input(&#39;axis_x&#39;, &#39;value&#39;),<br>    Input(&#39;axis_y&#39;, &#39;value&#39;),<br>    Input(&#39;numerical_age&#39;, &#39;value&#39;),<br>    Input(&#39;numerical_graph_style&#39;, &#39;value&#39;),<br>    Input(&#39;color_feature&#39;, &#39;value&#39;)<br>)<br>def update_numerical_graph(x, y, age, graph_style, color_feature):<br>    dff = spark.sql(f&quot;SELECT * FROM dataset_view WHERE Age==&#39;{age}&#39;&quot;).toPandas()<br><br>    if graph_style == &#39;line&#39;:<br>        fig = px.line(dff, <br>                 x = x, <br>                 y = y, <br>                 color = color_feature<br>                )<br>    elif graph_style == &#39;histogram&#39;:<br>        fig = px.histogram(dff, <br>                 x = x, <br>                 y = y, <br>                 color = color_feature<br>                )<br>    else:<br>        fig = px.scatter(dff, <br>                 x = x, <br>                 y = y, <br>                 color = color_feature<br>                )<br>        <br>    fig.update_layout(<br>        title=f&quot;Relationship between {x} vs. {y}&quot;,<br>    )<br><br>    return fig</pre><p>The callback function for Part 2 of the dashboard layout is defined as follows:</p><pre>@callback(<br>    Output(&#39;categorical-graph-content&#39;, &#39;figure&#39;),<br>    Input(&#39;categorical_feature&#39;, &#39;value&#39;),<br>    Input(&#39;categorical_age&#39;, &#39;value&#39;),<br>    Input(&#39;categorical_graph_style&#39;, &#39;value&#39;),<br>    Input(&#39;categorical_color&#39;, &#39;value&#39;)<br>)<br>def update_categorical_graph(feature, age, graph_style, color):<br>    dff = spark.sql(f&quot;SELECT * FROM dataset_view WHERE Age==&#39;{age}&#39;&quot;).toPandas()<br>    vc = dff[feature].value_counts()<br><br>    if graph_style == &#39;bar&#39;:<br>        fig = px.bar(vc, <br>                 x = vc.index, <br>                 y = vc.values<br>                )<br>    elif graph_style == &#39;histogram&#39;:<br>        fig = px.histogram(vc, <br>                 x = vc.index, <br>                 y = vc.values<br>                )<br>    elif graph_style == &#39;line&#39;:<br>        fig = px.line(vc, <br>                 x = vc.index, <br>                 y = vc.values<br>                )<br>    else:<br>        fig = px.pie(vc, <br>                 names = vc.index, <br>                 values = vc.values<br>                )<br>        <br>    if graph_style == &#39;line&#39;:<br>        fig.update_traces(line_color=color)<br>    elif graph_style != &#39;pie&#39;:<br>        fig.update_traces(marker_color=color)<br><br>    fig.update_layout(<br>        title=f&quot;Feature {feature} Value Counts&quot;,<br>        xaxis_title=feature,<br>        yaxis_title=&quot;Count&quot;<br>    )<br><br>    return fig</pre><p>Each callback function is associated with an annotation @<em>callback</em>. The annotation associated with a callback function controls which HTML components (e.g., dropdown) provide inputs to the callback function upon users’ request, and which HTML component (e.g., a graph within a <em>div</em> tag) receives the output of the callback function.</p><h4>4.5 Start Server</h4><p>The final step of a Dash Web application is to start a Web service server as below:</p><pre>if __name__ == &quot;__main__&quot;:<br>    app.run_server()</pre><p>The following diagram shows one scenario of the dashboard when a dashboard user has selected the following choices in the dashboard:</p><ul><li><em>age</em> from 21 to 30</li><li>pair of numeric features <em>patientid</em> and <em>Admission_Deposit</em></li><li>categorical feature <em>Type of Admission</em> for color coding of numeric features visualization</li><li><em>scatter</em> plot for numeric features visualization</li><li>Categorical feature <em>Stay </em>for calculating featue value counts</li><li>Color <em>blue</em> for bar, histogram, and line chart</li><li><em>pie</em> graph with automatic color coding for visualization of categorical feature value counts</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JMvwjoN18-BhJIN_o3byPA.png" /><figcaption><strong>Figure 8:</strong> One view of overall dashboard.</figcaption></figure><p>As an example of gaining possible useful insights, the above dashboard scenario reveals the following insights:</p><ul><li>the majority of patients between 21–30 years old had a deposit between $3,000 and $6,000 no matter how long they stayed in hospital</li><li>the majority of patients between 21–30 years old stayed in hospital for 11–30 days (27.6%) or 21–30 days (27.9%)</li></ul><p>The figure below shows another scenario of the dashboard when a dashboard user has selected the following choices in the dashboard:</p><ul><li>age from 21 to 30</li><li>pair of numeric features <em>patientid</em> and <em>Admission_Deposit</em></li><li>categorical feature <em>Type of Admission</em> for color coding of numeric features visualization</li><li>histogram for numeric features visualization</li><li>Categorical feature <em>Type of Admission</em></li><li>Color <em>green</em> for bar, histogram, and line chart</li><li>bar chart for visualization of categorical feature value counts</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LfqimQUzj6ERMdfUYZWj0g.png" /><figcaption><strong>Figure 9: </strong>Another view of the overall dashboard.</figcaption></figure><p>As another example of gaining possible useful insights, the above dashboard scenario reveals the following insights:</p><ul><li>the patients in urgent care had a higher total deposit than patients in other types of admission</li><li>the majority of patients were admitted as trauma</li></ul><p>In summary, the dashboard allows a user to visualize data in flexible ways to gain various useful insights interactively, including:</p><ul><li>visualizing both numeric and categorical features in a given range of age such as 0–10, 11–20, …, etc.</li><li>visualizing any pair of numeric features in scatter plots, histogram, and/or line chart</li><li>using any categorical feature values for color coding for numeric features visualization</li><li>visualizing value counts of any categorical features as bar/histogram chart, line plot, and/or pie chart with different color coding</li></ul><h3>5. Conclusion</h3><p>This paper presented an open-source based Web application framework in Python for the development of interactive and insightful dashboards using Spark [3] and Plotly Dash[4]. This framework allows us to analyze large-scale dataset from Cloud data lake, create interactive dashboards on Spark servers, and allow users to interact with the dashboards anywhere to visualize data in flexible ways to gain various useful insights.</p><h3>References</h3><p>[1] Yu Huang, <a href="https://towardsdatascience.com/predicting-hospitalized-time-of-covid-19-patients-f4e70456db9b">Predicting Hospitalized Time of Covid-19 Patients</a></p><p>[2] <a href="https://towardsai.net/p/programming/pyspark-aws-s3-read-write-operations">PySpark AWS S3 Read Write Operations</a></p><p>[3] <a href="https://spark.apache.org/examples.html">Apache Spark examples</a></p><p>[4] <a href="https://dash.plotly.com/?_gl=1*1wl0w5v*_ga*MTQ5MTkwNjIwMi4xNjg3MTkzNTM0*_ga_6G7EE0JNSC*MTY4NzIxNTgwOC4yLjAuMTY4NzIxNTgwOC4wLjAuMA..">Dash Python User Guide</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c0805341922" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/developing-interactive-and-insightful-dashboards-with-spark-and-plotly-dash-5c0805341922">Developing Interactive and Insightful Dashboards with Spark and Plotly Dash</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Clinical Named Entity Recognition Using spaCy]]></title>
            <link>https://medium.com/data-science/clinical-named-entity-recognition-using-spacy-5ae9c002e86f?source=rss-759013c23ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/5ae9c002e86f</guid>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[spacy]]></category>
            <category><![CDATA[ner]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Yu Huang, M.D., M.S. in CS]]></dc:creator>
            <pubDate>Wed, 13 Apr 2022 00:02:20 GMT</pubDate>
            <atom:updated>2022-04-13T15:54:15.169Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Healthcare Domain-Specific NER Method with spaCy</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_lLev6XKQXqWg0cS-N6Mig.jpeg" /><figcaption>photo by author</figcaption></figure><p>As described in my previous article [1], Natural Language Processing (<a href="https://en.wikipedia.org/wiki/Natural_language_processing">NLP</a>) becomes a hot topic in both <a href="https://en.wikipedia.org/wiki/Artificial_intelligence">AI</a> research and applications because text (e.g., English sentences) is a major type of natural language data. There are many AI methods for text data in healthcare domain such as clinical text classification, clinical Named Entity Recognition (<a href="https://en.wikipedia.org/wiki/Named-entity_recognition">NER</a>), …, etc.</p><p>In [1] I used an open source clinical text dataset [2][3] to present some of the common machine learning and deep learning methods for clinical text classification.</p><p>In this article, I use the same dataset to demonstrate how to implement a healthcare domain-specific Named Entity Recognition (<a href="https://en.wikipedia.org/wiki/Named-entity_recognition">NER</a>) method using spaCy [4].</p><p>The new NER method consists of the following steps:</p><ul><li>preprocessing dataset</li><li>defining domain-specific entities and types</li><li>generating annotated dataset</li><li>creating and training NER model</li><li>evaluating NER model</li><li>extending NER model</li><li>predicting and visualizing named entities</li></ul><h3>1. Preprocessing Dataset</h3><p>After downloading the dataset <em>mtsamples.csv</em> file from Kaggle [3], the dataset can then be loaded into memory as a Pandas DataFrame as follows:</p><pre>raw_df = pd.read_csv(&#39;./medical-nlp/data/mtsamples.csv&#39;, index_col=0)<br>raw_df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CiLMqjeqkeEFQyLeXiP5SA.png" /></figure><p>There are 40 unique categories of medical specialties in the dataset. As described in [1], the number of categories of medical specialties can be reduced from 40 to the following 9 categories by filtering the dataset in various ways:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*ahdxiKYEU1MaqX4ZGj4Aeg.png" /></figure><p>This filtered dataset is further preprocessed by changing the <em>transcriptions</em> column to lower case in this article for better demonstration.</p><h3>2. Defining Domain-Specific Entities and Types</h3><p>A NER model in spaCy is a supervised deep learning model. Thus labeled entities are required for each of the documents in the dataset for model training and testing.</p><p>Typically a text annotation tool such as <a href="https://prodi.gy/">prodigy</a> is used to annotate entities with types in documents. This article utilizes an alternative method to automatically generate annotated text dataset. To this end the entities (not their locations in documents) and types need to be specified.</p><p>For the purpose of demonstration, it is assumed that we are interested in the following entities (diseases or medicine) and types (medical specialties):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pwGa_YsIXUNFazI3Y9B5bA.png" /></figure><p>The entities for each entity type are stored as a set in the implementation. As an example, the following is the set of surgery:</p><pre>surgery = set([&#39;acute cholangitis&#39;, &#39;appendectomy&#39;, <br>               &#39;appendicitis&#39;, &#39;cholecystectomy&#39;, <br>               &#39;laser capsulotomy&#39;, &#39;surgisis xenograft&#39;, <br>               &#39;sclerotomies&#39;, &#39;tonsillectomy&#39; <br>               ])</pre><h3>3. Generating Annotated Dataset</h3><p>As described in the previous section, in order to train and test a spaCy NER model, each of the text documents that contain entities needs to be labeled with text annotations in the following template:</p><p><em>(‘document text’, {‘entities’: [(start, end, type), …, (start, end, type)]})</em></p><p>Once the entities and types are identified (see the previous section), an annotated text dataset can be automatically generated as follows:</p><ul><li>create a spaCy entity ruler model using the identified entities and types</li><li>use the entity ruler model to find the location and type of each entity from a given document</li><li>use the identified location and type and the template above to label the given document</li></ul><p>A spaCy entity ruler model can be created in three steps (see the __<em>init</em>__ method in the class <em>RulerModel</em> below):</p><ul><li>create an empty model for a given language (e.g., English)</li><li>add an <em>entity_ruler</em> pipeline component into the model</li><li>create entity rules and add them into the <em>entity_ruler</em> pipeline component</li></ul><pre>import spacy<br>from spacy.lang.en import English<br>from spacy.pipeline import EntityRuler</pre><pre>class RulerModel():<br>    def __init__(self, surgery, internalMedicine, medication, obstetricsGynecology):<br>        self.ruler_model = spacy.blank(&#39;en&#39;)<br>        self.entity_ruler =      self.ruler_model.add_pipe(&#39;entity_ruler&#39;)<br>        <br>        total_patterns = []</pre><pre>        patterns = self.create_patterns(surgery, &#39;surgery&#39;)<br>        total_patterns.extend(patterns)</pre><pre>        patterns = self.create_patterns(internalMedicine, &#39;internalMedicine&#39;)<br>        total_patterns.extend(patterns)</pre><pre>        patterns = self.create_patterns(medication, &#39;medication&#39;)<br>        total_patterns.extend(patterns)<br> <br>        patterns = self.create_patterns(obstetricsGynecology, &#39;obstetricsGynecology&#39;)<br>        total_patterns.extend(patterns)<br>        <br>        self.add_patterns_into_ruler(total_patterns)<br>        <br>        self.save_ruler_model()<br>        <br>    def create_patterns(self, entity_type_set, entity_type):<br>        patterns = []<br>        for item in entity_type_set:<br>            pattern = {&#39;label&#39;: entity_type, &#39;pattern&#39;: item}<br>            patterns.append(pattern)</pre><pre>        return patterns<br>    <br>    def add_patterns_into_ruler(self, total_patterns):<br>        self.entity_ruler.add_patterns(total_patterns)</pre><p>Once an entity ruler model has been created, it can be saved into file for later usage as follows:</p><pre>ruler_model.to_disk(&#39;./model/ruler_model&#39;)</pre><p>The method <em>assign_labels_to_documents</em>()<em> </em>in the<em> </em>class <em>GenerateDataset </em>below uses the above entity ruler model to find the location and type of entities and uses them to generate annotated dataset:</p><pre>class GenerateDataset(object):<br>    <br>    def __init__(self, ruler_model):<br>        self.ruler_model = ruler_model<br>        <br>    def find_entitytypes(self, text):<br>        ents = [] <br>        doc = self.ruler_model(str(text))<br>        for ent in doc.ents:<br>            ents.append((ent.start_char, ent.end_char, ent.label_))</pre><pre>        return ents     <br>    <br>    def assign_labels_to_documents(self, df):<br>        dataset = []<br>        text_list = df[&#39;transcription&#39;].values.tolist()</pre><pre>        for text in text_list:<br>            ents = self.find_entitytypes(text)<br>            if len(ents) &gt; 0:<br>                dataset.append((text, {&#39;entities&#39;: ents}))<br>            else:<br>                continue </pre><pre>        return dataset</pre><p>Once an annotated text dataset has been generated, it can be divided into subsets for model training, validation, and testing.</p><h3>4. Creating and Training NER Model</h3><p>Similar to entity ruler model, a spaCy NER model can be created in two steps (see the <em>__init__</em> method of the class <em>NERModel</em> below):</p><ul><li>create an empty model for a given language (e.g., English)</li><li>add a <em>ner</em> pipeline component into the model</li></ul><p>Once a NER model has been created, the method <em>fit</em>() of the class <em>NERModel</em> can then be called to train the model using the model training dataset.</p><p>The model training follows the typical deep learning model training process:</p><ul><li>set the number of epochs (default to 10 in this article)</li><li>shuffle training data and divide it into mini-batches for each epoch</li><li>setup the model hyper-parameters such as drop rate</li><li>get spaCy optimizer and use it to manage the calculation of objective function loss and the update of weights by back-propagation</li><li>show the model training progress (by using the <em>tqdm</em> library in this article)</li></ul><pre>import spacy<br>from spacy.util import minibatch<br>from spacy.scorer import Scorer<br>from tqdm import tqdm<br>import random</pre><pre>class NERModel():<br>    def __init__(self, iterations=10):<br>        self.n_iter = iterations <br>        self.ner_model = spacy.blank(&#39;en&#39;) <br>        self.ner = self.ner_model.add_pipe(&#39;ner&#39;, last=True)<br>        <br>    def fit(self, train_data):<br>        for text, annotations in train_data:<br>            for ent_tuple in annotations.get(&#39;entities&#39;):<br>                self.ner.add_label(ent_tuple[2])<br>        other_pipes = [pipe for pipe in self.ner_model.pipe_names <br>                       if pipe != &#39;ner&#39;]<br>        <br>        self.loss_history = []<br>        <br>        train_examples = []<br>        for text, annotations in train_data:<br>            train_examples.append(Example.from_dict(<br>               self.ner_model.make_doc(text), annotations))<br>        <br>        with self.ner_model.disable_pipes(*other_pipes): <br>            optimizer = self.ner_model.begin_training()<br>            for iteration in range(self.n_iter):<br>                print(f&#39;---- NER model training iteration {iteration + 1} / {self.n_iter} ... ----&#39;)<br>                random.shuffle(train_examples)<br>                train_losses = {}<br>                batches = minibatch(train_examples, <br>                  size=spacy.util.compounding(4.0, 32.0, 1.001))<br>                batches_list = [(idx, batch) for idx, batch in <br>                  enumerate(batches)]<br>                for idx, batch in tqdm(batches_list):<br>                     self.ner_model.update(<br>                         batch,<br>                         drop=0.5,<br>                         losses=train_losses,<br>                         sgd=optimizer,<br>                     )<br>                 <br>                self.loss_history.append(train_losses)<br>                print(train_losses)</pre><pre>        <br>    def accuracy_score(self, test_data):<br>        examples = []<br>        scorer = Scorer()<br>        for text, annotations in test_data:<br>            pred_doc = self.ner_model(text)<br>            try:<br>                example = Example.from_dict(pred_doc, annotations)<br>            except:<br>                print(f&#39;Error: failed to process document: \n{text}, <br>                      \n\n annotations: {annotations}&#39;)<br>                continue<br>            <br>            examples.append(example)<br>            <br>        accuracy = scorer.score(examples)<br>        <br>        return accuracy</pre><p>The following is the screenshot of the spaCy NER model training output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_A4j2mmXYeDVLECcdPWW6A.png" /></figure><p>The <em>loss</em> history has been saved in the <em>fit</em>() method of the <em>NERModel </em>class during model training.</p><pre>from matplotlib import pyplot as plt</pre><pre>loss_history = [loss[&#39;ner&#39;] for loss in ner_model.loss_history]<br>plt.title(&quot;Model Training Loss History&quot;)<br>plt.xlabel(&quot;Iterations&quot;)<br>plt.ylabel(&quot;Loss&quot;)<br>plt.plot(loss_history)</pre><p>The above code plots the following loss history diagram using the saved <em>loss</em> history data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*c4FqX3-541q6Q8wt3S1Xog.png" /></figure><p>Once a <em>NER</em> model has been trained, it can be saved into file for later usage as follows:</p><pre>ner_model.to_disk(&#39;./model/ner_model&#39;)</pre><h3>5. Evaluating NER Model</h3><p>Once a spaCy <em>NER</em> model has been trained, then the <em>accuracy_score</em>() method of the <em>NERModel </em>class can be called with the model testing dataset to get the model accuracy performance results.</p><p>The original accuracy result is a Python dictionary that contains the precision, recall, and f1 score for tokens, entities, and per entity type. Such a dictionary can be formatted as a Pandas DataFrame as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/475/1*fdRFKEubBxjIXSnr6dz1sg.png" /></figure><h3>6. Extending NER Model</h3><p>As described before, in this article an entity ruler model is created and used for automatically generating annotated text dataset.</p><p>The following code shows how to extend the trained <em>NER</em> model by combining the same entity ruler model and the <em>NER</em> model into one. This is useful because an entity recognized by the entity ruler model can be missed by the <em>NER</em> model due to the limitation of training data and/or time.</p><pre>from spacy.language import Language</pre><pre>def extend_model(surgery, internalMedicine, <br>                 medication, obstetricsGynecology):<br>    ruler_model = spacy.load(&#39;./model/ruler_model&#39;) <br>    base_ner_model = spacy.load(&#39;./model/ner_model&#39;) </pre><pre>    @Language.component(&quot;my_entity_ruler&quot;)<br>    def ruler_component(doc):<br>        doc = ruler_model(doc)<br>        return doc<br>    <br>    for entity_type_set in [surgery, internalMedicine, <br>                            medication, obstetricsGynecology]:<br>        for item in entity_type_set:<br>            base_ner_model.vocab.strings.add(item)</pre><pre>    if &#39;my_entity_ruler&#39; in base_ner_model.pipe_names:<br>        base_ner_model.remove_pipe(&#39;my_entity_ruler&#39;)</pre><pre>    base_ner_model.add_pipe(&quot;my_entity_ruler&quot;, before=&#39;ner&#39;) <br>    <br>    return base_ner_model</pre><h3>7. Recognizing and Visualizing Named Entities</h3><p>Once a spaCy NER model has been trained and/or extended, then we can use it to recognize named entities from a given text document as follows:</p><pre>doc = <em>extended_ner_model</em>(text)</pre><p>The recognized entities can be accessed via <em>doc.ents</em>.</p><p>The function below is to display the recognized named entities using color encoding in Jupyter notebook:</p><pre>def display_doc(doc): <br>    colors = { &quot;surgery&quot;: &quot;pink&quot;,<br>               &quot;internalMedicine&quot;: &quot;orange&quot;,<br>               &quot;medication&quot;: &quot;lightblue&quot;,<br>               &quot;obstetricsGynecology&quot;: &quot;lightgreen&quot;, <br>             }<br>    options = {&quot;ents&quot;: [&quot;surgery&quot;, <br>                        &quot;internalMedicine&quot;, <br>                        &quot;medication&quot;,<br>                        &quot;obstetricsGynecology&quot;,<br>                       ], <br>                &quot;colors&quot;: colors<br>              }</pre><pre>     displacy.render(doc, style=&#39;ent&#39;, <br>                     options=options, jupyter=True)</pre><pre>display_doc(doc)</pre><p>The following is the display of the annotated entities in two sample documents:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lzwzqBH-gmC0Qqt3eMqFiQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eYEZYjm-NjonFgE-BIyYzA.png" /></figure><h3>8. Conclusion</h3><p>In this article, I used the same dataset [2][3] as described in [1] to show how to implement a healthcare domain-specific Named Entity Recognition (<a href="https://en.wikipedia.org/wiki/Named-entity_recognition">NER</a>) method using spaCy [4]. In this method, first a set of medical entities and types was identified, then a spaCy entity ruler model was created and used to automatically generating annotated text dataset for model training and testing, after that a spaCy NER model was created and trained, and finally the same entity ruler model was used to extend the capability of the trained NER model. The results demonstrated that this new NER method is effective in recognizing clinical domain specific named entities.</p><h3>References</h3><p>[1] Y. Huang, <a href="https://towardsdatascience.com/common-machine-learning-and-deep-learning-methods-for-clinical-text-classification-188473477a32">Common Machine Learning and Deep Learning Methods for Clinical Text Classification</a></p><p>[2] MTSamples: <a href="https://www.mtsamples.com/">https://www.mtsamples.com</a></p><p>[3] T. Boyle, <a href="https://www.kaggle.com/tboyle10/medicaltranscriptions">https://www.kaggle.com/tboyle10/medicaltranscriptions</a></p><p>[4] spaCy: <a href="https://allenai.github.io/scispacy/">https://allenai.github.io/scispacy/</a></p><p>ACKNOWLEDGEMENT: I would like to thank MTSamples and Kaggle for the dataset.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ae9c002e86f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/clinical-named-entity-recognition-using-spacy-5ae9c002e86f">Clinical Named Entity Recognition Using spaCy</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Common Machine Learning and Deep Learning Methods for Clinical Text Classification]]></title>
            <link>https://medium.com/data-science/common-machine-learning-and-deep-learning-methods-for-clinical-text-classification-188473477a32?source=rss-759013c23ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/188473477a32</guid>
            <category><![CDATA[text-classification]]></category>
            <category><![CDATA[spacy]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[deep-dives]]></category>
            <dc:creator><![CDATA[Yu Huang, M.D., M.S. in CS]]></dc:creator>
            <pubDate>Mon, 21 Mar 2022 05:42:00 GMT</pubDate>
            <atom:updated>2022-03-27T04:05:29.589Z</atom:updated>
            <content:encoded><![CDATA[<h4>Natural Language Processing (NLP) for Healthcare Using NLTK, spaCy, TF-IDF, Word Embedding with non-sequence and sequence modeling such as LSTM and Transformer</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J_pYdTaKt2QgmO19aeOd-w.jpeg" /><figcaption>Photo by Author</figcaption></figure><p>With the advance of machine learning and deep learning algorithms, Natural Language Processing (<a href="https://en.wikipedia.org/wiki/Natural_language_processing">NLP</a>) becomes a hot topic in both AI research and applications because text (e.g., English sentences) is a major type of natural language data. For example, text frequently appears in healthcare datasets such as Electronic Health Records (<a href="https://en.wikipedia.org/wiki/Electronic_health_record">EHR</a>). There are many AI methods for text data in healthcare domain such as clinical Named Entity Recognition (<a href="https://en.wikipedia.org/wiki/Named-entity_recognition">NER</a>), clinical text classification, …, etc.</p><p>In this article I use an open source clinical text dataset [1][7][8] to present some of the common machine learning and deep learning methods for clinical text classification.</p><p>Similarly to the <a href="https://en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining">CRISP-DM</a>, a machine learning process for NLP consists of the following common steps:</p><ul><li>Business understanding</li><li>Data understanding</li><li>Data Preparation</li><li>Modeling</li><li>Model evaluation</li><li>Model Deployment</li></ul><h3>1. Business Understanding</h3><p>For any data science project, it is very important to understand what is the problem to be solved.</p><p>The problem to be addressed in this article is how to predict medical speciality from medical transcription.</p><h3>2. Data Understanding</h3><p>To solve the identified problem, we need to collect and understand the data.</p><p>As described before, I use the open source clinical text dataset from Kaggle [1]. After downloading the dataset <em>mtsamples.csv</em> file, the code below loads the dataset into memory as a Pandas DataFrame:</p><pre>data = pd.read_csv(&#39;./medical-nlp/data/mtsamples.csv&#39;, index_col=0)<br>data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CiLMqjeqkeEFQyLeXiP5SA.png" /></figure><p>As shown in the above table, the feature column <em>transcription</em> are English sentences, while the target/label column <em>medical_specialty</em> are categorical. This information indicates that the problem to be addressed is a clinical text classification issue.</p><p>There are 40 unique categories of medical specialties. The dataset is significantly imbalanced. The number of data samples in different categories varies from 6 to 1,103.</p><h3>3. Data Preparation</h3><p>As described before, the dataset is significantly imbalanced.</p><h4>3.1 Handling Imbalanced Data</h4><p>To alleviate the data imbalance issue, the following code removes the categories that either have less than 100 data samples or are not medical specialties.</p><pre>filtered_data = data[[&#39;transcription&#39;, &#39;medical_specialty&#39;]]<br>filtered_data.loc[:, &#39;medical_specialty&#39;] = filtered_data[&#39;medical_specialty&#39;].apply(lambda x:str.strip(x))<br>mask = (filtered_data[&#39;medical_specialty&#39;] == &#39;SOAP / Chart / Progress Notes&#39;) | \<br>       (filtered_data[&#39;medical_specialty&#39;] == &#39;Office Notes&#39;) | \<br>       (filtered_data[&#39;medical_specialty&#39;] == &#39;Consult - History and Phy.&#39;) | \<br>       (filtered_data[&#39;medical_specialty&#39;] == &#39;Emergency Room Reports&#39;) | \<br>       (filtered_data[&#39;medical_specialty&#39;] == &#39;Discharge Summary&#39;) | \<br>       (filtered_data[&#39;medical_specialty&#39;] == &#39;Letters&#39;)</pre><pre>filtered_data = filtered_data[~mask]</pre><pre>data_categories  = filtered_data.groupby(filtered_data[&#39;medical_specialty&#39;])<br>filtered_data_categories = data_categories.filter(lambda x:x.shape[0] &gt; 100)<br>filtered_data_categories[&#39;medical_specialty&#39;].value_counts()</pre><p>The resulting dataset has the following 9 categories of medical specialties:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*ahdxiKYEU1MaqX4ZGj4Aeg.png" /></figure><p>The dataset is shuffled as follows to avoid any artificial data patterns.</p><pre>data = filtered_data_categories.sample(frac=1.0)</pre><p>In addition, similarly to [1], the <a href="https://pypi.org/project/imbalanced-learn/">SMOTE</a> algorithm is also used to automatically upsampling minority data samples to improve data balance after text encoding.</p><h4>3.2 Data Preprocessing</h4><p>The major difference between a normal dataset and a NLP dataset is that the NLP dataset contains text features such as the medical transcription. A text feature consists of a list of sentences (called text or document corpus in NLP terminology). In terms of types of features, the difference is as follows:</p><p>There are two types of features in a normal dataset:</p><ul><li>numeric features (e.g., Temperature)</li><li>categorical features (e.g., States such as ‘IL’, ‘WA’, etc.)</li></ul><p>However there are three types of features in a dataset with NLP data:</p><ul><li>numeric feature (e.g., Temperature)</li><li>categorical feature (e.g., the feature <em>medical_specialty</em> as shown the above table)</li><li>text feature (e.g., the feature <em>transcription</em> in the above table)</li></ul><p>In order to handle missing data, and reduce the size of vocabulary and training data, except for extracting medical entities, the following data preprocessing steps are normally applied to a text feature (e.g., the <em>transcription</em> feature) for text classification:</p><ul><li>handle missing data</li><li>extract medical entities (for clinical text classification only)</li><li>standardize text</li><li>tokenize text</li><li>remove stopwords</li><li>stemming</li><li>lemmatize</li></ul><h4>3.2.1 Handle Missing Data</h4><p>Missing data is not allowed in machine learning. Missing data can be simply dropped if the number of missing entries is relatively small. The other common method in handling missing data in NLP is to copy information from other related feature in the dataset. For example the missing entries in the NLP feature ‘transcription’ can be filled up by copying the corresponding entries from the feature ‘description’ because the description is a summary of the transcription.</p><p>In this article the missing entries in the feature <em>transcription</em> column are simply dropped without significantly compromise of model prediction accuracy because there are only 33 missing entries.</p><pre>df = data.dropna(subset=[&#39;transcription&#39;])</pre><h4>3.2.2 Extract Medical Entities</h4><p>The most important contents in the feature column <em>transcription </em>are medical notes. As described in [1], it proves very useful to extract medical entities for clinical text classification. This is achieved by using spaCy [3] library as follows:</p><pre>import spacy<br>import en_ner_bionlp13cg_md</pre><pre>nlp = en_ner_bionlp13cg_md.load()</pre><pre>def medical_entities( text):<br>    entities = []<br>    doc = nlp(text)<br>    for ent in doc.ents:<br>        entities.append(ent.text)<br>    return &#39; &#39;.join(entities)</pre><pre>df[&#39;transcription&#39;] = df[&#39;transcription&#39;].apply(medical_entities)</pre><h4>3.2.3 Standardize Text</h4><p>There are two common steps to standardize text:</p><ul><li>change text to lower case: this helps to avoid the effort for a machine learning model to learn the case of characters.</li><li>remove punctuation: this helps to reduce the size of vocabulary</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qFSKaMPUE_By4_OcnD-HDw.png" /></figure><p>As an example, given the text string above, the two steps for standardizing text can be done in one statement as follows:</p><pre>text = text.lower().translate(str.maketrans(&#39;&#39;, &#39;&#39;, string.punctuation))</pre><p>The following is the resulting text:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A-jqZTTIi-t9p0VtfWT4zw.png" /></figure><h4>3.2.4 Tokenize text</h4><p>This helps to divide a text into elements of vocabulary. Word-level (1-gram) tokenization is commonly used if a text is treated as a sequence of words. Word-level tokenization also enables the activities of removing stopwords, stemming, and lemmatization.</p><p>N-gram ( typically N = 2 or 3) tokenization is usually used if a text is treated as a bag of words (the original ordered sequence of words is lost).</p><p>As an example, the code below uses scikit-learn library to tokenize a text string into individual words/tokens.</p><pre>from nltk.tokenize import word_tokenize<br>tokens = word_tokenize(text)</pre><h4>3.2.5 Remove Stopwords</h4><p>this helps to reduce the size of vocabulary without compromise of model prediction accuracy. The following code uses the NLTK library to remove English stopwords from tokenized words.</p><pre>from nltk.corpus import stopwords<br>english_stopwords = set(stopwords.words(&#39;english&#39;))</pre><pre>clean_tokens = []<br>for tok in tokens:<br>    tok = tok.strip() # remove space<br>    if tok not in english_stopwords:<br>        clean_tokens.append(tok)</pre><h4>3.2.6 Stemming</h4><p>This helps to reduce the size of vocabulary. The following code uses the NLTK [4] library for stemming a word/token.</p><pre>from nltk.stem import PorterStemmer<br>stemmer = PorterStemmer()<br>stemming_word = stemmer.stem(‘being’) </pre><h4>3.2.7 Lemmatize</h4><p>This helps to reduce the size of vocabulary. The code below uses the NLTK library to lemmatize a word/token.</p><pre>from nltk.stem import WordNetLemmatizer<br>lemmatizer = WordNetLemmatizer()<br>root_word = lemmatizer.lemmatize(&#39;words&#39;)</pre><p>The data preprocessing steps above can be combined into one function to clean up a text string:</p><pre>def clean_text(text):<br>    # get English stopwords<br>    english_stopwords = set(stopwords.words(&#39;english&#39;))<br>    <br>    # change to lower case and remove punctuation<br>    text = text.lower().translate(str.maketrans(&#39;&#39;, &#39;&#39;, string.punctuation))<br>    <br>    # divide string into individual words<br>    tokens = word_tokenize(text)<br>    <br>    stemmer = PorterStemmer()<br>    lemmatizer = WordNetLemmatizer()</pre><pre>    clean_tokens = []<br>    for tok in tokens:<br>        tok = tok.strip() # remove space<br>        if tok not in english_stopwords:<br>            clean_tok = lemmatizer.lemmatize(tok) # lemmatizition<br>            clean_tok = stemmer.stem(clean_tok) # Stemming<br>            clean_tokens.append(clean_tok)</pre><pre>     return &quot; &quot;.join(clean_tokens)</pre><p>The following is a sample clean text produced by calling the clean_text() function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eQSU72roRaF7jV45yCYc6w.png" /></figure><p>Note that Steps 3.2.3 and 3.2.5 are useful in general, but not necessary for the clinical text classification in this article because these two steps are already done in Step 3.2.2.</p><h4>3.3. Feature Engineering</h4><p>Once a text feature is preprocessed, each text (document) needs to be encoded as a vector of numbers (called vectorization) since machine learning and deep learning models only understand numbers. The selection of encoding methods depends on how text is interpreted. In text classification, a text string (document) is typically treated as either a bag of words or a sequence of words. The following are some of the common text encoding/vectorization methods.</p><p>If a text is treated as a bag of words, then we usually use the methods below to convert a text/document into one vector of numbers:</p><ul><li><a href="https://en.wikipedia.org/wiki/Tf%E2%80%93idf">TF-IDF</a> vectorization</li><li>Multi-Hot encoding</li></ul><h4>3.3.1 TF-IDF Vectorization</h4><p>The Multi-Hot [2] encoding method encodes one text/document as a sparse vector of the length of vocabulary with the values of 0’s and 1’s, which 1’s indicate the presence of the corresponding words/tokens in the vocabulary.</p><p>Similarly to Multi-Hot, the TF-IDF vectorization also encodes one text/document as a sparse vector of the length of vocabulary. The major difference is that a TF-IDF encoded vector contains not only the presence information of the corresponding words/tokens in the vocabulary but also the token frequency (TF) weighted by the inverted document frequency (IDF). Because of this, a TF-IDF encoded vector has more prediction power then a Muti-Hot encoded vector.</p><p>Like Multi-Hot, the length of a TF-IDF encoded vector can be the same as the length of the entire vocabulary. A typical truncated length is 20,000. Similarly to [1], the <a href="https://en.wikipedia.org/wiki/Principal_component_analysis#:~:text=Principal%20component%20analysis%20(PCA)%20is,components%20and%20ignoring%20the%20rest.">PCA</a> algorithm is applied to reduce dimensionality.</p><p>The following function converts the categorical labels into integers and uses 3-gram TF-IDF with PCA to encode the <em>transcription</em> feature into sparse vectors:</p><pre>from sklearn.preprocessing import LabelEncoder<br>from sklearn.decomposition import PCA<br>from imblearn.over_sampling import SMOTE</pre><pre>def tfidf_pca(X_feature, y_label):<br>    &#39;&#39;&#39;<br>    X_feature = df[&#39;transcription&#39;].values<br>    y_label = df[&#39;medical_specialty&#39;].values<br>    &#39;&#39;&#39;<br>    <br>    le = LabelEncoder()<br>    y_int = le.fit_transform(y_label)</pre><pre>    tfidf = TfidfVectorizer(tokenizer=clean_text, ngram_range=(1,3))<br>    pca = PCA(n_components=0.95)<br>    X_tfidf = tfidf.fit_transform(X_feature)<br>    X_pca = pca.fit_transform(X_tfidf.toarray())</pre><pre>    smote_over_sample = SMOTE(sampling_strategy=&#39;minority&#39;)</pre><pre>    X, y = smote_over_sample.fit_resample(X_pca, y_int.tolist())</pre><pre>    X = np.array(X)<br>    y = np.array(y)<br>    <br>    return X, y</pre><h4>3.3.2 Word Embedding</h4><p>If a text is treated as a sequence of words, then we can use the following common methods to convert each word in a text/document as one vector and the whole text/document as an array of vectors:</p><ul><li><a href="https://en.wikipedia.org/wiki/One-hot">One-Hot</a> Encoding: encode each word as a vector of the length of vocabulary (typically tens or hundreds of thousands)</li><li><a href="https://en.wikipedia.org/wiki/Word_embedding#:~:text=In%20natural%20language%20processing%20(NLP,to%20be%20similar%20in%20meaning.">Word Embedding</a>: encode each word as a dense vector of much shorter length (e.g., 100, 250, 500)</li></ul><p>In addition to shorter length, the other big advantage of word embedding over one-hot encoding is that word embedding encodes the meaning of the word in such a way so that the words that are closer in the vector space are similar in meaning.</p><p>Given a text/document, there are two steps to map it to a dense vector of a given size:</p><ul><li>Step 1: convert the text into a vector of word/token indices in the vocabulary</li><li>Step 2: use a custom trained or pre-trained word embedding algorithm (e.g., Word2Vec and GloVe) to convert each word index into a dense vector of float numbers</li></ul><p>The following function converts the categorical labels into integers and the <em>transcription</em> feature into vectors of word/token indices in the vocabulary.</p><pre>from tensorflow.keras import layers</pre><pre>def indecies_vectorize(X_feature, y_label, max_length = 1500, max_tokens = MAX_TOKENS):<br>    &#39;&#39;&#39;<br>    X_feature: df[&#39;transcription&#39;].values<br>    max_length: maximum number of words in one text/document<br>    max_tokens: the length of vocabulary<br>    &#39;&#39;&#39;<br>    <br>    le = LabelEncoder()<br>    y_int = le.fit_transform(y_label)<br>    <br>    text_vectorization = layers.TextVectorization(<br>        max_tokens=max_tokens,<br>        output_mode=&quot;int&quot;,<br>        output_sequence_length=max_length,<br>    )</pre><pre>    # training the text vectorization object<br>    text_vectorization.adapt(X_feature) </pre><pre>    # convert preprocessed training text into anrray of integers/indecies<br>    indecies_array = text_vectorization(X_feature) <br>    <br>    smote_over_sample = SMOTE(sampling_strategy=&#39;minority&#39;)<br>    X, y = smote_over_sample.fit_resample(indecies_array, y_int)<br>    <br>    X = np.array(X)<br>    y = np.array(y)</pre><pre>    return X, y</pre><p>The code below shows how to use a custom trained word embedding layer from Keras to encode a vector of word indices as a sequence of dense vectors of length 256.</p><pre>import keras<br>inputs = keras.Input(shape=(None,), dtype=&quot;int64&quot;)<br>embedded = layers.Embedding(input_dim=max_tokens, output_dim=256, mask_zero=True)(inputs)</pre><h4>3.3.3 Positional Embedding</h4><p>As described in [2], the word order information plays an important role in text classification. The following code from [2] is to add the word order information into word embedding to improve the performance of text classification using <a href="https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)">Transformer</a> [2].</p><pre>class PositionalEmbedding(layers.Layer):<br>    def __init__(self, sequence_length, input_dim, output_dim, **kwargs):<br>        super().__init__(**kwargs)<br>        self.token_embeddings = layers.Embedding(<br>            input_dim=input_dim, output_dim=output_dim)<br>        <br>        self.position_embeddings = layers.Embedding(<br>            input_dim=sequence_length, output_dim=output_dim)<br>        <br>        self.sequence_length = sequence_length<br>        self.input_dim = input_dim<br>        self.output_dim = output_dim</pre><pre>    def call(self, inputs):<br>        length = tf.shape(inputs)[-1]<br>        positions = tf.range(start=0, limit=length, delta=1)<br>        embedded_tokens = self.token_embeddings(inputs)<br>        embedded_positions = self.position_embeddings(positions)<br>        return embedded_tokens + embedded_positions</pre><pre>    def compute_mask(self, inputs, mask=None):<br>        return tf.math.not_equal(inputs, 0)</pre><pre>    def get_config(self):<br>        config = super().get_config()<br>        config.update({<br>            &quot;output_dim&quot;: self.output_dim,<br>            &quot;sequence_length&quot;: self.sequence_length,<br>            &quot;input_dim&quot;: self.input_dim,<br>        })<br>        return config</pre><p>The table below is a summary of the selections of different text encoding methods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*FX3dGqXoSgITlQkc6Zb1zg.png" /></figure><h3>4. Modeling</h3><p>Once the dataset is prepared for modeling, we need to select a machine learning/deep learning model and train it with the prepared data.</p><h4>4.1 Model Selection</h4><p>Similarly to the selection of text encoding methods, the selection of a machine learning or deep learning model for text classification depends on how text is encoded. Specifically, we can select a model as shown in the table below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*62TgULwJ28HuWEwKgX1EcQ.png" /></figure><p>The following TransformerEncoder class from [2] implements a multi-head attention-based transformer for clinical text classification in this article.</p><pre>import tensorflow as tf<br>from tensorflow import keras<br>from tensorflow.keras import layers</pre><pre>class TransformerEncoder(layers.Layer):<br>    def __init__(self, embed_dim, dense_dim, num_heads, **kwargs):<br>        super().__init__(**kwargs)<br>        self.embed_dim = embed_dim<br>        self.dense_dim = dense_dim<br>        self.num_heads = num_heads<br>        self.attention = layers.MultiHeadAttention(<br>            num_heads=num_heads, key_dim=embed_dim)<br>        self.dense_proj = keras.Sequential(<br>            [layers.Dense(dense_dim, activation=&quot;relu&quot;),<br>             layers.Dense(embed_dim),]<br>        )<br>        self.layernorm_1 = layers.LayerNormalization()<br>        self.layernorm_2 = layers.LayerNormalization()</pre><pre>   def call(self, inputs, mask=None):<br>        if mask is not None:<br>            mask = mask[:, tf.newaxis, :]<br>        attention_output = self.attention(<br>            inputs, inputs, attention_mask=mask)<br>        proj_input = self.layernorm_1(inputs + attention_output)<br>        proj_output = self.dense_proj(proj_input)<br>        return self.layernorm_2(proj_input + proj_output)</pre><pre>   def get_config(self):<br>        config = super().get_config()<br>        config.update({<br>            &quot;embed_dim&quot;: self.embed_dim,<br>            &quot;num_heads&quot;: self.num_heads,<br>            &quot;dense_dim&quot;: self.dense_dim,<br>        })<br>        return config</pre><p>The following models are selected for this article:</p><p>Non-sequence models for TF-IDF encoding:</p><ul><li>Logistic Regression</li><li>Random Forest</li><li><a href="https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html">Multi-Output Classifier</a> with <a href="https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm">k-NN</a></li></ul><p>Sequence models for word embedding:</p><ul><li>LSTM</li></ul><p>Sequence models for positional embedding:</p><ul><li>Transformer (i.e., the TransformerEncoder class)</li></ul><h4>4.2. Model Training</h4><p>Typically we need to split the dataset into training, validation, and testing subsets before model training begins. For simplicity, the dataset is only split into two subsets in this article: one for model training and the other for model testing/evaluation.</p><pre>from sklearn.model_selection import train_test_split</pre><pre>X_feature = df[&#39;transcription&#39;].values<br>y_label = df[&#39;medical_specialty&#39;].values</pre><pre>if SEQUENCE_MODEL:<br>    X, y = indecies_vectorize(X_feature, y_label)<br>else:   <br>    X, y = tfidf_pca(X_feature, y_label)</pre><pre>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)<br>X_train.shape, X_test.shape, y_train.shape, y_test.shape</pre><h4>4.2.1 Non-Sequence Modeling with TF-IDF</h4><p>As described before, the following non-sequence models for TF-IDF encoding are used in this article:</p><ul><li>Logistic Regression</li><li>Random Forest</li><li><a href="https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html">Multi-Output Classifier</a> with <a href="https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm">k-NN</a></li></ul><p>As an example, the following code creates and trains a multi-output classifier.</p><pre>from sklearn.multioutput import MultiOutputClassifier<br>from sklearn.neighbors import KNeighborsClassifier</pre><pre>if not SEQUENCE_MODEL:<br>    multi_class_clf = MultiOutputClassifier(KNeighborsClassifier())<br>    y_train = y_train.reshape(-1, 1)<br>    multi_class_clf.fit(X_train, y_train)</pre><h4>4.2.2 Sequence Modeling Using LSTM with Word Embedding</h4><p>In this article I use a bidirectional LSTM with custom word embedding to demonstrate how to perform sequence modeling for text classification (see the code below).</p><pre>import keras</pre><pre>def create_lstm_model(embedding_layer = None):<br>    # create input layer<br>    inputs = keras.Input(shape=(None,), dtype=&quot;int64&quot;)</pre><pre>    # add word embedding layer<br>    if embedding_layer is not None:<br>        embedded = embedding_layer(inputs)<br>    else:<br>        embedded = layers.Embedding(input_dim=MAX_TOKENS, output_dim=256, mask_zero=True)(inputs)</pre><pre>    # add LSTM layer<br>    x = layers.Bidirectional(layers.LSTM(32))(embedded)</pre><pre>    # add dropout layer<br>    x = layers.Dropout(0.5)(x)</pre><pre>    # add output layer<br>    outputs = layers.Dense(9, activation=&quot;softmax&quot;)(x)</pre><pre>    # combine all layers into one model<br>    lstm_model = keras.Model(inputs, outputs)</pre><pre>    # specifiy optimizer, loss, and metrics for the model<br>    lstm_model.compile(optimizer=&quot;rmsprop&quot;,<br>                  loss=&quot;sparse_categorical_crossentropy&quot;,<br>                  metrics=[&quot;accuracy&quot;])</pre><pre>    # print the summay of the model architecture<br>    lstm_model.summary()<br>    <br>    return lstm_model</pre><pre>if SEQUENCE_MODEL:<br>    if USE_GROVE:<br>        lstm_model = create_lstm_model(embedding_layer)<br>    else:<br>        lstm_model = create_lstm_model()<br>    <br>    # define callback function<br>    callbacks = [<br>        keras.callbacks.ModelCheckpoint(&quot;embeddings_bidir_gru_with_masking.keras&quot;,<br>                                        save_best_only=False)<br>    ]</pre><pre>    # train model<br>    lstm_model.fit(X_train, y_train, epochs=10, callbacks=callbacks)</pre><h4>4.2.3 Sequence Modeling with Transformer</h4><p>Recently the attention-based Transformer algorithm becomes a preferred algorithm over LSTM for sequence modeling. The code below uses a multi-head attention Transformer algorithm with positional embedding for clinical text classification.</p><pre>def create_transformer():</pre><pre>    vocab_size = 20000<br>    sequence_length = 1500<br>    embed_dim = 256<br>    num_heads = 2<br>    dense_dim = 32</pre><pre>    inputs = keras.Input(shape=(None,), dtype=&quot;int64&quot;)<br>    x = PositionalEmbedding(sequence_length, vocab_size, embed_dim)(inputs)<br>    x = TransformerEncoder(embed_dim, dense_dim, num_heads)(x)<br>    x = layers.GlobalMaxPooling1D()(x)<br>    x = layers.Dropout(0.5)(x)<br>    outputs = layers.Dense(9, activation=&quot;softmax&quot;)(x)<br>    transformer_model = keras.Model(inputs, outputs)<br>    transformer_model.compile(optimizer=&quot;rmsprop&quot;,<br>                  loss=&quot;sparse_categorical_crossentropy&quot;,<br>                  metrics=[&quot;accuracy&quot;])<br>    transformer_model.summary()<br>    <br>    return transformer_model</pre><pre>if SEQUENCE_MODEL:<br>    transformer_model = create_transformer()</pre><pre>    callbacks = [<br>        keras.callbacks.ModelCheckpoint(&quot;full_transformer_encoder.keras&quot;,<br>                                        save_best_only=False)<br>    ]</pre><pre>    transformer_model.fit(X_train, y_train, epochs=10, callbacks=callbacks)</pre><h3>5. Model Evaluation</h3><p>The common metrics for evaluating a classification model are:</p><ul><li>Accuracy (for balanced data)</li><li>Precision and Recall</li><li><a href="https://en.wikipedia.org/wiki/F-score">F1 score</a></li><li><a href="https://en.wikipedia.org/wiki/Receiver_operating_characteristic">AUC</a> (for binary classification only)</li><li><a href="https://en.wikipedia.org/wiki/Confusion_matrix">Confusion Matrix</a></li></ul><p>As an example, the code below calculates the accuracy and F1 scores and creates a Confusion Matrix diagram for the multi-class clinical text classification in this article.</p><pre>from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay</pre><pre>def plot_confusion_matrix(y_test, y_pred, labels):<br>    cm = confusion_matrix(y_test, y_pred, labels=rf_clf.classes_)<br>    disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=rf_clf.classes_) <br>    fig, ax = plt.subplots(figsize=(15,15))<br>    disp.plot(ax=ax)</pre><pre>if not SEQUENCE_MODEL:<br>    y_pred = logistic_clf.predict(X_test)</pre><pre>    acc_score = accuracy_score(y_test, y_pred)<br>    print(&#39;logistic_clf Accuracy score: &#39;, acc_score)</pre><pre>    f1 = f1_score(y_test, y_pred, average=&#39;weighted&#39;)<br>    print(&#39;logistic_clf F1 score: &#39;, f1)<br>    <br>    plot_confusion_matrix(y_test, y_pred,   labels=logistic_clf.classes_)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CYnMAvPTXJ7TE5S9E1ZygQ.png" /></figure><p>The following table is a summary of the model evaluation results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*KFpAGBnRiKuamxNAt3C5Ig.png" /></figure><h3>6. Model Deployment</h3><p>Once the best model for text classification has been identified in model evaluation for deployment, the next step is to deploy the identified model into a production system. One common method is to deploy the model as a Web service on a server, which can be called by other components in a target production system to get text classification results.</p><p>To this end both the selected trained model (e.g., Logistic Regression) and the related encoding objects shall be saved into files (e.g., Python pickle files). These saved encoding objects and model can then be loaded back into memory for prediction in the deployment environment.</p><h3>7. Conclusion</h3><p>In this article I used an open source clinical text dataset [1][7][8] to demonstrate some of the common text/document encoding methods and related machine learning and deep learning algorithms for clinical text classification.</p><p>There are many different types of text encoding methods and machine learning/deep learning algorithms available for text classification. This article provided a summary of mapping between different text encoding methods and corresponding applicable machine learning/deep learning models.</p><p>In addition to training word embedding using custom dataset, I also did experiment using the pre-trained word embedding GloVe [5] and noticed that using custom data to train word embedding has better model performance in accuracy.</p><p>The model evaluation results show that it is important to use spaCy to extract medical entities and use the SMOTE algorithm to balance data for better model performance in accuracy in clinical text classification.</p><h3>References</h3><p>[1] Clinical Text Classification: <a href="https://www.kaggle.com/ritheshsreenivasan/clinical-text-classification">https://www.kaggle.com/ritheshsreenivasan/clinical-text-classification</a></p><p>[2] F. Chollet, Deep Learning with Python, Second Edition, Manning, 2021</p><p>[3] spaCy: <a href="https://allenai.github.io/scispacy/">https://allenai.github.io/scispacy/</a></p><p>[4] NLTK: <a href="https://www.nltk.org/">https://www.nltk.org/</a></p><p>[5] GloVe: <a href="https://nlp.stanford.edu/projects/glove/">https://nlp.stanford.edu/projects/glove/</a></p><p>[6] Y. Huang, <a href="https://github.com/yuhuang3/machine-learning/tree/master/clinical_text_classification">Jupyter notebook in Github</a></p><p>[7] MTSamples: <a href="https://www.mtsamples.com/">https://www.mtsamples.com</a></p><p>[8] T. Boyle, <a href="https://www.kaggle.com/tboyle10/medicaltranscriptions">https://www.kaggle.com/tboyle10/medicaltranscriptions</a></p><p>ACKNOWLEDGEMENT: I would like to thank MTSamples and Kaggle for the dataset.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=188473477a32" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/common-machine-learning-and-deep-learning-methods-for-clinical-text-classification-188473477a32">Common Machine Learning and Deep Learning Methods for Clinical Text Classification</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Hospitalized Time of Covid-19 Patients]]></title>
            <link>https://medium.com/data-science/predicting-hospitalized-time-of-covid-19-patients-f4e70456db9b?source=rss-759013c23ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4e70456db9b</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mlflow]]></category>
            <category><![CDATA[covid19]]></category>
            <dc:creator><![CDATA[Yu Huang, M.D., M.S. in CS]]></dc:creator>
            <pubDate>Wed, 17 Nov 2021 18:24:49 GMT</pubDate>
            <atom:updated>2021-11-25T19:45:09.691Z</atom:updated>
            <content:encoded><![CDATA[<h4>Supervised Machine Learning in Healthcare</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UgE3_kiU8ZFIKJCFL6CKSA.jpeg" /><figcaption>Photo by Author</figcaption></figure><p>The outbreak of Covid-19 pandemic in 2020 caused tremendous equipment, materials deficiency and beds shortage problems in healthcare system across the U.S. [1]. The hospitals not only need to take care of the routine patients, but also need to take care of the sudden increased Covid-19 patients. A good planning and management of a hospital system becomes very important.</p><p>This article presents new methods on how to use machine-learning to predict how long (in days) a Covid-19 patient needs to stay in a hospital at the time of admission. This can help hospital professionals to make an optimized planning for patient treatment and resources (e.g., room, bed, etc.) allocation. This also could reduce the amount of hospital visitors and so decrease the chance of staff and visitor infection.</p><p>The rest of this article is arranged as follows:</p><ul><li>Data understanding</li><li>Data preparation</li><li>Modeling</li><li>Model evaluation</li><li>Deployment consideration</li></ul><h3>1. Data Understanding</h3><p>The dataset is obtained from <a href="https://www.kaggle.com/nehaprabhavalkar/av-healthcare-analytics-ii">Kaggle’s website</a> [1]. There are three files used in this article:</p><ul><li><em>train_data.csv</em>: containing features related to patient, hospital and Length of stay (label)</li><li><em>test_data.csv</em>: containing features related to patient, hospital. Need to predict the “length” of stay for each case id</li><li><em>train_data_dictionary.csv</em>: containing the information of the features in train and test files.</li></ul><p>As shown in Figure 1, there are there are 318,438 data samples, total 17 features and 1 label column (<em>Stay</em>) in this dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A-p8Ws7pUhJJd53bEaZDGQ.png" /><figcaption><strong>Figure 1: </strong>Transposed view of the train dataset.</figcaption></figure><p>Figure 2a shows the statistical summary of the following numeric features in the dataset:</p><ul><li>case_id</li><li>Hospital_code</li><li>City_Code_Hospital</li><li>Available Extra Rooms in Hospital</li><li>Bed Grade</li><li>patientid</li><li>City_Code_Patient</li><li>Visitors with Patient</li><li>Admission_Deposit</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uP9eytaOkRIlVkAKKbHuhw.png" /><figcaption><strong>Figure 2a:</strong> Statistical summary of the numeric features in the dataset.</figcaption></figure><p>The visualization of the distribution of the above numeric features is shown in Figure 2b.</p><p>It can be seen that the <em>case_id</em> feature values are uniformly distributed over bins because they are unique sequential numbers. This feature can be dropped due to lack of prediction power.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zBGI8Tdzs8VCYQEf59cclw.png" /><figcaption><strong>Figure 2b:</strong> Distribution of numeric features.</figcaption></figure><p>Figure 3 shows the patient visit distribution. We can see that many patients revisited the hospital many times (from 10 up to 50). So patient IDs are important in prediction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/1*6iq94zR8nAsOb5xPuRKcAw.png" /><figcaption><strong>Figure 3:</strong> Patient visit distribution.</figcaption></figure><p>Figure 4a shows the distribution of the categorical features and label column:</p><ul><li>Hospital type code</li><li>Hospital region code</li><li>Department</li><li>Ward Type</li><li>Ward Facility Code</li><li>Type of Admission</li><li>Severity of Illness</li><li>Age</li><li>Stay (Label)</li></ul><p>We can see that the distribution of the labels is significantly skewed to the right. In other words, the data is not balanced. There are very few data samples in the categories from ‘41–50’ to ‘61–70’. This will have significant negative impact on prediction power.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MsnOlLfDCp1ETEX-qqMOPg.png" /><figcaption><strong>Figure 4a: </strong>Distribution of categorical features.</figcaption></figure><p>Figure 4b shows the number of data samples in each of the label classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*UMN3-ofi31q6lx8WeMcdNw.png" /><figcaption><strong>Figure 4b:</strong> Count of label classes.</figcaption></figure><h3>2. Data Preparation</h3><p>With data understanding, The next step is to explore, clean and transform the collected raw dataset into appropriate format so that the transformed data can be effectively consumed by a target machine learning model.</p><h4>2.1. Handling missing data</h4><p>As shown in Figure 5, there are 113 missing data in the <em>Bed Grade</em> feature column and 4,532 missing data in the <em>City_Code_Patient</em> feature column. The total number of missing data is relatively small (total of 4,645) compared with the dataset size of 318,438 rows. In this case we can either drop the rows with missing data or replace missing data with 0. I choose to replace the missing data with 0 in order to be able to predict results for the <em>test_data</em> dataset with missing feature values in deployment. Refer to the DataCleaning class in data_preprocessing.py for details [6].</p><pre>train_data.isnull().sum()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/369/1*i8A6UPqVWgCg9R0vr2iWvg.png" /><figcaption><strong>Figure 5:</strong> Count of missing data.</figcaption></figure><h4>2.2 Dropping features (columns) without prediction power</h4><p>As described before, the <em>case_id</em> feature doesn’t have prediction power, so it is dropped in this project (see the DataCleaning class in data_preprocessing.py[6]).</p><h4>2.3 Categorical Encoding</h4><p><strong>2.3.1. Categorical label encoding</strong></p><p>The label column <em>Stay </em>in this dataset is categorical. It must be transformed into numbers for categorical feature target encoding [7] and deep learning model. The LabelEncoder algorithm [8] is used for the transformation (see TargetEncoding and OneHotEncoding classes in data_preprocessing.py[6]).</p><p><strong>2.3.2 Categorical feature target encoding</strong></p><p>The advantage of target encoding [7] is that it does not increase the dimensionality of the dataset. It has been used in this project for transforming categorical features into numbers for ensemble machine learning models XGBoost [2] and Random Forest [3] because these models don&#39;t work well with one-hot encoding due to high dimensionality. See the TargetEncoding class in data_preprocessing.py[6].</p><p><strong>2.3.3. Categorical feature one-hot encoding</strong></p><p>For comparison, the popular one-hot encoding method is used for transforming categorical features as well for deep learning model (see the OneHotEncoding class in data_preprocessing.py[6]).</p><p><strong>2.3.4. Other categorical feature transformation</strong></p><p>I noticed that the categorical Age feature (e.g. “21–30”) has more prediction power once it is transformed into numbers because the order of ages makes difference. In this project each range of age (e.g., “21–30”) has been transformed into an average number like (21+30)/2 = 25.5 (see the OneHotEncoding class in data_preprocessing.py[6]).</p><h4>2.4. Feature Normalization</h4><p>The numeric features are normalized into the range of [-1, 1] for deep learning (see the FeatureNormorlization class in data_preprocessing.py[6]).</p><h4>2.5. Data Splitting</h4><p>Finally, the preprocessed dataset is divided into two subsets: one for model training and the other for model evaluation.</p><h4>2.6. Data Preprocessing Pipelines</h4><p>The data preparation steps 2.1–2.5 have been combined into data preprocessing pipelines for convenience:</p><ul><li>target_encoding_preprocessing</li><li>target_encoding_preprocessing_for_prediction</li><li>onehot_encoding_preprocessing</li><li>onehot_encoding_preprocessing_for_prediction</li></ul><p>See data_preprocessing.py for details [6].</p><h3>3. Modeling</h3><p>After data preparation, we are ready for modeling. The main goals of modeling include:</p><ul><li>Identify potential machine learning models</li><li>Train models and tune the hyper-parameters of models</li></ul><h4>3.1. Model Selection</h4><p>This project addresses a classification issue because the label is categorical. This is suitable for supervised machine learning classification models such as <em>XGBoost</em> [2], <em>Random Forest </em>[3], deep learning multilayer perceptron (<a href="https://en.wikipedia.org/wiki/Multilayer_perceptron"><em>MLP</em></a><em>)</em> classifiers [4].</p><p>Because the data is in tabular format and the number of features is relatively small, <em>XGBoost</em> and <em>Random Forest</em> are generally preferred compared with deep learning models.</p><p>In this project <em>XGBoost</em>, <em>Random Forest,</em> and <a href="https://en.wikipedia.org/wiki/Multilayer_perceptron"><em>MLP</em></a> classifiers are selected for experiments.</p><h4>3.2. Model Training and Hyper-Parameters Tuning</h4><p>Both <em>XGBoost </em>and <em>Random Forest</em> model training are performed with 10 folder cross validation. <em>Grid search</em> is used for selecting the best combination of hyper-parameters. Cross validation is also used for training the deep learning model. See the following functions in train_test_classifier.py:</p><ul><li><em>build_xgboost_model</em></li><li><em>build_rf_model</em></li><li><em>build_deeplearning_model</em></li></ul><h3>4. Model Evaluation</h3><p>Once different machine learning models have been trained, the performance of these models needs to be evaluated so that we can select the best model for deployment.</p><p>The classification accuracy, F1 score, and confusion matrix are used as the main evaluation metrics for this project. Refer to the following functions in train_test_classifier.py for details:</p><ul><li><em>evaluate_xgboost_model</em></li><li><em>evaluate_rf_model</em></li><li><em>evaluate_dl_model</em></li></ul><p>Typically the accuracy is not a good metric in the case of binary classification with imbalanced data because it can easily achieve high percentage of accuracy by simply predicting the majority class. However this is not necessary the case in multi-classes classification. For example, in the dataset for this project, there are only 87,491 data samples for the majority class 21–30. The total number of data samples is 318,438. If a model always predicts the majority class, the accuracy would be about 27.5%. Thus the accuracy can still be a good model performance measurement in the case of multi-classes classification since it represents the average accuracy of classifications of different label classes.</p><p>In addition to accuracy, F1 score is used to measure both XGBoost and Random Forest model performance as well since it can measure the balance of precision and recall, which is suitable for imbalanced data.</p><p>The confusion matrix is used because it can clearly tell us what prediction mistakes a model made.</p><h4>4.1. MLflow</h4><p>In order to effectively tracking model hyper-parameters and performance metrics, the <em>MLflow</em> tool [5] is used. In particular, I developed the function mlFlow() in train_test_classifier.py to combine the following activities into one procedure:</p><ul><li>Loading training data</li><li>Preprocessing data</li><li>Training model</li><li>Evaluating model</li></ul><p>As an example, the following <em>mlFlow</em>() function call produces the results (e.g., the confusion matrix in Figure 6) for a XGBoost model.</p><pre>target_encoders, label_encoder = mlFlow()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hk8QDrAQ9YEiGhAurbxuUA.png" /><figcaption><strong>Figure 6: </strong>XGBoost confusion matrix.</figcaption></figure><p>The confusion matrix in Figure 6 shows that the more data the better the prediction results. For example, the maximum number 14,429 of true positives is in the category of 21–30.</p><p>Figure 7 shows the results of four model evaluations:</p><ul><li><em>XGBoost</em> with target encoding</li><li><em>Random Forest</em> with target encoding</li><li><em>MLP</em> with target encoding</li><li><em>MLP</em> with one-hot encoding</li></ul><p>We can see that <em>XGBoost </em>has the best performance in both accuracy and F1 scores for the given dataset and so we can select it for deployment if its performance meets business requirements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NNOxo8LEx5f2T7ZOGzD65Q.png" /></figure><p>As shown in Figure 7, the best accuracy score is only about 42.4%. This is because there are too little data in the majority of the labeled categories such as the following ranges: 41-50, from 61-70 all way up to more than 100.</p><p>The accuracy score will increase if more data can be collected to balance the dataset. As an experiment, I selected only the data samples in the categories of 11-20 and 21-30 and noticed that the accuracy score was increased to about 63%. Figure 8 shows the corresponding confusion matrix from a trained Random Forest model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3U0UFcZNWq8qiidq0isZXQ.png" /><figcaption><strong>Figure 8:</strong> Confusion Matrix for a Random Forest model with two label categories.</figcaption></figure><p>In order to understand how different features impacted the prediction, Figure 9 shows the feature importance produced by a trained Random Forest model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ETVc2L3eHf1zET_Zx725JA.png" /><figcaption><strong>Figure 9:</strong> Feature importance produced by a Random Forest model.</figcaption></figure><p>I also looked into the correlations between the features and the target labels as shown in Figure 10.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*iQ2EclQWYYUq2Cx9rCkbUQ.png" /><figcaption><strong>Figure 10:</strong> Features and target label (Stay) correlation coefficients.</figcaption></figure><p>It is tricky to use correlation coefficient to determine the strength of relationship between a feature and the target because the correlation coefficient only indicates linear relationship. A feature (e.g., <em>patientid</em>) with very low correlation coefficient does not necessary indicate that there is no relationship between the feature and the target. For example, the patient id has a very small correlation coefficient, but its feature importance is high.</p><p>I tried to remove some of the features with low feature importance (e.g., Hospital_region_code, City_Code_Hospital, etc.) and features with very small correlation coefficients (e.g., City_Code_Hospital) and observed that it didn’t help to increase accuracy score.</p><h3>5. Deployment</h3><p>Once the best model has been identified for deployment in model evaluation, we can move to the final step to deploy the identified model into a production environment. One common deployment method is to deploy the model as a Web service on a server, which can be called by other components in a target production system to predict results.</p><p>In order to support deployment, both the selected trained model (e.g., <em>XGBoost</em>) and the related encoding objects (e.g., <em>LabelEncoder </em>object, <em>TargetEncoder</em> objects) are saved into Python pickle files after model training. These saved encoding objects and model are to be loaded back for prediction in deployment.</p><p>As an example, the following code loads in thetest_data.csv for prediction in deployment. Figure 11 shows the transposed view of the loaded test dataset. Note that this dataset does not have the label column <em>Stay</em>.</p><pre>test_data = load_data(&#39;test_data.csv&#39;)<br>print(test_data.shape)<br>test_data.transpose().head(100)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qtIZiR86tJcvbvPJvUSs3A.png" /><figcaption><strong>Figure 11:</strong> Test data before prediction.</figcaption></figure><p>The code below does the following:</p><ul><li>loading the saved model and encoding objects</li><li>using them to predict the number of hospitalized days for each of the patients in the test dataset</li><li>showing the first 100 records in the resulting DataFrame with the features and predicted labels (the <em>Stay</em> column) (see Figure 12)</li><li>displaying the distribution of the predicted labels.</li></ul><p>We can see from Figure 4 and Figure 13 that the distribution pattern of the predicted labels in Figure 13 is very similar to the distribution pattern of the labels Figure 4.</p><pre>label_encoder, target_encoders = load_encoders()<br>result_df = predict(label_encoder, target_encoders, test_data_file=&#39;test_data.csv&#39;)<br>result_df[&#39;Stay&#39;].value_counts().plot(kind=&#39;bar&#39;)<br>result_df.transpose().head(100)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mPI_IjMYPyI0jUb9fcKrYw.png" /><figcaption><strong>Figure 12:</strong> Test data with predicted labels.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*2DTLV9o71c2HDB1tkUDupQ.png" /><figcaption><strong>Figure 13:</strong> Distribution of predicted labels.</figcaption></figure><h3>Improvement</h3><p>As shown in Section 4.1, the selected best model XGBoost achieved only about 42.4% in accuracy and a F1 score of 0.39. One of the major issues is that the data is significantly skewed as shown in Figure 4a and 4b. One possible improvement is to collect more data when possible and/or use <a href="https://en.wikipedia.org/wiki/Data_augmentation">data argumentation</a> techniques to generate more data samples to balance the dataset.</p><h3>Conclusion</h3><p>This article presented how to use different machine learning models to predict the hospitalized time for Covid-19 patients using a challenging dataset from Kaggle [1]. This dataset is difficult in that it is a multi-classes single label case and the dataset is significantly skewed.</p><p>The experimental results show that using <em>XGBoost </em>with target encoding and features engineering (e.g., transformation of the Age feature) achieved the best performance in prediction accuracy (42.4%) and F1 score of 0.39. This result is competitive to the results described in Kaggle [1].</p><h3>References</h3><ol><li><a href="https://www.kaggle.com/nehaprabhavalkar/av-healthcare-analytics-ii">Covid-19 dataset in Kaggle</a></li><li><a href="https://xgboost.readthedocs.io/en/latest/">XGBoost</a></li><li><a href="https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html">Random Forest Classifier</a></li><li><a href="https://keras.io/guides/sequential_model/">Keras sequential model</a></li><li><a href="https://mlflow.org/docs/latest/index.html">MLflow</a></li><li><a href="https://github.com/yuhuang3/machine-learning/tree/master/Udacity/capstone_project">Source code in Github</a></li><li><a href="https://contrib.scikit-learn.org/category_encoders/targetencoder.html">Target encoder</a></li><li><a href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html">Label encoder</a></li></ol><p>ACKNOWLEDGEMENT: I would like to thank Udacity reviewer for instructive comments and Kaggle for the dataset.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4e70456db9b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/predicting-hospitalized-time-of-covid-19-patients-f4e70456db9b">Predicting Hospitalized Time of Covid-19 Patients</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Explainable Deep Learning in Breast Cancer Prediction]]></title>
            <link>https://medium.com/data-science/explainable-deep-learning-in-breast-cancer-prediction-ae36c638d2a4?source=rss-759013c23ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/ae36c638d2a4</guid>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[cnn]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[explainable-ai]]></category>
            <dc:creator><![CDATA[Yu Huang, M.D., M.S. in CS]]></dc:creator>
            <pubDate>Sun, 02 Feb 2020 22:59:20 GMT</pubDate>
            <atom:updated>2020-02-06T02:52:59.073Z</atom:updated>
            <content:encoded><![CDATA[<h4>Understanding Convolutional Neural Network Prediction Results in Healthcare</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7dydKmnlxYYzQf9o6f-jVQ.jpeg" /></figure><p>Advanced machine learning models (e.g., Random Forest, deep learning models, etc.) are generally considered not explainable [1][2]. As described in [1][2][3][4], those models largely remain black boxes, and understanding the reasons behind their prediction results for healthcare is very important in assessing trust if a doctor plans to take actions to treat a disease (e.g., cancer) based on a prediction result. In [2], I used the Wisconsin Breast Cancer Diagnosis (WBCD) tabular dataset to present how to use the Local Interpretable Model-agnostic Explanations (LIME) method to explain the prediction results of a Random Forest model in breast cancer diagnosis.</p><p>In this article, I use the Kaggle Breast Cancer Histology Images (BCHI) dataset [5] to demonstrate how to use LIME to explain the image prediction results of a 2D <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network">Convolutional Neural Network (ConvNet)</a> for the Invasive Ductal Carcinoma (IDC) breast cancer diagnosis.</p><h3>1. Preparing Breast Cancer Histology Images Dataset</h3><p>The BCHI dataset [5] can be downloaded from <a href="https://www.kaggle.com/kernels/scriptcontent/2064134/data">Kaggle</a>. As described in [5], the dataset consists of 5,547 50x50 pixel RGB digital images of H&amp;E-stained breast histopathology samples. These images are labeled as either IDC or non-IDC. There are 2,788 IDC images and 2,759 non-IDC images. Those images have already been transformed into Numpy arrays and stored in the file <em>X.npy</em>. Similarly the corresponding labels are stored in the file <em>Y.npy</em> in Numpy array format.</p><h4>1.1 Loading Data</h4><p>Once the <em>X.npy</em> and <em>Y.npy</em> files have been downloaded into a local computer, they can be loaded into memory as Numpy arrays as follows:</p><pre>X = np.load(&#39;./data/X.npy&#39;) # images<br>Y = np.load(&#39;./data/Y.npy&#39;) # labels (0 = Non IDC, 1 = IDC)</pre><p>The following are two of the data samples, the image on the left is labeled as 0 (non-IDC) and the image on the right is labeled as 1 (IDC).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*DcT4_oxQ-kmIJKkX3f3KCA.png" /><figcaption><strong>Figure 1. Two samples: The left one is labeled as 0 (non-IDC) and the right one is labeled as 1 (IDC).</strong></figcaption></figure><h4>1.2 Shuffling Data</h4><p>In the original dataset files, all the data samples labeled as 0 (non-IDC) are put before the data samples labeled as 1 (IDC). To avoid artificial data patterns, the dataset is randomly shuffled as follows:</p><pre>indices = np.arange(Y.shape[0])<br>np.random.shuffle(indices)<br>indices = list(indices)<br>X = X[indices]<br>Y = Y[indices]</pre><h4>1.3 Transforming Dataset</h4><p>The pixel value in an IDC image is in the range of [0, 255], while a typical deep learning model works the best when the value of input data is in the range of [0, 1] or [-1, 1]. The class <em>Scale</em> below is to transform the pixel value of IDC images into the range of [0, 1].</p><pre>class Scale(BaseEstimator, TransformerMixin):<br>    def __init__(self):<br>        pass <br>        <br>    def fit(self, X, y):<br>        return self<br>    <br>    def transform(self, X): <br>        X1 = X.copy()<br>        X1 = X1 / 255.0<br>        return X1</pre><h4>1.4 Dividing Dataset for Model Training and Testing</h4><p>The dataset is divided into three parts, 80% for model training and validation (1,000 for validation and the rest of 80% for training) , and 20% for model testing.</p><pre>X_train_raw, X_test_raw, y_train_raw, y_test_raw = train_test_split(X, Y, test_size=0.2)</pre><pre>X_train = X_train_raw.copy()<br>X_val   = X_train[:1000]<br>X_train = X_train[1000:]<br>X_test  = X_test_raw.copy()</pre><pre>y_train = y_train_raw.copy()<br>y_val   = y_train[:1000]<br>y_train = y_train[1000:]<br>y_test  = y_test_raw.copy()</pre><h3>2. Training 2D ConvNet Model</h3><p>The BCHI dataset [5] consists of images and thus a 2D ConvNet model is selected for IDC prediction.</p><h4>2.1 Creating 2D ConvNet</h4><p>Similarly to [5], the function <em>getKerasCNNModel</em>() below creates a 2D ConvNet for the IDC image classification.</p><pre>def getKerasCNNModel():<br>    batch_size = BATCH_SIZE<br>    epochs = EPOCH_SIZE  <br>    img_rows, img_cols = X_train.shape[1], X_train.shape[2]<br>    input_shape = (img_rows, img_cols, 3) <br>    model = Sequential()<br>    model.add(Conv2D(16, kernel_size=(3,3), activation=&#39;relu&#39;, input_shape=input_shape))<br>    model.add(MaxPooling2D(pool_size=(2, 2)))  <br>    model.add(Dropout(0.25)) <br>    model.add(Conv2D(32, (3,3),  activation=&#39;relu&#39;)) <br>    model.add(MaxPooling2D(pool_size=(2, 2)))  <br>    model.add(Dropout(0.25)) <br>    model.add(Flatten())<br>    model.add(Dense(128, activation=&#39;relu&#39;))  <br>    model.add(Dropout(0.5)) <br>    model.add(Dense(1, activation=&#39;sigmoid&#39;))<br>    <br>    model.compile(loss= keras.losses.binary_crossentropy, <br>                  optimizer=keras.optimizers.rmsprop(), <br>                  metrics=[&#39;accuracy&#39;])<br>    <br>    return model</pre><h4>2.2 Creating Pipeline Component</h4><p>The class <em>KerasCNN</em> is to wrapper the 2D ConvNet model as a sklearn pipeline component so that it can be combined with other data preprocessing components such as <em>Scale</em> into a pipeline.</p><pre>class KerasCNN(BaseEstimator, TransformerMixin):<br>    def __init__(self, X_val=None, y_val=None):<br>        self._model      = getKerasCNNModel()<br>        self._batch_size = BATCH_SIZE<br>        self._epochs     = EPOCH_SIZE<br>        self._X_val      = X_val / 255.0<br>        self._y_val      = y_val<br>    <br>    def fit(self, X, y):  <br>        self.history = self._model.fit(X, y,<br>                        batch_size=self._batch_size,<br>                        verbose=1,<br>                        epochs=self._epochs,<br>                        validation_data=(self._X_val, self._y_val))<br>        return self<br>    <br>    def transform(self, X): <br>        return X</pre><pre>    def predict_proba(self, X):<br>        y_pred = self._model.predict(X) <br>        return y_pred  <br>    <br>    def evaluate(self, X, y):<br>        return self._model.evaluate(X,y)</pre><h3>3. Explaining Model Prediction Results</h3><p>As described before, I use LIME to explain the ConvNet model prediction results in this article.</p><h4>3.1 Setting Up a Pipeline</h4><p>Similarly to [1][2], I make a pipeline to wrap the ConvNet model for the integration with LIME API.</p><pre>from sklearn.pipeline import Pipeline</pre><pre>simple_cnn_pipeline = Pipeline([<br>    (&#39;scale&#39;, Scale()),<br>    (&#39;CNN&#39;, KerasCNN(X_val=X_val, y_val=y_val))<br>    ])</pre><h4>3.2 Training the ConvNet Model</h4><p>The ConvNet model is trained as follows so that it can be called by LIME for model prediction later on.</p><pre>simple_cnn_pipeline.fit(X_train, y_train)</pre><h4>3.3 Selecting LIME Explainer</h4><p>As described in [1][2], the LIME method supports different types of machine learning model explainers for different types of datasets such as image, text, tabular data, etc. The LIME image explainer is selected in this article because the dataset consists of images.</p><p>The 2D image segmentation algorithm <a href="https://scikit-image.org/docs/stable/auto_examples/segmentation/plot_segmentations.html"><em>Quickshift</em></a><em> </em>is used for generating LIME super pixels (i.e., segments) [1].</p><pre>from lime import lime_image<br>from lime.wrappers.scikit_image import SegmentationAlgorithm</pre><pre>explainer = lime_image.LimeImageExplainer() </pre><pre>segmenter = SegmentationAlgorithm(‘quickshift’, kernel_size=1, max_dist=200, ratio=0.2)</pre><h4>3.4 Explaining Model Prediction</h4><p>Once the ConvNet model has been trained, given a new IDC image, the <em>explain_instance</em>() method of the LIME image explainer can be called to generate an explanation of the model prediction.</p><p>An explanation of an image prediction consists of a template image and a corresponding mask image. These images can be used to explain a ConvNet model prediction result in different ways.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*oK1Y23ndoEhxE9b8SB9SVQ.png" /><figcaption><strong>Figure 2. The predictions for above two samples are to be explained. The left image is predicted as negative (IDC: 0) and the right image is predicted as positive (IDC: 0) by the ConvNet model.</strong></figcaption></figure><p><strong>Explanation 1: Prediction of Positive IDC (IDC: 1)</strong></p><p>Figure 3 shows a positive IDC image for explaining model prediction via LIME.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*v0HjfDDOmmAuJuWwstXUQw.png" /><figcaption><strong>Figure 3. IDC_1_sample: Prediction of a positive IDC sample to be explained</strong></figcaption></figure><p>The code below is to generate an explanation object <em>explanation_1</em> of the model prediction for the image <em>IDC_1_sample (IDC: 1) in Figure 3.</em></p><p>In this explanation, white color is used to indicate the portion of image that supports the model prediction (<em>IDC: 1)</em>.</p><pre>explanation_1 = explainer.explain_instance(IDC_1_sample, <br>                classifier_fn = simple_cnn_pipeline.predict_proba, <br>                top_labels=2, <br>                hide_color=0, <br>                num_samples=10000,<br>                segmentation_fn=segmenter)</pre><p>Once the explanation of the model prediction is obtained, its method <em>get_image_and_mask</em>() can be called to obtain the template image and the corresponding mask image (super pixels):</p><pre>from skimage.segmentation import mark_boundaries</pre><pre>temp, mask = explanation_1.get_image_and_mask(explanation_1.top_labels[0], <br>                                            positive_only=True, <br>                                            num_features=20, <br>                                            hide_rest=True)<br>plt.imshow(mark_boundaries(temp, mask))</pre><p>Figure 4 shows the hidden portion of given IDC image in gray color. The white portion of the image indicates the area of the given IDC image that supports the model prediction of positive IDC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/1*DwFOqSTnEq1iq_ajLdODeQ.png" /><figcaption><strong>Figure 4: Explanation of model prediction of a positive IDC in Figure 3 by hiding original image details. White color indicates the area that supports the model prediction. Gray part does not support or is irrelevant to the model prediction.</strong></figcaption></figure><p>The code below is to show the boundary of the area of the IDC image in yellow that supports the model prediction of positive IDC (see Figure 5).</p><pre>temp, mask = explanation_1.get_image_and_mask(explanation_1.top_labels[0], <br>                                            positive_only=True, <br>                                            num_features=20, <br>                                            hide_rest=False)<br>plt.imshow(mark_boundaries(temp, mask))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*gDfG1dBAC7V1WauAhUgRxw.png" /><figcaption><strong>Figure 5: Explanation of model prediction of a positive IDC in Figure 3 with original image details. Yellow color indicates the boundary of the white area in Figure 4 that supports the model prediction. The gray area is either not support or is not relevant to the prediction.</strong></figcaption></figure><p><strong>Explanation 2: Prediction of non-IDC (IDC: 0)</strong></p><p>Figure 6 shows a non-IDC image for explaining model prediction via LIME.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/1*0SFXCW4uctuo8-2te-uhcw.png" /><figcaption><strong>Figure 6. IDC_0_sample: Prediction of a negative IDC sample to be explained</strong></figcaption></figure><p>The code below is to generate an explanation object <em>explanation_2</em> of the model prediction for the image <em>IDC_0_sample in Figure 6. </em>In this explanation, white color is used to indicate the portion of image that supports the model prediction of non-IDC.</p><pre>explanation_2 = explainer.explain_instance(IDC_0_sample, <br>                                         classifier_fn = simple_cnn_pipeline.predict_proba, <br>                                         top_labels=2, <br>                                         hide_color=0, <br>                                         num_samples=10000,<br>                                         segmentation_fn=segmenter<br>                                        )</pre><p>Once the explanation of the model prediction is obtained, its method <em>get_image_and_mask</em>() can be called to obtain the template image and the corresponding mask image (super pixels):</p><pre>temp, mask = explanation_2.get_image_and_mask(explanation_2.top_labels[0], <br>                                            positive_only=True, <br>                                            num_features=20, <br>                                            hide_rest=True)<br>plt.imshow(mark_boundaries(temp, mask))</pre><p>Figure 7 shows the hidden area of the non-IDC image in gray. The white portion of the image indicates the area of the given non-IDC image that supports the model prediction of non-IDC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*Bd2TWtdoTA2LMCJqkox9pQ.png" /><figcaption><strong>Figure 7: Explanation of model prediction of a negative IDC (IDC: 0) of Figure 6 by hiding original image details. White color indicates the area that supports the model prediction. Gray part does not support or is irrelevant to the model prediction.</strong></figcaption></figure><p>The code below is to show the boundary of the area of the IDC image in yellow that supports the model prediction of non-IDC (see Figure 8).</p><pre>temp, mask = explanation_2.get_image_and_mask(explanation_2.top_labels[0], <br>                                            positive_only=True, <br>                                            num_features=20, <br>                                            hide_rest=False)<br>plt.imshow(mark_boundaries(temp, mask))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/1*7u35N94oA_6CUb8uhFo3Qg.png" /><figcaption><strong>Figure 8. Explanation of model prediction of non-IDC of Figure 6 (IDC: 0) with original image details. Yellow color indicates the boundary of the white area in Figure 7 that supports the model prediction. The gray area is either not support or is not relevant to the prediction.</strong></figcaption></figure><h3>Conclusion</h3><p>In this article, I used the Kaggle BCHI dataset [5] to show how to use the LIME image explainer [3] to explain the IDC image prediction results of a 2D <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network">ConvNet</a> model in IDC breast cancer diagnosis. Explanations of model prediction of both IDC and non-IDC were provided by setting the number of super-pixels/features (i.e., the num_features parameter in the method <em>get_image_and_mask</em>()) to 20.</p><p>I observed that the explanation results are sensitive to the choice of the number of super pixels/features. Domain knowledge is required to adjust this parameter to achieve appropriate model prediction explanation. Quality of the input data (images in this case) is also very important for a reasonable result. Accuracy can be improved by adding more samples.</p><p>A Jupyter notebook with all the source code used in this article is available in Github [6].</p><h3>References</h3><p>[1] M. T. Ribeiro, S. Singh, and C. Guestrin, <a href="https://arxiv.org/pdf/1602.04938.pdf">“Why Should I Trust You?” Explaining the Predictions of Any Classifier</a></p><p>[2] Y. Huang, <a href="https://towardsdatascience.com/explainable-machine-learning-for-healthcare-7e408f8e5130">Explainable Machine Learning for Healthcare</a></p><p>[3] <a href="https://github.com/marcotcr/lime/blob/master/doc/notebooks/Tutorial%20-%20Image%20Classification%20Keras.ipynb">LIME tutorial on image classification</a></p><p>[4] <a href="https://christophm.github.io/interpretable-ml-book/">Interpretable Machine Learning, A Guide for Making Black Box Models Explainable</a></p><p>[5] <a href="https://www.kaggle.com/paultimothymooney/predicting-idc-in-breast-cancer-histology-images/data">Predicting IDC in Breast Cancer Histology Images</a></p><p>[6] Y. Huang, <a href="https://github.com/yuhuang3/machine-learning/tree/master/lime/image_explainer">Jupyter notebook</a></p><p>DISCLOSURE STATEMENT: © 2020. Opinions expressed in this article are those of the author and do not necessarily represent those of Argonne National Laboratory.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae36c638d2a4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/explainable-deep-learning-in-breast-cancer-prediction-ae36c638d2a4">Explainable Deep Learning in Breast Cancer Prediction</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Explainable Machine Learning for Healthcare]]></title>
            <link>https://medium.com/data-science/explainable-machine-learning-for-healthcare-7e408f8e5130?source=rss-759013c23ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e408f8e5130</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[explainable-model]]></category>
            <category><![CDATA[lime]]></category>
            <dc:creator><![CDATA[Yu Huang, M.D., M.S. in CS]]></dc:creator>
            <pubDate>Thu, 26 Dec 2019 22:40:42 GMT</pubDate>
            <atom:updated>2020-02-02T22:29:51.849Z</atom:updated>
            <content:encoded><![CDATA[<h4>Understanding Machine Learning in Healthcare</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4HobEkYU9_ZRSMaKUchiwQ.jpeg" /></figure><p>Machine learning has widespread applications in healthcare such as medical diagnosis [1]. One of the frequently used datasets for cancer research is the Wisconsin Breast Cancer Diagnosis (WBCD) dataset [2]. Like in other domains, machine learning models used in healthcare still largely remain black boxes. As described in [3], understanding the reasons behind machine learning model predictions is very important in determining trust if a doctor plans to take cancer treatment action based on a prediction of diagnosis. Such understanding can also help doctors with domain expertise to find errors with machine learning model predictions. This can be achieved in different methods. One of the methods is called “explainable machine learning” [4].</p><p>In this article, I use the WBCD dataset [2] to demonstrate how to implement explainable machine learning to make an untrustworthy prediction trustworthy.</p><h3>1. Explainable Machine Learning</h3><p>As described in [4], explainable machine learning in this article refers to <a href="https://en.wikipedia.org/wiki/Post_hoc_analysis">post hoc</a> analysis and methods that are used to understand the predictions of a pre-trained model. There are different post hoc methods such as reason code generation, local and global visualizations of model predictions, etc. [4]. I use the reason code generation as the post hoc method in this article. In particular, the Local Interpretable Model-agnostic Explanations (LIME) [3] is used to explain which features are used by a machine learning model to make a prediction decision.</p><h3>2. Dataset Preparation</h3><p>As described before, the dataset used in this article is the public WBCD dataset [2]. In this dataset, the following 9 features were provided for each sample and will be used as inputs to the machine learning model:</p><ul><li>Clump Thickness</li><li>Uniformity of Cell Size</li><li>Uniformity of Cell Shape</li><li>Marginal Adhesion</li><li>Single Epithelial Cell Size</li><li>Bare Nuclei</li><li>Bland Chromatin</li><li>Normal Nucleoli</li><li>Mitoses</li></ul><h4>2.1 Loading Data</h4><p>Assuming that the WBCD dataset has already been downloaded onto a local computer as a <em>csv</em> file <em>breast-cancer-wisconsin.csv</em>, the dataset file can then be loaded into a Pandas DataFrame as follows:</p><pre>feature_names = [&quot;CT&quot;,&quot;UCSize&quot;,&quot;UCShape&quot;,&quot;MA&quot;,&quot;SECSize&quot;,&quot;BN&quot;,&quot;BC&quot;,&quot;NN&quot;,&quot;Mitoses&quot;]<br>columns = [&quot;ID&quot;]<br>columns.extend(feature_names)<br>columns.extend([&quot;Diagnosis&quot;])<br>raw_data = pd.read_csv(&#39;breast-cancer-wisconsin.csv&#39;, na_values=&#39;?&#39;, header=None, index_col=[&#39;ID&#39;], names = columns)<br>raw_data = raw_data.reset_index(drop=True)<br>raw_data.head(10)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ce5FknPTQpnZxGTpmSQjLw.png" /></figure><h4>2.2 Preprocessing Data</h4><p>The WBCD dataset shares many common issues and thus needs preprocessing to resolve those issues before it can be consumed by a machine learning model.</p><ul><li><strong>Missing Data</strong></li></ul><p>The following command is to check which features have missing data:</p><pre>data = raw_data.copy()<br>data.isnull().sum()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*KWkJ7YPoIRFQCATfugG6Gw.png" /></figure><p>As shown in the table above, there are 16 Bare Nuclei missing data entries. These missing data entries are simply replaced with 0 in this article since the total number of missing entries is relatively small.</p><ul><li><strong>Small Size of Dataset</strong></li></ul><p>The WBCD dataset [2] includes only 699 samples, which is too small for machine learning. This small data size issue can be alleviated by data augmentation (see Unbalanced Dataset for details).</p><ul><li><strong>Different Scales of Feature Values</strong></li></ul><p>The value ranges of different features are different. The class <em>Scale</em> below is to transform the feature values into the same range of [0, 1]. This class is designed in a way that it can be used with other classes to form a pipeline later on.</p><pre>from sklearn.preprocessing import MinMaxScaler<br>from sklearn.base import BaseEstimator, TransformerMixin</pre><pre>minMaxScaler = MinMaxScaler()<br>minMaxScaler.fit(features[feature_names])</pre><pre>class Scale(BaseEstimator, TransformerMixin):<br>    def __init__(self):<br>        pass</pre><pre>    def fit(self, X, y=None):<br>        return self<br>    <br>    def transform(self, X, y=None):  <br>        if type(X) == pd.Series or type(X) == pd.DataFrame:<br>            X1 = X.values <br>            X1 = X1.reshape(1, -1)<br>        else:<br>            X1 = X.copy()<br>         <br>        X1 = minMaxScaler.transform(X1) <br>            <br>        return X1</pre><ul><li><strong>Unbalanced Dataset</strong></li></ul><p>The following code is to check whether or not the dataset is balanced in terms of the labeled values of diagnosis:</p><pre>data[&#39;Diagnosis&#39;].value_counts()</pre><p>The numbers of data samples with different labels are not balanced. There are 458 data samples labeled as 2 (i.e., B (Benign)), but there are only 241 data samples labeled as 4 (i.e., M (Malignant)).</p><p>In order to balance the dataset, for simplicity a copy of the 241 data samples labeled as M is added into the original dataset.</p><ul><li><strong>Skewed Dataset</strong></li></ul><p>The following code can be used to visualize the distribution of feature values:</p><pre>data[numerical].hist(bins=20, figsize=(15, 10))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZPOCiNY7yNzUPtnjKHPktg.png" /></figure><p>It can be seen that the values of most features are significantly skewed to the right. The class <em>Sqrt</em> below is to alleviate the data skewness issue by taking squared root of the feature values. Like the <em>Scale</em> class, this class is designed as a pipeline component.</p><pre>class Sqrt(BaseEstimator, TransformerMixin):<br>    def __init__(self):<br>        pass</pre><pre>    def fit(self, X, y=None):<br>        return self<br>    <br>    def transform(self, X, y=None):  <br>        if type(X) == pd.Series or type(X) == pd.DataFrame:<br>            X1 = X.copy()<br>            <br>            if X1.isnull().values.any():<br>                X1 = X1.fillna(0)<br>                <br>            X1 = X1.abs() # avoid negative values<br>        else:<br>            X1 = X.copy()<br>            X1 = np.absolute(X1) # avoid negative values<br>        <br>        X1 = np.sqrt(X1) <br>            <br>        return X1</pre><h4>2.3 Separating Labels from Features</h4><p>The labels (i.e., diagnosis) are separated from features since labels and features are required for model training and testing.</p><pre>labels   = data[&#39;Diagnosis&#39;]<br>features = data.drop([&#39;Diagnosis&#39;], axis = 1)</pre><h4>2.4 Splitting Dataset into Training and Testing Datasets</h4><p>The features and labels are randomly separated into two parts, 75% for model training and 25% for model testing:</p><pre>from sklearn.model_selection import train_test_split</pre><pre>X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size=0.25, random_state=42)</pre><h3>3. Machine Learning Model</h3><p>The Random Forest Classifier with default settings is used for demonstration purpose. The model training and testing can be performed as follows:</p><pre>from sklearn.ensemble import RandomForestClassifier</pre><pre>rfc = RandomForestClassifier(n_estimators=100)<br>rfc.fit(X_train, y_train)<br>score = rfc.score(X_test, y_test)</pre><p>The trained model achieved 98.23% in accuracy.</p><h3>4. LIME Method for Explaining Predictions</h3><p>As described in [3], determining trust in individual predictions is important in decision making by machine learning models. For example, when machine learning is used for medical diagnosis such as the Wisconsin Breast Cancer Diagnosis [1], predictions cannot simply be used for actions as the consequences may be catastrophic. This section shows how to use LIME [3] to provide explanations for individual predictions as a solution to determining trust.</p><h4>4.1 Creating Pipeline</h4><p>The key point behind reason code generation in LIME is to associate a model prediction result with the original non-transformed feature values. To this end, the LIME method takes a machine learning pipeline (including a data preprocessing pipeline at the beginning and a machine learning model at the end of the pipeline) as input in explaining model prediction results. The following pipeline is to serve this purpose:</p><pre>from sklearn.pipeline import make_pipeline</pre><pre>scale = Scale()<br>sqrt  = Sqrt()<br># rfc is a pre-trained Random Forest model</pre><pre>machine_learning_pipeline = make_pipeline(scale, sqrt, rfc)</pre><h4>4.2 Selecting LIME Explainer</h4><p>The LIME method supports different types of machine learning model explainers for different types of datasets such as image, text, tabular data, etc. The tabular explainer is selected in this article since the format of the WBCD dataset is tabular:</p><pre>from lime.lime_tabular import LimeTabularExplainer</pre><pre>class_names = [&#39;Benign&#39;, &#39;Malignant&#39;]<br>explainer = LimeTabularExplainer(feature_names=feature_names, <br>                                 class_names=class_names, <br>                                 training_data=X_train.values)</pre><h4>4.3 Explaining Model Prediction</h4><p>Once a machine learning model has been trained, given any vector of original feature values (i.e., feature vector), the following <em>explain</em> function can be used to explain the model prediction:</p><pre>def explain(feature_vector, label=1): # feature_vector - a Pandas Series of features<br>    exp = explainer.explain_instance(feature_vector, machine_learning_pipeline.predict_proba, num_features=9)<br>    <br>    fig = plot(exp, label)<br>    exp.show_in_notebook(show_table=True, show_all=False)</pre><p>The method <em>show_in_notebook</em>() of LIME explainer shows the associations between the prediction and corresponding original feature values in notebook format.</p><p>The <em>plot</em> function below is to create a bar chart to show the LIME model explanation results:</p><pre>def plot(exp, label=1):<br>        exp_list = exp.as_list()<br>        fig = plt.figure()<br>        vals = [x[1] for x in exp_list]<br>        names = [x[0] for x in exp_list]<br>        vals.reverse()<br>        names.reverse()<br>        colors = [&#39;green&#39; if x &lt;= 0 else &#39;red&#39; for x in vals]<br>        pos = np.arange(len(exp_list)) + .5<br>        plt.barh(pos, vals, align=&#39;center&#39;, color=colors)<br>        plt.yticks(pos, names)<br>        if exp.mode == &quot;classification&quot;:<br>            title = &#39;Local explanation for class {}&#39;.format(exp.class_names[label])<br>        else:<br>            title = &#39;Local explanation&#39;<br>        plt.title(title)<br>        <br>        return fig</pre><p><strong>Example 1: Explaining prediction of malignant</strong></p><p>The following is a sample (feature vector) of M (Malignant):</p><pre>sample_M = raw_data.loc[5, feature_names]</pre><p>The prediction result of this feature vector can be explained using LIME as follows:</p><pre>explain(m_feature_vector, machine_learning_pipeline, label=1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YVNpBAh3P6zAr-C-FVbvSA.png" /></figure><p>The prediction result of Malignant is shown at the top-left corner of the figure above. The top-right corner and bottom show which features are positively correlated to the prediction of malignant and which features are negatively correlated to the prediction of malignant. In particular the features in orange/red indicate positive correlation, while the features in blue/green represent negative correlation.</p><p><strong>Example 2: Explaining prediction of benign</strong></p><p>The following is a sample of B (Benign) feature vector:</p><pre>sample_B = raw_data.loc[1, feature_names]</pre><p>The prediction result of this feature vector can be explained by LIME as follows:</p><pre>explain(sample_B, machine_learning_pipeline, label=0)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kgKFThwtpFEP8f9Pze87QA.png" /></figure><p>The prediction result of Benign is shown at the top-left corner of the figure above. The top-right corner and bottom show which features are positively correlated to the prediction of malignant and which features are negatively correlated to the prediction of malignant. In particular the features in orange/red indicate positive correlation, while the features in blue/green represent negative correlation.</p><h3>5. Conclusion</h3><p>In this article I used the LIME method [3] and the WBCD dataset [2] to demonstrate how to explain the prediction results of machine learning model in breast cancer diagnosis.</p><p>As described at the beginning of this article, understanding the reasons behind the machine learning model predictions is very important in assessing trust if a doctor plans to take cancer treatment action based on a prediction of diagnosis. Such understanding can also help doctors with domain expertise to detect errors with machine learning model predictions so that the machine learning model in use can be further improved.</p><p>A Jupyter notebook with all of the source code in this article is available in Github [6].</p><h3>References</h3><p>[1] <a href="https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)">Wisconsin Breast Cancer Diagnostic Data Set</a></p><p>[2] <a href="https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/">WBCD dataset and description</a></p><p>[3] M. T. Ribeiro, S. Singh, and C. Guestrin, <a href="https://arxiv.org/pdf/1602.04938.pdf">“Why Should I Trust You?”<br>Explaining the Predictions of Any Classifier</a></p><p>[4] P. Hall and N. Gill, An Introduction to Machine Learning Interpretability, An Applied Perspective on Fairness, Accountability, Transparency, and Explainable AI, Second Edition, O’Reilly Media, Inc., 2019</p><p>[5] M. T. Ribeiro, S. Singh, and C. Guestrin, <a href="https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime">Local Interpretable Model-Agnostic Explanations (LIME): An Introduction</a></p><p>[6] Yu Huang, Jupyter notebook in <a href="https://github.com/yuhuang3/machine-learning/tree/master/lime/tabular_explainer">Github</a></p><p>DISCLOSURE STATEMENT: © 2020. Opinions expressed in this article are those of the author and do not necessarily represent those of Argonne National Laboratory.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e408f8e5130" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/explainable-machine-learning-for-healthcare-7e408f8e5130">Explainable Machine Learning for Healthcare</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>