It sucks. And there should definitely be a better system, right? Having volunteered at Toronto Western Hospital Emergency Department and being able to experience both sides of the system, I wanted to dig deep at what’s really going on and to actually do something about it.

Technologies Used
* Python
*Pandas
*pipenv — for local storage of credentials
*Beautiful Soup — Web Scraping
*SKLearn pipelines
*DataFrameMapper
*HDBSCAN clustering
*Flask
*Google Maps API

### Introduction

PATIENTLY WAITING was created as my capstone project for the Data Science Immersive program at Bitmaker / General Assembly. A beta version of this application can be found here! Check emergency room wait times around Toronto by choosing a day: https://patientlywaiting.herokuapp.com/

Data Gathering

Detailed administrative data on date, patient flow, current and past examinations in Ontario was provided by:
* [National Ambulatory Care Reporting System (NACRS)](https://www.cihi.ca/en/national-ambulatory-care-reporting-system-metadata)

* [Canadian Institude for Health Information (CIHI)](https://www.cihi.ca/en/access-data-and-report)

Data Modeling

The ability to accurately and reliably predict waiting times at walk-in hospital facilities can increase both patient satisfaction and hospital efficiency via a better management of patient flow. My web-app implements machine learning (ML) models to predict waiting times in the Emergency Room (ER) of the largest public hospital in the Greater Toronto Area (GTA).

Several machine learning (ML) techniques were evaluated to find the most accurate and useful prediction to a user. I chose [Gradient Boosting](https://medium.com/mlreview/gradient-boosting-from-scratch-1e317ae4587d) among other regression models explored for predicting wait times.

*PATIENTLY WAITING** is currently in beta testing. If you notice any bugs, have any questions or suggestions, I’d love to hear from you: [anita.tran38@gmail.com](anita.tran38@gmail.com?subject=PatientlyWaiting).

https://media.makeameme.org/created/me-patiently-waiting-399b1150e6.jpg

Planned Future Enhancements

* Docker for hosting database, nginx and flask web app
* Google API to read your location
* Actively scrape hospital data as it is released per month on hqontatio.ca
* Generate best route to hospital using combinatorial optimization

If you’d like to play around with the tool, a beta version is running here: https://patientlywaiting.herokuapp.com/

The complete project source code (which is still in the process of being updated) can be found on [GitHub](https://github.com/anitatea/patiently_waiting).

```python

```

Data visualization is one of my favourite creative outlets — to draw out deeper insights from meaningful data.

Visualization packages in Python is far from scarce and there are great options for static graphs.

However, as data gets more complex and understanding becomes more distant, there is an obvious need for interactive visualizations to easily explore data. What I’m currently exploring is Plotly and my favourite part so far is the extensive documentation and highlighted projects made with Plotly.

What’s Plotly/Dash

Their mission is “Bringing data science out of the lab and into the business.”, which I translate to making visualization a lot easier.

Formally only available for enterprise, Dash is now an open source library (free!) for creating interactive web-based visualizations. Highly interactive web application using Python or R.

HTML/CSS/JavaScript not required (… and the door is that-a-way).

What I think makes it powerful is it is able to incorporate all the functionalities of Python for data manipulation and much more.

Amazon, Cisco, Shell, among other worldwide enterprises have been embracing this tool for large scale deployments.

Let’s Get Started

pip install dash

Sample Interactive Bar Chart

I used data from the Centers for Disease Control and Prevention on Drug Poisoning Mortality by State in the United States. I’m going to build an interactive bar chart that shows absolute deaths per state in 2016 and 2017. (The dataset has been updated on April 29, 2019 and so I believe this is the most accurate dataset to this date.)

Bring in all the dash modules and pandas for reading and manipulating the data. I saved this file as bar_app.py:

import dash
import dash_core_components as dcc
import dash_html_components as html
import plotly.graph_objs as go
import pandas as pd # to manipulate our data

For this simple example, I’m choosing to import the plotly.graph_objs whereas the documentation uses dcc.graph to build the plot.

Writing this article, I felt that it was easier to use the plotly graph object since there were a lot more examples online than there were the plain dcc.graph for the simple cases.

This is my preference and I do find it more straightforward to demonstrate this example.

Reading in the first few pieces of data and filtering for deaths by state in 2016 and 2017:

df = pd.read_csv('data/NCHS_-_Drug_Poisoning_Mortality_by_State__United_States.csv')

df.iloc[:,:7].head()

In our example, we’ll only be looking at the State, Year and Deaths columns.

df = df[['State', 'Year','Deaths']]

Taking a look at all the state names.

df['State'].unique()

Output:

array([‘Alabama’, ‘Alaska’, ‘Arizona’, ‘Arkansas’, ‘California’,
‘Colorado’, ‘Connecticut’, ‘Delaware’, ‘District of Columbia’,
‘Florida’, ‘Georgia’, ‘Hawaii’, ‘Idaho’, ‘Illinois’, ‘Indiana’,
‘Iowa’, ‘Kansas’, ‘Kentucky’, ‘Louisiana’, ‘Maine’, ‘Maryland’,
‘Massachusetts’, ‘Michigan’, ‘Minnesota’, ‘Mississippi’,
‘Missouri’, ‘Montana’, ‘Nebraska’, ‘Nevada’, ‘New Hampshire’,
‘New Jersey’, ‘New Mexico’, ‘New York’, ‘North Carolina’,
‘North Dakota’, ‘Ohio’, ‘Oklahoma’, ‘Oregon’, ‘Pennsylvania’,
‘Rhode Island’, ‘South Carolina’, ‘South Dakota’, ‘Tennessee’,
‘Texas’, ‘United States’, ‘Utah’, ‘Vermont’, ‘Virginia’,
‘Washington’, ‘West Virginia’, ‘Wisconsin’, ‘Wyoming’],
dtype=object)

“United States’” is obviously not a state. I suspect it captures deaths that are uncategorized.
To keep our visualization simple, we’ll drop these rows.

indexUS = df[df['State'] == 'United States'].index
df = df.drop(indexUS)

# filter for 2017
df_2016 = df[df['Year'] == 2016]

# filter for 2017
df_2017 = df[df['Year'] == 2017]

Now that the data is simply tabulated by state and total deaths; we can build out the bars in our graph.

The convention for plotly is that each item being plotted is usually called a trace. In this example, we want to plot State vs. Deaths.

trace1 = go.Bar(x=df_2016['State'], y=df_2016['Deaths'], name='2016',
        marker=dict(color='rgb(55, 83, 109)') # darker blue
        )
trace2 = go.Bar(x=df_2017['State'], y=df_2017['Deaths'], name='2017',
        marker=dict(color='rgb(26, 118, 255)') # lighter blue
        )

Now we’re ready to build the Dash app.

app = dash.Dash()

app.layout = html.Div(children=[
    html.H1(children='Drug Poisoning in the United States in 2016 & 2017'),
    html.Div(children='''(Hover over bars to view death count)'''),
    dcc.Graph(
        id='example-graph',
        figure={
            'data': [trace1, trace2],
            'layout':
            go.Layout(title='Drug Poisoning Mortality by State in 2016 & 2017', barmode='group')
        })
])

Most of the structure is dash building the app, using HTML components and creating the figure dictionary, containing instructions to build the graph and customize layout.

What I love is the simplicity of how dash integrates HTML components beautifully without having to deal with the standard HTML/CSS/JS boilerplate required for most modern web apps that you see.

Since I’m dealing with various code styles, Atom is my preferred editor to manage matching brackets, parentheses and such.

dcc.Graph defines the figure using a using a dictionary that contains the actual figure, plotted data and layout options. In this case, the layout was needed to define the title and bar type graph.

Once the app is laid out, we need to make sure it can run:

if __name__ == '__main__':
    app.run_server(debug=True)

Running this in your terminal:

python bar_app.py

Here’s our nice interactive graph!

Pretty great when you think how much engagement you can get with a simple application of 35 lines of code. You can find the above example available on github.

Not only is the layout and look of dash clean and report-ready; Dash also took care of scaling the interactivity on different browsers. So we can focus on beautiful, pure python!

The entire code:

import dash
import dash_core_components as dcc
import dash_html_components as html
import plotly.graph_objs as go
import pandas as pd

df = pd.read_csv('NCHS_-_Drug_Poisoning_Mortality_by_State__United_States.csv')

df = df[['State', 'Year','Deaths']]

indexUS = df[df['State'] == 'United States'].index
df = df.drop(indexUS)

df_2016 = df[df['Year'] == 2016]
df_2017 = df[df['Year'] == 2017]

trace1 = go.Bar(x=df_2016['State'], y=df_2016['Deaths'], name='2016',
        marker=dict(color='rgb(55, 83, 109)'))
trace2 = go.Bar(x=df_2017['State'], y=df_2017['Deaths'], name='2017',
        marker=dict(color='rgb(26, 118, 255)'))

app = dash.Dash()
app.layout = html.Div(children=[
    html.H1(children='Drug Poisoning in the United States in 2016 & 2017'),
    html.Div(children='''(Hover over bars to view death count)'''),
    dcc.Graph(
        id='example-graph',
        figure={
            'data': [trace1, trace2],
            'layout':
            go.Layout(title='Drug Poisoning Mortality by State in 2016 & 2017', barmode='group')
        })
])
if __name__ == '__main__':
    app.run_server(debug=True)

For Fun: A More Complex Example

It took some time and I focus more on aesthetics than I should but Plotly is pretty easy to pick up overall to put together any dashboard that’d impress your friends.

Dash’s real power shines in its ability to combine complex interactions — again all with Python. Dash provides several interactive components out of the box including Dropdowns, Multi-Select Dropdowns, Mapping, Radio Buttons, Checkboxes, Sliders, and Text Input. All of them can be easily constructed and tied into your plots to drive various interactive options.

Here’s the link to the dashboard. Layout inspired by examples from the Dash-Plotly gallery of even more amazing dashboards!

Final Thoughts

I’ve only scratched the surface of this powerful yet simplistic tool. My favourite pieces:

• Quick for building web-based visualization tools without prior knowledge of JavaScript and other web technologies
• Default dashboard is already responsive
• Ability to save, zoom, pan and interact with the display, on the display

Presenting data-driven visualizations to non-technical colleagues can be a struggle, as if you are speaking two different languages without Google Translate. I’ve seen more than my fair share of presentations and the truly successful ones were the most engaging with their audience.

Originally published at https://anitatran.ca on January 27, 2020.