Machine learning visualization: sidetable and bokeh
I find sidetable very convenient to work with in summarizing data frame information as a preliminary step in machine learning. I decided to experiment with bokeh following a recent short course. I used output from sidetable to generate a data frame on the count of missing observations for each feature. Then I used bokeh to visualize the data frame. The strength of bokeh is in interactive visualization. Otherwise for static plots seaborn is a wonderful option; I found some aspects of bokeh slightly laborious e.g. the process of creating a bar plot. That said, let's get into it.
Step 1: Import libraries
import sidetable as stb #DataFrame summaries
from bokeh.plotting import figure #importing the figure
from bokeh.io import output_notebook,show #to output plot in notebook,to show
from bokeh.palettes import Category20_14 #palette for categories
from bokeh.transform import factor_cmap #assign different colors to categories
from bokeh.models import ColumnDataSource #bokeh data object
Step 2: Create the data frame.
Get Kavengik’s stories in your inbox
Join Medium for free to get updates from this writer.
I loaded a csv file containing the data. Then I assigned the variable data to the file. Then I used sidetable stb.missing() syntax to get the count of missing observations per feature in the data frame.
#1.Load data
data = pd.read_csv('/content/gdrive/MyDrive/YOURFOLDER/FILENAME.csv')
#2.Use sidetable stb.missing() to get count of missing observations then assign
#output to a dataframe (data_missing)
data_missing = data.stb.missing()
#3.Create column known as features and assign index column values (
#name of the feature is the index column in the data_missing df)
data_missing['features']=data_missing.index
#4.Resetting index column
data_missing.reset_index(drop=True)Step 3: Creating the bar plot
#i.to output my plot in the notebook
output_notebook()
#ii.Pass data to bokeh data source object
source =ColumnDataSource(data_missing)
#iii.create a list containing all categories in the feature column
#note: if data is not grouped use groupby first
labels = data_missing['features'].values.tolist()
#iv.Create a figure
fig= figure(x_axis_label='features',
y_axis_label='percentage of missing observations',
title='Bar plot showing percentage of missing observations',
x_range= labels)
#v.Add bar plot
fig.vbar(x=labels, top=data_missing['percent'],width=0.9)
#vi.X_axis text orientation
fig.xaxis.major_label_orientation = 45
#vii.place title at center of plot
fig.title.align= 'center'
#viii.display plot
show(fig) 
Step 4: Assigning different colors to the bars representing each category.
To do this you have to go through an extra step of evoking factor_cmap to which you pass: the name of the categorical column (‘features’ in mine), palette (Category20; the ‘_14) is to specify14 colors from this palette to match each category in my feature column which had a total of 14 categories, factors= labels (the list containing category names). Also, this time in the vbar argument we assign the name of the categorical column in the data frame to x.
#i.to output my plot in the notebook
output_notebook()
#ii.Pass data to bokeh data source object
source=ColumnDataSource(data=data_missing)
#iii.create a list containing all categories in the feature column
#note: if data is not grouped use groupby first
labels = data_missing['features'].values.tolist()
#iv.Create a figure
fig= figure(x_axis_label='features',
y_axis_label='percentage of missing observations',
title='Bar plot showing percentage of missing observations',
x_range= labels)
#v.Add bar plot
fig.vbar(x='features', top='percent',source=source,width=0.9,legend='features',
fill_color=factor_cmap('features',palette=Category20_14,
factors=labels))
#vi.X-axis text orientation
fig.xaxis.major_label_orientation = 45
#vii.place title at center of plot and adjust font size
fig.title.align= 'center'
fig.title.text_font_size='20px'
#viii.Show plot
show(fig)
