Python Flask Dashboards

One of the main problems I have with a lot of visualization and dashboarding platforms is their difficulty in meshing scripting, SQL and visuals into a single layer. Take the case, which is more common than people care to admit, where two or more distinct data sources have to be loaded and matched to generate a intermediate dataset, a calculation or scoring performed on this data, which in turn needs to be checked against a third dataset before being presented.

Typically the likes of Looker, QlikView or Tableau will insist on taking a clean, ordered, and well-structured dataset and without further ado, presenting the data using snazzy visuals to the end-user. Quite rightly too, it’s their job to make your crappy dataset look like some insightful, trending and business oriented narrative… not to pre-calculate stuff which should be done somewhere else. Unfortunately, there are many many legacy data warehouse systems out there, which take all sorts of inputs to generate reports, from flat files, to MS Access outputs to a hodge-podge of mid-1990’s enterprise RDBMs. The accessibility of these data sources can itself range from SFTP, to ODBC, to CSV or even good old fashioned emails… something which would see the visualization platform gurus recoiling in horrified mirth.

Previously, I had one such problem, how to automate a very disparate set of data sources into an automated, reliable and accessible report, which had been previously performed using more antiquated muscle-power, basically a step above someone feeding coal into a steam-powered calculator. Not only was a process difficult and somewhat error prone, it required all manner and type of data sources to be accessed, along with various on-the-spot scoring and validations, with everything rounded off with a good old fashioned matching against the previously generated output to search for duplicates. Plugging in an enterprise-grade visualization system into this wild ménage would be an exercise in supreme folly, there would be no single view of the entirety of the data, and no way of melding the various parts together into a usable entirety.

Get Shane Brennan’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The difficulties in visualising the various disparate data sources without having a single source of data made it almost common sense to create a framework to automate all the boring, pointless and buggy reporting into a single view. Enter Python Flask, because

• It’s Python and therefore works well, simply and effectively.
• Provides a nice API to automate tasks through a REST interface.
• It has libraries for data manipulation (numpy), database connectivity (pyodbc) and visualization (matplotlib).
• Can be setup and run with about 10 lines of code.

So what happened was a couple of hours no-holds barred coding session to get an MVP up and running on a virtual server, processing one task using a web-interface.

from jobs.FirstTask import FirstTask
from flask import render_template
from flask import Flask

app = Flask(__name__)

@app.route('/')
def index():
 return render_template('index.html', title='Home')


@app.route('/taskOne', methods=['GET','POST'])
def taskOne():
 task= TaskOne()
 rows = task.getRows()
 return render_template('list.html', title='Task Output', header='Task Output', rows=rows)


@app.route('/about', methods=['GET'])
def about():
 return render_template('about.html', title='About')

if __name__ == '__main__':
    app.run(host='0.0.0.0', debug=True)

There’s a few hidden bits and pieces here, but mostly this is all there is to Flask. Entering a particular URL will effectively route you to a particular piece of code. This in turn can use a HTML Form or whatever to route you to another piece of Python code, and et voilá, there you have a nice method of navigating between different pages and API calls. The render_template() code takes a pre-defined HTML page with some parameters and fields replaced by a meta-data tag, for example if you wanted to specify the HTML title value dynamically you’d have something like this in the HTML template code

<html>
  <title>{{title}}</title>

And then populate the title code using

@app.route('/about', methods=['GET'])
def about():
 return render_template('about.html', title='About')

This allow the blending of various types of code (HTML, PHP, Python, CSS) using a single point of contact, namely the Python Flask API code. This allows for the merging of webscrapers, flat files, database connections, uploaded JSON data to name but a few types. Since the output is effectively HTML you can then use whatever tools you wish to create an automated reporting dashboard. In this case, I used the Google Materialize CSS to provide a nice ‘bootstrap-style’ web front end.