Auto Data Science
What is it and how is it helping product managers to achieve their objectives faster?
Since its popularization in the mid 2010s, Data Science has been through many trends. Some with more solid technical bases than others. What is clear though, is that Data Science and AI are becoming increasingly used by professionals from all fields and with all levels of technical knowledge. Thus, the latest efforts into trying to democratize its use seem legitimate.
The Terminologies
The democratization of Data Science has come from multiple sides and with different names. Some of the most popular are:
- Self-service BI: the reportings and dashboards that once were done by data analysts can now be created easily by business analysts thanks to tools like QlikView or Tableau
- AutoML: the machine learning project’s pipeline (model selection, tuning, training and evaluation) is automated thanks to tools like H20, DataRobot or Microsoft Azure AutoML
- EverydayAI: this term is used with a wide range of meanings, from automation of tasks to automation of parts of the ML process
So where does AutoDS fit in this landscape?
AutoDS vs AutoML
AutoML has been listed by Forbes as one of the 5 biggest data science trends for 2022. As stated in the previous section, AutoML aims at automating the machine learning pipeline. This, in general, means to automate the model selection, the tuning of the model’s hyperparameters, the model training and the model evaluation. So AutoML purpose is to automate the building of a predictive machine learning model. The output of it is a set of predictions (numbers or categories).
Get Compact Data Science’s stories in your inbox
Join Medium for free to get updates from this writer.
This approach has two flaws when it comes to be used by business decision makers (business analysts, product managers, business executives, …):
- the input data entered into the model needs to be pre-processed. This includes a lot of tasks such as dealing with missing data, changing mismatching formats, removing non-informative variables,…
- the output is a file of numbers, not easy to be interpreted or to be used to generate business value
Regarding task number 1, a KDnuggets article from 2020 shows how previous manual data pre-processing by data scientists actually improves the results of AutoML. Regarding task number 2, there is still not a lot of material out there about how to solve this, although explainable ML has been getting an increasing interest lately.
AutoDS (standing for Automated Data Science) is aimed at solving those above two pain points.
In short, AutoDS is intended to automate the whole data science pipeline which includes data pre-processing, machine learning and interpretability of the results.
The industry is leaning towards building tools with the whole data science pipeline automated so they can be user-ready. What is meant by users is business users (business analysts, product managers, business executives, …), they are the ones with more knowledge in their industry, their processes and their products. And more importantly, they are the final decision makers. Those tools, however, won’t replace the job of data scientists; instead, they will allow them to accelerate their redundant work or to be used as a starting point for more in-depth analysis and modelizations.
Conclusion
As stated by Bernard Marr in his article about data democratization:
Data democratisation will be a game-changer for organisations that implement it properly with the right training and tools to allow their employees to quickly and easily extract powerful business meaning from the data.
Compact Data Science is one of the first tools to have achieved this complete automation of the data science pipeline. It allows business decision makers to get meaningful information from their data and gives them concrete business actions recommendations. All of this in just a few clicks and with no data science knowledge required.
