Building your first Machine Learning model

A step-by-step guide to build Machine Learning model from scratch

Are you a Machine Learning enthusiast and looking to start your journey with the help of Python ? Confused about how to start building your first ML model, reading through lots of blogs about different ML algorithms which explains every bit of models, their mathematical intuition but failed to show how exactly use it with datasets?

You may have came across lots of reading material, free online courses, videos which will give you lots of recipes and snippets, but you never get to see how this all fits together. I will help you, with this story here, to complete your first machine learning project using Python.

In this step-by-step tutorial you will get to know:

• How to load a dataset and understand it’s structure using statistical summaries and data visualization.
• How to create different machine learning models, pick the best and build confidence that the accuracy is reliable.

When we start working with data for machine learning problem, we call it a project. And irrespective of the type of problem (ie. linear, classification, clustering etc.), steps involved in ML project are:

1. Define a problem
2. Understand and prepare data
3. Choose and train the model
4. Evaluate your models performance
5. Improve your model
6. Predict results

Before starting make sure you have all the libraries required are installed. We will use following libraries :

• numpy
• matplotlib
• pandas
• seaborn
• sklearn

If you are using Jupyter Notebook (Anaconda 3), it includes everything we need.

As a beginner, when I started my journey in Machine Learning, I built first model on Iris flower dataset. It is classical example to understand classification model and has multi nominal classification. A small dataset with five columns (features) and 150 rows (instances) will not take much time in training a model and can run on virtually any machine. In this dataset, all of the numeric attributes are in the same units and the same scale, not requiring any special scaling or transforms to get started.

Let’s get started with your first machine learning project in Python.

1. Import Libraries

Let’s import all the modules, functions and objects we are going to use in this project.

It is good practice to import all we need first hand. Preprocessing libraries, ML Models and Performance Metrics are imported in the sequence we use them.

2. Import dataset

we will use seaborn method to load iris dataset. There are other ways available as well, but seaborn will give you a ready dataframe object, and I found this the easiest way to do it.

3. Data Description

Get Sachin Urunkar’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

It is good practice to check your data and peek at it. shape, columns, head() will give you idea of number of features and instances (columns,rows), what are the features and type of data you have in it.

info function will give all this information with additional information of null values present in data (there is no null instances in iris dataset) and describe function will summarize the statistical information about the numerical attributes. It includes count, min, mean, max, std (standard deviation) and some percentiles as well.

All the numerical attributes are in same scale and range. Hence, we need no scaling, standardization or normalization methods to be used here.

The only categorical feature we have is “species”, let’s see how many categories it has and number of instances of each category.

Setosa, Versicolor and Virginica are three categories and all of them have fifty instances each. This is an example of balanced data and thus no balancing techniques are required to be used.

4. Data Visualization

Visualization will extend the findings in last step and help us in finalizing the problem we need to solve. Two types of plots namely “Uni-variate” and “Bi-variate” are helpful in understanding each attribute and relationship between attributes respectively.

Box plot and histogram are used for uni-variate plots and for bi-variate analysis we use pair plot.

5. Evaluating Algorithms

In this steps, we need to prepare our data for model training and testing before building a model. First four numeric attributes will make our X part (ie. independent attributes or input) and categorical attribute (“species”) will form our Y (ie. dependent variable or output ). We will keep 20 % data for testing part and remaining 80% will be used for testing purpose.

We will use stratified K-fold cross validation to estimate model accuracy with K=10.This will split our dataset into 10 parts, train on 9 and test on 1 and repeat for all combinations of train-test splits. Stratified means that each fold or split of the dataset will aim to have the same distribution of example by class as exist in the whole training dataset.

To evaluate model, we will use metric of “Accuracy”.

Five different algorithms are tested above, namely, Logistic Regression (LR), K-Nearest Neighbors (KNN), Decision Tree (DT), Gaussian Naive Bayes (NB) and Support Vector Machines (SVM) with accuracy estimation for each model.

Note: Your results can be way different than this given the stochastic nature of algorithms or randomness of data in train test split.

6. Predictions

As SVM resulted the highest accuracy, we will use this as our final model and will check it for our validation dataset.

We can evaluate the predictions by comparing them to the expected results in the validation set, then calculate classification accuracy, as well as a confusion matrix and a classification report.

We can see the accuracy on the hold out dataset. The confusion matrix provides an indication of the errors made.

Finally, the classification report provides a breakdown of each class by precision, recall, f1-score and support showing results

Note: As the validation dataset was small, we can often see overfitting to the training set or a data leak issues. Both of these issues will result in an overly optimistic result.

Find the relevant notebook here

Summery

In this post, I tried to cover every step for your first machine learning project with Python. As the Iris dataset itself is clean and balanced, it does not require any preprocessing, and that one step can be the biggest work to do, if you came across some other messy dataset.

With real datasets, you may have to spend a lot of time understanding the data, cleaning and balancing it, scaling or transforming data and do feature selection / feature engineering, before actually deciding about the ML model to be fit.