Occam’s Razor | Thumb rule of Machine Learning
In this series of Machine Learning, we will focus on some basic and fundamentals of AI-ML. Let’s understand one of the most fundamental thumb rule in machine learning.
Machine learning is not just building models to fit the available data. The actual challenge is to identify patterns and explain the behavior on real unseen data.
A machine learning model can be a function, logical rule or an algorithm that takes a set of input, processes it and gives an output. It shouldn’t be too complex or too simple to make predictions and hence we need to make balance between the two to come up with a good model that is capable to make right predictions, even on unseen data. Here we need to define simplicity?. So simplicity varies with the type of model.
Get Mansi Singh’s stories in your inbox
Join Medium for free to get updates from this writer.
Occam’s Razor does not say that a model should be unjustly simplified until further simplification is impossible. It states that when faced with a tradeoff between a complex and a simple model, with all other things being roughly equal, choose the simpler one.
Why we need to build a simple model?
A complex model makes a lot of assumptions on unseen data. That’s how simple model differs from complex models, these are generic, requires few training datasets, more robust, though simple models may make more errors in training phase, but it outperforms complex model when it sees an unseen testing data, whereas a complex model might learn all the training dataset, giving high accuracy and error as 0, but might miserably fail on testing data.
Thank you for learning with me!
