Generalisation, Training-Validation & Test data. Machine Learning- Part 6
# Less complex & Generalisation in life all we need 😊 😊
If you have followed this last session-3, then this session make most sense for you. You will learn couple of new things that we have to follow while creating a model. We are talking about:
Generalisation: Your model should be generalise enough to do prediction for unknown data.
Training Data: This you need to understand to trained your model.
Test Data: Validate your model prediction capaability.
Validation Data: To avoid overfitting(will talk here) we need this.
Let’s understand in detail.
Generalisation: This is another one element of Machine Learning(#ML). It describe how well model can be perform on new data. Is your model generalise enough to do prediction for new unknown data !!
You build your model, train it using training data & validate it using test data & this way we make sure it will work for new unknown data set.
Cool !! How we do that !! How we make sure it is generalise enough for new data as well !! We optimise it & optimise it & optimise it & keep on check against our Test data. Yeeee…we got it ..let’s celebrate it.
Get Shubham Patni’s stories in your inbox
Join Medium for free to get updates from this writer.
But waittt, I have heard about a term “Overfitting” what’s that !! Hmm…well it’s about overfitting your model for given training & test data set.
Ohh…does that mean we optimised & build our model so perfect for given data set..niceee!! (pause) but how about new data set!! Is our model generalise enough for this new data set !!! Will the new data fall other than model line or under the line !!!
So you got it right! We have two data balancing problem Underfitting & Overfitting, we just wanted to build a generalise model with minimum loss.
Well !! I can fix it little. I can put some constraint like:
* I’ll fix my data source, no new source of information I’ll accept.
* I can do some data clipping, say data should fall between -5 to +5.
* Data will not change over the year.
BUT BUT this is not what Generalisation word speak about, your model should work with new data as well. Putting data clipping can be good idea but only if you know your data. To know your data is an very important element of ML.
Overfitting occurs when a model tries to fit the training data so closely that it does not generalize well to new data
Ok, it seems it is a continuous improvement process & that’s where type of machine learning speak about. You may require manual monitoring based on type & level of effort you have put to create your model.
Mr. William of Ockham from the 14th century talked about the same thing “Keep model simple as much you can”. Good theory !! Let’s see now how we can approach this.
We talked about empirical risk minimisation in last session. We already have seen how to take a draw from Training data called Test set & check validation. (Try exercise here.)
But is it enough !! Well when you worked with hyperparameter then to avoid overfitting issue we divide our training data into another one draw we called it Validation data set.
Here we perform all our training with Training set & use the validation set to evaluate results from it. All tweaking for hyperparameter we do by validating against validation set.
In this improved workflow:
1. Pick the model that does best on the validation set.
2. Keep Test set away until model is not done with rest of two set.
3. Double-check that model against the test set.
4. That mean, validate Validation set result with Test set result. If it’s matching enough that mean we are ready to launch else it’s again overfitting we did.
Couple of things to must consider while making draw of training, validation & test set:
# always randomise your data set before to take draw for Training & Test.
# never ever train your model with Test set, else you will get 100% accurate false result.
# you may need to change data ratio divided among these three set. Considering the logic that training set should be sufficient enough to train your model :)
That’s it for this session. In next session I’ll summarise these 6 session before to start Machine Learning with Tensor flow SDK.
Have any queries/anything to discuss, do here. This is how we can talk more about Machine Learning. See you in next 👋 👋 .
