Data Leakage and Evaluation Integrity
Data Leakage is the phenomena of Non-availability of data when intended during prediction or any other methods or pre-availability of data for prediction
Most common case happens when we apply standardisation on whole dataset without splitting it into training dataset, validation set and test set
Get Akshat Gupta’s stories in your inbox
Join Medium for free to get updates from this writer.
While doing standardisation we find mean and deviation of all input feature including non-divided dataset, leading the model to know some or full test set leading the model to learn and memorize the whole dataset and eventually letting our model prediction for validation and test set to be high, but in reality it just learnt the data it was never meant to see or know
x = x -mean/deviation
Data Leakage can happen through various other reasons, just one of which is Standardisation, else Data Splitting, randomness, model selection and corresponding it’s correct algorithm and methods selection, further Avoding and considering features that tell about future possibilities and results must also be avoided and reconsidered, correct data preprocessing and transformation is also essential to avoid Data Leakage
Data Leakage is one of the most important and critical phenomenas to be avoided in Machine Learning feild otherwise it can cause large havocs and disasters
