_edited_edited.png)
Machine Learning has umpteen algorithms for prediction or forecasting. ML algorithms are tuned, so there is always minimal Bias and moderate Variance to get expected predictions. What are Bias and Variance?
Before learning about them, let us understand a few other terms.
1. Underfitting — The model did not learn the data patterns during training and does not predict precisely while testing or with new data.
2. Overfitting — The model fits all the data points, including the unwanted data called “noise”. On testing, it shows that the results are unsuitable for real-time predictions.
Understanding Bias and Variance
The meaning of bias and variance in Machine Learning ,is not very far from their meanings in the Oxford English dictionary. Let me help you here with some examples.
Bias - How much the prediction is away from the actual value?
Suppose, the above picture is used to train a model to classify a dog and a cat. When you give it a picture of a cat, the model will not make an accurate classification, as it was not taught anything about the cat.
Since the data is biased, the model would be too simple, thereby making biased predictions. An ideal model will be one that is less biased.
So what should the data be like to decrease the bias? How about adding some cat pictures to the data? Let us add a variety of cats to help our machine classify a cat from a dog, like a picture below.
Now we have increased the Variance. There is a possibility that the model accurately predicts the cat as this small data.
Now, how about data that has a billion pictures? The model will now have multiple parameters to learn from. Thus, the model becomes complex.
Now a new data is introduced, the model shows the difficulty in classifying. Why? Because the model learned to memorize the data rather than learning data patterns. It memorizes the data along with the random noise and fluctuations. It is overfitting the data.
Thus, a model with high variance is unsuitable for predictions.
Low bias or high variance: What to choose?
The picture below shows the various combinationd of bias and variance.The circle is the prediction area.
Out of all the predictions, the only picture that shows most of the values fitting right in our prediction is the one with Low bias and High variance. It is not a plausible solution. This is the Bias Variance tradeoff. It occurs in supervised learning algorithms.
For accurate prediction, reducing the variance without increasing the bias is crucial.
We can reduce the complexity of the model to reduce the variance. Other techniques like regularisation and early stopping help make a model with low bias and moderate variance.
Hope this blog helps in making the right decisions with the model.
Happy Machine Learning!