Least Square Classification

Good to know why we don’t do least square regression like technique for classification in most of the cases.

Three classes problem

Steps involved are :

• Firstly convert response variable into indicator variable with (no. of rows, K), where k is number of classes possible for y. any row * col takes value 1 if corresponding data row and column corresponds to the response class. otherwise it is a 0.
• Then simply apply least square formula . The criterion is a sum of square Euclidean distance of fitted vectors from their targets. A new observation is classified by computing its fitted vector and classifying to the closest target. OR we can simply find argmax(k) f(x), where f(x) = (1, x.T) w, and w = (X.T* X)^(-1) *(X.T) *y

Problems with using above techniques:

• It is very dynamic to presence of outliers in data, and decision boundary changes a lot with addition of points.
• It is associated with masking problem, which makes it un suitable for classification of more than three classes.

I really recommend reading through the code from scratch to understand the algorithm more closely.

Get Shaily jain’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Consider following if you find this informative.

See you again.