For python2

we have raw_input() function it will take different values from the user and return the output in the form of a string irrespective of the input values.

For example, this problem in python2 gives string.

Whereas the input() function in python2 will return the same value type as the user entered.

For Python3

The raw_input() function is not available for python3. The input() function in python3 is similar to the raw_input() function in python2. It means whatever type of data you input in python3 it will give you string as an output.

For example:

Here, we can see that if we input integer values in python3 using the input function it will give a string as an output. To get an integer as an output we have to use the int() function with the input() function.

Similarly, we have to use different functions with input function for getting different outputs types like float, list, etc.

Python input() and raw_input() function. was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

The classic definition of machine learning says, how you learn from your experience and apply that learning for future prediction, with that being told it is a little confusing to understand unsupervised learning. So when you are dealing with unsupervised learning there is no Target variable. You can not calculate your model’s accuracy/performance, so how will you improve or how will you judge your learning.

So, when you start looking at the actual application of unsupervised learning you realize its true potential.

Imagine you are a part of a Data Science Team and your goal is to suggest a scheme or offer for a credit card holders for a bank, or suggest movies for an OTT platform subscriber in these type of problem where you are taking a new step for the first time, you don’t have any experience but just based on the Hidden Pattern in your Data you are about to take some action, this is what is the main power of Unsupervised learning is.

So Unsupervised learning is a Prerequisite in Supervised learning.

Types of Unsupervised Learning:

The most Unsupervised learning we see are-

· K means Clustering

· Hierarchical Clustering

· Recommendation Systems

Here we will briefly discuss Clustering techniques and the ideas behind them. More on the Recommendation system later.

K means:

The idea of K means is that, you cluster points based on the distance between them and then you can assume that points belonging to the same cluster will have similar characteristics. In easy words, you can say it is like grouping based on centroids of clusters. Now how those clusters are formed is a very interesting concept on its own.

K Means clustering works on the principle of Expectation-Maximization. It is a two-step process.

Imagine you have a huge distribution of Data Points. Our goal is to find k clusters in the distribution where each point of the cluster is closest to its cluster center than any other cluster’s center. This step can not be achieved at once(it is one of the reasons that Clustering takes time for a huge Dataset).

First, we will randomly choose k points in our distribution and then based on them assign each point to a particular cluster based on the distance between the points, this way we will form an initial cluster. Now when we have made an initial cluster, we will adjust the center of this cluster to the centroid position of the cluster, after this, we will again assign all the points based on this new cluster center.

This process will be repeated again and again, till we converge to a point where not much change happens in the clustering process. So in our process, the step where we assign the points to the cluster is called the Expectation step, and the step where we adjust the cluster center is called the Maximization step.

As we have seen that, the lack of pre-requisite is the main problem in unsupervised learning. The biggest challenge in K means clustering is of finding the optimal value of K . It can also be the case that the business expects us to do the clustering for a pre-decided number K, but most of the time it is up to us to decide the optimal value of K.

To do that we have two metrics to calculate,

WSS — Within sum of squares = Sum of the distance of each point from the center

Silhouette score- The idea is similar to that of WSS, the formula will be a little different.

Once we have calculated these values for different values of K, we can plot the value of WSS/Silhouette score we are getting for each, and the point where the value starts to converge can be taken as the right value of K, which then has to be verified by the business anyway.

For example, imagine a Credit Card company trying to divide its customers into segments, so if you find 3 or 4 as the optimal value of K, then they will be fine, but let’s say you found K=11 then that does not make much sense.

Another issue with K-means clustering is that people confuse it with the K nearest neighbors since both of them are classic machine learning techniques, the difference being that K means clustering in an Unsupervised learning Algorithm and K nearest Neighbor is a Supervised learning Algorithm.

Hierarchical:

This is the other most used way of clustering you have two options either you move from top to bottom, giving rise to Divise Hierarchical clustering or you move from bottom to top, giving rise to Agglomerative clustering.

Generally, we notice that Agglomerative Clustering is more used, as you can limit the grouping forming a given number of clusters.

So you start from a single data point and start grouping points that are closer to each other. So let’s say we are in the middle of the process, so you would have the chance to assign a particular point to any closer forming group and that distance between two groups or one group and one point is calculated by the idea of Linkages.

There can be many types of distances that we can have between two points in distribution, like Euclidean , Manhattan Distance, etc, but when we want to calculate the distance between two groups of data points it little tricky.

We may calculate the distance between the two closest points of two groups or two farthest points from both the groups, or we may take the distance from the centroid of one cluster to the centroid of the other, or you may take an average of the distance of all points in one group with all points in another group. All of these forms different kinds of Linkages.

But there is a more intuitive method, so for every group, we can calculate WSS, ie the sum of distances of all points from the center of that cluster.

So, now for G1, we calculate WSS1 and G2 we calculate WSS2 then we take G1 and G2 as a single group and calculate WSS12, now we find the value of

(WSS1+WSS2) — WSS12 = delta.

So now for 3 groups G1, G2, G3 we can use this delta value to find which groups are closer to each other. Closer groups will have a low value of delta.

This is called the Wards method of finding distances between two clusters of data points and is a backbone of what we know as Hierarchical Clustering. It may not tell the exact distance but will tell you which groups are closer.

What we get from Hierarchical clustering is a Dendrogram, and based on our understanding of how many clusters we want we can cut the dendrogram by a horizontal line, which gives us a clear view of the kind of clusters formed.

We saw , the two most classic clustering technique, more on similar topics later.

Hope , these clustering techniques are used with cautions and don't create a division based on some stereotype of the society.

Clustering — How we are “Closely Related”! was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

When it comes to evaluating your model’s performance there are different metrics that you can choose from. To choose the best evaluation metric to assess your model, it is necessary that you understand the advantages and disadvantages of each and every metric. As a Data Scientist, you might have come across statements like your model is extremely accurate but depending on the business problem your model is not performing up to the mark, in such cases maybe be you should go with a different metric.

Classification

Let us first look into the evaluation metrics used in classification models. Here we will not only get to know the metrics we will also see which ones are used in a specific business problem.

For a classification-based model, all the parameters are included inside a metric called Confusion Matrix.

A Confusion matrix is an M x M matrix used for assessing the performance of a classification model, where M is the number of predicted classes. The matrix compares the actual predicted values with those predicted by our classification model. This gives us a comprehensive view of how well our classification model is performing and what kind of metric is reliable.

Since we will be looking at binary classification, let us take a 2 x 2 matrix as shown below with 4 values:

Here ‘Actual Class’ is what actually happened and ‘Predicted Class’ is what my model predicted. Now let me explain the diagram with a realistic example. Let us imagine a classification model that predicts whether it will rain or not rain. ‘Rain’ is a positive class and ‘Not rain’ is a negative class. True positive (TP) is when my model predicts Rain and it actually rains. True Negative (TN) is when my model predicts Not Rain and it doesn't rain in real. Whereas False Positive (FP) is when my model predicts a positive class(Rain) but it turns out to be false and it doesn't rain. Similarly, False Negative (FN) is when my model predicts a negative class ( Not Rain) but it rains on that day.

Based on these four parameters we will define evaluation metrics like Accuracy, Recall, Precision, and F1_Score.

ACCURACY

In very simple terms accuracy is an evaluation metric that measures the total number of predictions that my model got right out of the total number of predictions it made. In terms of confusion matrix, the formula for accuracy is :

Accuracy takes into account both True positives and True negatives, many a time we might need any one of them, for which there are other metrics defined. Every term in the formula of Accuracy is present in the confusion matrix and we can calculate it accordingly. Many a time the model built is fairly accurate but it still can’t solve a business problem. Such cases arise when we have an imbalanced dataset i.e. a dataset that has a majority of observations in the positive class and very few observations in the negative class and vice versa.

For example, you build a model to predict if the email you got is spam or not. In this case, 1 maybe 2 emails out of 100 that you get, turn out to be spam. So if you build a model which shows all mails are legitimate you will still achieve an accuracy of 98% but it will neglect all the spam emails and the model does not work in any way. When accuracy is not a good metric to evaluate your model you need to look at other metrics.

PRECISION

In very simple terms precision measures how precisely a model predicts the positive labels. In other words, out of all the positive predictions how many were correctly predicted. The formula for precision is given below:

The numerator of this formulae essentially tells us the positive observations that my model predicts correctly. The denominator essentially shows all the points that my models predicted as positive.

RECALL

Also known as true positive rate(TPR) or Sensitivity, recall can be defined as out of all the actual positive values, how many of them were predicted correctly. The formula for the recall is given below :

The numerator is the same as precision. but the denominator has all the actual positive values irrespective of which class the model predicted it to.

Now let me give explain to you both precision and recall in realistic terms to get a better intuition. Imagine your friend gives you a present at every birthday for the past 10 years. In the 11th year she makes a bet, she says that I will give you your favorite PlayStation if you can correctly guess all the 10 presents that I gave you over the course of 10 years. Now you really want PlayStation so you try hard to recall each present one by one. Suppose you were able to recall 7 presents out of 10, therefore you have a recall of 7/10 i.e. 70 %. Now to further the bet she says okay you try to guess the other presents you can take infinite guesses. you try your best and were able to name all 10 presents in 20 tries. Though this makes your recall as 100% you were not precise enough, your Precision gets as low as 10/20 i.e. 50%. Hope this gives more clarity on both the terms.

How do you choose between precision and recall for your business problem?

The best explanation for this is to choose Precision when your False-positives cost you more than your False-negatives. For example imagine you sell gym products, your invest in a brand of protein supplements as suggested by your ML model. But it turns out that you got a false positive signal and your customers hate the supplement and you incur a huge loss. Any business problem which involves investing in your output prediction such as giving loans, customer churn analysis, other banking systems you use Precision.

Now imagine you have to deploy a model in the health sector, detecting cancer in patients. Here if you get any false negatives i.e. you get a prediction that the patient doesn't have cancer whereas, in reality, he has cancer, in this case, his life is at risk because of wrong predictions. So any business problem where a False-negative can cost you more than a False-positive you use Recall. Other places that you can use recall are credit card fraud prediction, all healthcare domains, etc.

What happens if your data has class imbalance but you cant choose between precision and recall as both hold equal importance. In such cases, you go for an amalgamation of both, there is a metric that combines both precision and recall and is named f1_score.

F1_score also known as F Measure is the harmonic mean of both precision and recall. F1 score can be calculated using the following formula.

REGRESSION

Evaluation metrics play a very significant role in building any machine learning model. The Mean squared error, root mean squared error, mean absolute error, R-Squared, and Adjusted R-squared are used to evaluate the performance of the model in regression algorithms.

Mean Squared Error(MSE) measures the arithmetic mean of the squared difference between the actual and predicted values in the dataset. It is a measure of the variance of residuals.

Mean absolute error(MAE) measures the arithmetic mean of the absolute difference between the actual and the predicted values in the dataset. It measures the average of residuals.

Root Mean Squared Error(RMSE) is nothing but the square root of the mean squared error that we calculated earlier. It measures the root of variance i.e. standard deviation of residuals.

R-squared measures the degree to which variance in the dependent variable can be explained by the independent variables(features). It will always range from 0 to 1, the higher the R-squared better is my model. It is a relative metric that can be used to compare when we apply more than one algorithm.

Adjusted R squared is a slight modification to the R-square value, this measures the variance in the target variable, explained by only the features that are helpful in making predictions, it will always be less than or equal to R² because it will penalize you for adding features which are not useful in making predictions. In the formula below n is the number of rows in the data and k is the number of columns in the data.

What to choose between MSE, RMSE, MAE, and MAPE?

Whenever you have to penalize the outliers present in the dataset you use either MSE or RMSE. I personally prefer RMSE over MSE as the latter does not have the same unit as my predicted value.

Use MAE when your predictions do not get affected by outliers as MAE is robust to outliers.

RMSE and MSE both are differentiable and are generally chosen over MAE, as MAE is not differentiable when actual value = predicted value.

Whenever your mean predicted values are large(in thousands or millions), choose MAPE(mean absolute percentage error) over the others as very large error values might look misleading.

R-Squared is a relative metric and should be preferred to compare models when we apply more than one algorithm.

Both R-Squared and the Error metrics hold important information, as the former tells us the relationship between independent and dependent variables and the latter tells us about how close are actual and predicted values.

CONCLUSION

We often make the mistake of randomly applying these evaluation metrics to access our model’s performance, but data science is all about asking the question “why?”.That is exactly what we should be doing while choosing certain evaluation metrics over others. With this blog, I hope I am able to clear the intuitions between different evaluation metrics used in classification as well as regression analysis.

Happy Learning!

References:

• MAE, MSE, RMSE, Coefficient of Determination, Adjusted R Squared — Which Metric is Better?
• Measures of Model Fit for Linear Regression Models
• RMSE vs. R-Squared: Which Metric Should You Use?

Which evaluation metric should you choose for evaluating your Classification and Regression models? was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

Disclaimer: This article is written with humorous intent. The Reserve Bank of India is admired globally and is the epitome of excellence in monetary policy and economic decision-making. Indian Police officers, in fact all UPSC personnel, are crème de la crème meritocrats. We do not condone any heinous activities.

The professor and his team are planning the heist in the RBI bank because they came to know that India is developing at an incredible rate. The professor is explaining how to minimize the risk of getting caught, by the Indian police, using the Gradient Descent Algorithm.

Tokyo: What is this Gradient Descent Algorithm?

Professor: Gradient Descent Algorithm is used to minimize a function by optimizing its parameter.

Raquel: Which function does it minimize?

Professor: It minimizes the cost function.

Denver: What is the cost function, and what is the need to minimize it?

Professor: The cost function represents the error between the actual value and the predicted value. Put another way: by minimizing the cost function, we are minimizing the error between the actual value and the predicted value.

Rio: How will this FUNCTION help us not to get caught?

Professor: By minimizing the function of the risk of getting caught. The Gradient Descent Algorithm is used to minimize the given function. In our case, it is a risk function. It does so by first calculating the slope of a function by the first-order derivative of the function and, second, it moves in the opposite direction to the gradient by alpha times the gradient at that point. (Professor took the chalk and wrote this on the blackboard)

New Value = Old Value - Step Size

Professor: In the world of mathematics, the ‘Step Size’ will be equal to the product of Learning Rate and Slope. So the above equation can be written like this.

Tokyo: Professor! In a simple language, please.

Professor: (Professor looked at Tokyo and slowly nodded and took the black color remote of the projector and presented this slide) In simple language, suppose this is a function that we want to minimize. And we want to find the value of X when Y is minimum. For this, we will be using Gradient Descent Algorithm.

Denver ((hastily answered): zero, zero X value should be zero for Y to be minimum.

Raquel: Why use Gradient Descent when we can clearly see the minimum value?

Professor: In this case, we are using a single variable. So the minimum value of a function is apparent. But what if there are multiple variables in the function? Say, 10 or 50? Then, how would you inspect the minimum value of that function? And by the minimum, I mean “local minima”, not global minima; because this algorithm finds the local minima of the function. For this very reason, we are using the gradient descent algorithm. Here we will start with some approximate values, and then we will be moving towards local minima. Let’s see this diagram.

Tokyo: Hold on, Professor. Do you want to risk our life by taking some approximate values?

Rio: Tokyo… calm down Tokyo.

Raquel: I am still not satisfied with this algorithm. Ah…Just show me what the graph of risk looks like after we enter the bank.

Rio: Yeah and also the risk function that we have to minimize. We’re supposed to minimize this function with respect to what? I am just asking what is X in our risk function?

Professor(calmly stares at Rio and then Raquel for a few seconds and then again to Rio and says): Time…We have to minimize the risk function with respect to time. Here’s how the graph of risk versus time looks like (professor took the remote and changed the slide) We need to escape from the bank at point A because after that it will become almost impossible to escape.

Denver: Okay. I want to ask why we are focusing so much on the Gradient Descent Algorithm. Can you please give some of the benefits of this algorithm?

Professor: Sure. There are around 4 major benefits of using this algorithm.

1. Fast enough to scale on big data.
2. The exact method will take more time. And we will never do it exactly, so we have to go with approximation.
3. The loss function gives the direction of the Optimal Solution.
4. Easy to understand.

See this graph of what this algorithm does. (Professor took the remote and showed this slide to everyone)

Professor: It is relatively very easy to understand.

Tokyo: Easy! What do you mean that I am a fool and everyone here is a genius? Come on, Professor. We will just keep on calculating these so-called minimum values of a function, and the Indian police will get us and maybe even shoot us. The lecture is over, and I am done with this ALGORITHM (Tokyo angrily left the class).

Meanwhile, the Indian Government came to know about this plan from their intelligence agency, and they have decided to contact AlmaBetter for a better understanding of this algorithm.

An Algorithm That Can Be Used To Rob a Bank was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is unsupervised learning?

With supervised learning, we always had a target variable. We knew what output we desire to predict and had many metrics, like accuracy, recall, precision, r score, etc to evaluate a supervised learning model. With unsupervised learning, we don’t really have any target variable. We just try to cluster data points that have some similarities between them.

So, in short, unsupervised learning in ML is where a model looks for patterns in a dataset with no labels and with minimal human supervision. We do not tell the model what to learn, but we allow it to find patterns and draw conclusions from the dataset. Unsupervised learning tasks mostly involve grouping data into similar clusters, dimensionality reduction, and density estimation.

In this blog, we will discuss the types of clustering.

What is clustering?

Clustering can be loosely defined as organizing objects into different groups or clusters, where members of the same cluster have some similarities and are dissimilar to data points that belong to other clusters. In the above plot, we can safely say that data points in cluster 0 are very different from cluster 1, but data points within cluster 0 are similar to each other.

TYPES OF CLUSTERING ALGORITHMS

Clustering can be connectivity-based, centroids-based, distribution-based, density-based, fuzzy, or constraint-based. In this blog, we will discuss K-Means clustering(centroids based with E-M algorithm) and hierarchical clustering(connectivity-based).

KMeans Clustering Algorithm

Clustering algorithms seek to learn underlying patterns and properties from data, to optimally divide data into labeled clusters. The simplest clustering algorithm is K-Means clustering. This algorithm searches for a pre-determined number of clusters within an unlabeled multidimensional dataset. The clusters have a center, which is the arithmetic mean of all the points inside the cluster and each point is closer to its own cluster than to other cluster centers. K means is limited to linear cluster boundaries, meaning this algorithm will be often ineffective for clusters with complex geometries. Since this is iteration-based, K means will be slower for a large number of samples.

So, how does KMeans clustering algorithm assigns data to the pre-defined clusters?
It follows the Expectation-Maximization (E-M) algorithm.
These are the steps followed by the E-M algorithm:

• Guess some clusters
• Assign points to nearest cluster center (E -step) -This step updates our expectation of which point each cluster belongs to.
• Set the cluster centers to mean (M -step) -This step maximizes the function that defines the location of cluster centers
• Repeat until all data points are clearly converged to their respective data points.

Some red flags about the E-M algorithm:

• The most optimal result may not be achieved -Even though every E-M step will produce better clusters than the previous E-M step, but it is not guaranteed that it will necessarily produce a globally optimal solution.
• The number of clusters (n_clusters) has to be selected beforehand.

There are 2 ways by which we can guess an optimal number of clusters. One is the Silhouette score, the other is the elbow method.

Silhouette Analysis

Silhouette analysis is used to study the separation distance between the resulting clusters. The silhouette plot displays a measure of how close each point in one cluster is to the points in the neighboring clusters. This plot ranges from [-1,1]. Silhouette coefficients nearer to +1 indicate that individual clusters are well defined. A value of 0 means the clusters are very close to or overlap the decision boundary between them. A value of -1 indicated that the data points have been assigned to the wrong clusters.

Silhouette score/coefficient is calculated as 𝑆=(𝑏−𝑎)/𝑚𝑎𝑥(𝑎,𝑏).

Elbow Method

In this method, we try to vary the number of clusters and for each value of k, we try to find the WCSS(within-cluster sum of squares). WCSS is the sum of squared distance between each point and the centroid of a cluster. For different values of K, we get different values of WCSS. WCSS has the highest value at K=1 and then it gradually decreases with an increase in K value. Since the plot looks like an elbow, it is known as the elbow method.

The elbow point gives us the optimal/desirable number of clusters that must be chosen for a particular dataset.

Hierarchical Clustering Algorithm

With hierarchical clustering, we do not have to pre-define several clusters. There are 2 types of hierarchical clustering.

• Agglomerative -Bottom-Up approach -We assign each point to a cluster and then we merge the closest pair of clusters and repeat this step until one cluster is left. Since we are merging the clusters at each step, this is also known as additive hierarchical clustering.

• Divisive -Top Bottom approach -This is the inverse of agglomerative clustering. Instead of starting with n clusters, we start with one cluster and assign points to that cluster. So, initially, all data points will belong to one cluster only. Then, at each iteration, the farthest point in the cluster is split and this process is repeated until each cluster contains only a single point.

How does hierarchical clustering work?

Let’s take an example of how agglomerative hierarchical clustering works.

First, let’s try to understand what a proximity matrix is. In the given figure, we have a student_id column and a marks column.

Imagine trying to find out the differences in marks between student 1 and the rest of the students and assign the differences in a list, just for our understanding. Let’s try that out now. For finding out the differences, we will use euclidean distance.

difference_list_student_1 = sqrt((10–10)²), sqrt((10–7)²), sqrt((10–28)²), sqrt((10–20)²), sqrt((10–35)²)
=> difference_list_student_1 = 0,3,18,10,25

Similarly, we can do that for the other student_id’s too and we can form a 5×5 matrix, also known as the distance matrix or the proximity matrix.

In this final matrix, one can notice that all the diagonal elements are 0, as they should be.

Now, the first step to hierarchical clustering is to assign points to individual clusters. Once we have done that, we will try to look for the smallest distance, using the proximity matrix and merge the points with the smallest distance. Then we will again update the proximity matrix and we will keep repeating until just a single cluster is left.

Linkage:

Linkage is a way to find out how individual clusters are linked to other clusters. There are 4 very popular linkages: single-linkage, complete-linkage, average-linkage, and centroid linkage. In this blog, we will focus on the first three types of linkages.

Single Linkage: This is the shortest distance between 2 clusters. This approach can separate non-elliptical shapes as long as the gap between 2 clusters is not too small. But, if there is any noise between the clusters, then this approach will not be helpful.

Complete Linkage: This is the longest distance between datapoints within two different clusters. So, if there is no noise between these clusters, complete linkage works well but it is more biased towards globular structures.

Average Linkage: This is the distance between each pair of observations in each cluster that is added up and divided by the number of pairs to get an average inter-cluster distance. Average linkage and complete linkage are the two most popular distance metrics in hierarchical clustering. This is also biased towards globular clusters.

CONCLUSION

Clustering is used to finding groups that have not been explicitly labeled in the data. Clustering finds uses in Search engines, Crime hot-spot detection, Insurance fraud detection, customer segmentation, diagnostic systems, etc.

REFERENCES

• AlmaBetter Materials
• Content from Youtube
• Content from medium articles, and other online webpages

Unsupervised Learning -1 was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

Although terms like artificial intelligence, machine learning and deep learning are used interchangeably but, they are not the same thing. Machine learning is the subset of artificial intelligence and deep learning is a subset of machine learning. Here, the key takeaway is:

• Artificial Intelligence deals with imitating human intelligence.
• Machine Learning deals with imitating how humans learn.
• Deep learning deals with imitating how human brain processes data with the help of neurons.

How Great Personalities Think about Machine Learning?

Alan Turing’s vision towards machine learning is being explained in one of his seminal paper such as “ Machine learning is an application of artificial intelligence where a computer/machine learns from the past experiences (input data) and make future predictions. The performance of such a system should be at least human level.”

A Computer Scientist Arthur Samuel who had experience for very long years in artificial intelligence, has described machine learning as “the study that gives computer the ability to learn without being explicitly programmed.”

Tom M. Mitchell’s technical definition is explianed as “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”

Machine learning in todays world is being treated as the emerging technology where all the problem in any of the field can be treated with solution of this technology. In my view machine learning is the concept where we are making the machines to learn from the previous data it may be labelled or not labelled with the foundation of mathematics and statistics.

Types of Learning used by Machine

• Supervised learning: This type of learning includes input data with labels which makes machines to identify them with the features which have been already predefined. In this type of learning by machines are supervised.
• Unsupervised learning: This type of learning not includes input data with any labels which makes machines hard to identify them. It is hard to tell what is good learning and what is not. Here machines are not supervised
• Reinforcement learning: This type of learning are directed of using task oriented algorithms, where learning will be achieved through series of steps.

Supervised Learning

Machines often learn from sample data that has both an example input and an example output. For example, one data-sample pair may be input data about an list of animals which are labelled and the associated output is the corresponding labelled identified animals. Given enough of these input-output samples, the machine learns how to construct a model that is consistent with the samples it trained on. From there, the model can be applied to new data that it has never seen before — in this case, which animal is given as input. After learning from sample data, the model applies what it has learned to the real world.

This class of machine learning is called “supervised learning,” since the desired predicted outcome is given, and the model is “supervised” to learn the associated model parameters. Humans know the right answer, and they supervise the model as it learns how to find it. Since humans must label all of the data, supervised learning is a time-intensive process.

Supervised Learning can be further classified into two types:

• Regression
• Classification

Regression
Regression learning allows us to predict continuous outcome variables based on the value of one or more predictor variables. In simple words, it estimates the relationship between the input and output variables. The goal of regression model is to build a mathematical equation that defines y as a function of the x variables.

Classification
Classification methods have a similar goal to regression models. Thus classification is a predictive modeling technique where a label is predicted for a given input. The difference between regression and classification is that the dependent attributes are numerical for regression while they are categorical for classification. In classification the output variable is categorical, that means it can be values like yes-no, true-false, spam-not spam, etc.

Unsupervised Learning
In unsupervised learning, the machine learns from data for which the outcomes are not known. It’s given input samples, but no output samples. Unsupervised learning is less common in practical business settings, but it is attractive: you don’t need labeled data and can avoid the human effort and cost of doing so. Unsupervised learning is potentially applicable in many more areas, since it’s not narrowly restricted to applications with labeled data.

Reinforcement Learning
As we know that supervised learning requires input and output examples. Reinforcement learning is like unsupervised learning in the sense that outputs are usually not given. The central concept of reinforcement learning is based around an “agent” (a computer or robot) that is interacting with an “environment” (here defined as everything that is not the agent). The agent performs actions on the environment (for instance, a robot takes a step forward). Then, the environments will then provide some sort of feedback to the agent, usually in a form called the “reward.” By “reward”, we don’t mean we give the machine a jolt of electrons. We literally just add to the program’s reward counter. The agent’s goal is to maximize the number in that counter. Critically, however, no one is telling the agent how to maximize the reward or explaining why it gets a reward. That’s what the agent figures out for itself by taking actions and observing its environment.

In many forms of reinforcement learning, the agent does not know what the objective is because it does not have examples of success. All it knows is whether it receives the reward or not. Reinforcement learning has echoes of human psychology — the brain experiences something good and a dopamine rush makes a person want more of it. A bad experience, like touching a hot stove, causes pain that discourages the person from repeating the behavior. However, despite the parallels to human psychology, humanizing it too much is a mistake.

1. Virtual Personal Assistants

Siri, Alexa, Google Now are some of the popular examples of virtual personal assistants. As the name suggests, they assist in finding information, when asked over voice. All you need to do is activate them and ask “What is my schedule for today?”, “What are the flights from Germany to London”, or similar questions. For answering, your personal assistant looks out for the information, recalls your related queries, or send a command to other resources (like phone apps) to collect info. You can even instruct assistants for certain tasks like “Set an alarm for 6 AM next morning”, “Remind me to visit Visa Office the day after tomorrow”.

Machine learning is an important part of these personal assistants as they collect and refine the information on the basis of your previous involvement with them. Later, this set of data is utilized to render results that are tailored to your preferences.

Virtual Assistants are integrated to a variety of platforms. For example:

• Smart Speakers: Amazon Echo and Google Home
• Smartphones: Samsung Bixby on Samsung S8
• Mobile Apps: Google Allo

2. Predictions while Commuting

Traffic Predictions: We all have been using GPS navigation services. While we do that, our current locations and velocities are being saved at a central server for managing traffic. This data is then used to build a map of the current traffic. While this helps in preventing the traffic and does congestion analysis, the underlying problem is that there are fewer cars that are equipped with GPS. The machine learning models in such scenarios helps to estimate the regions where congestion can be found on the basis of daily experiences.

Online Transportation Networks: When booking a cab, the app estimates the price of the ride. When sharing these services, how do they minimize the detours? The answer is machine learning. Jeff Schneider, the engineering lead at Uber ATC reveals in an interview that they use ML to define price surge hours by predicting the rider demand. In the entire cycle of the services, ML is playing a major role.

3. Videos Surveillance

Imagine a single person monitoring multiple video cameras! Certainly, a difficult job to do and boring as well. This is why the idea of training computers to do this job makes sense.

The video surveillance system nowadays is powered by AI that makes it possible to detect crime before they happen. They track unusual behaviour of people like standing motionless for a long time, stumbling, or napping on benches etc. The system can thus give an alert to human attendants, which can ultimately help to avoid mishaps. And when such activities are reported and counted to be true, they help to improve the surveillance services. This happens with machine learning models doing their job at the backend.

4. Social Media Services

From personalizing your news feed to better ads targeting, social media platforms are utilizing machine learning for their own and user benefits. Here are a few examples that you must be noticing, using, and loving in your social media accounts, without realizing that these wonderful features are nothing but the applications of ML.

• People You May Know: Machine learning works on a simple concept: understanding with experiences. Facebook continuously notices the friends that you connect with, the profiles that you visit very often, your interests, workplace, or a group that you share with someone etc. On the basis of continuous learning, a list of Facebook users are suggested that you can become friends with.
• Face Recognition: You upload a picture of yourself with a friend and Facebook instantly recognizes that friend. Facebook checks the poses and projections in the picture, notice the unique features, and then match them with the people in your friend list. The entire process at the backend is complicated and takes care of the precision factor but seems to be a simple application of ML at the front end..
• Similar Pins: Machine learning is the core element of Computer Vision, which is a technique to extract useful information from images and videos. Pinterest uses computer vision to identify the objects (or pins) in the images and recommend similar pins accordingly.

Machine Learning: Makes Human to Train Them was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is Python?

Python is a widely used general purpose, high-level object-oriented programming language that was initially created by Guido van Rossum in the year 1991 and developed by Python Software Foundation. It is used in almost every domain we can think of as mentioned below:

• Web Development
• Software Development
• Game Development
• AI & ML
• Data Analytics

The main purpose of using python as a programming language is code readability, and it syntax which is easy for the programmers to express concepts in fewer lines. In the late 1980's, It was that time when working on Python started. Soon after that, Guido Van Rossum began doing its application based work in December of 1989 by at Centrum Wiskunde & Informatica (CWI) which is situated in Netherland. It was started firstly as a hobby project because he was looking for an interesting project to keep him occupied during Christmas. The programming language which Python is said to have succeeded is ABC Programming Language, which had the interfacing with the Amoeba Operating System and had the feature of exception handling. He had already helped to create ABC earlier in his career and he had seen some issues with ABC but liked most of the features. After that what he did as really very clever. He had taken the syntax of ABC, and some of its good features. It came with a lot of complaints too, so he fixed those issues completely and had created a good scripting language which had removed all the flaws. The inspiration for the name came from BBC’s TV Show — ‘Monty Python’s Flying Circus’, as he was a big fan of the TV show and also he wanted a short, unique and slightly mysterious name for his invention and hence he named it Python! He was the “Benevolent dictator for life” (BDFL) until he stepped down from the position as the leader on 12th July 2018. For quite some time he used to work for Google, but currently, he is working at Dropbox.
The language was finally released in 1991. When it was released, it used a lot fewer codes to express the concepts, when we compare it with Java, C++ & C. Its design philosophy was quite good too. Its main objective is to provide code readability and advanced developer productivity. When it was released it had more than enough capability to provide classes with inheritance, several core data types exception handling and functions.

The Release dates for the major and minor versions of Python are

• Python 1.0-January 1994

1. Python 1.5-December 31, 1997
2. Python 1.6-September 5, 2000

• Python 2.0-October 16, 2000

1. Python 2.1-April 17, 2001
2. Python 2.2-December 21, 2001
3. Python 2.3-July 29, 2003
4. Python 2.4-November 30, 2004
5. Python 2.5-September 19, 2006
6. Python 2.6-October 1, 2008
7. Python 2.7-July 3, 2010

• Python 3.0-October 3, 2008

1. Python 3.1-June 27, 2009
2. Python 3.2-February 20, 2011
3. Python 3.3-September 29, 2012
4. Python 3.4-March 16, 2014
5. Python 3.5-September 13, 2015
6. Python 3.6-December 23, 2016
7. Python 3.7-June 27, 2018
8. Python 3.8-October 14, 2019
9. Python 3.9-October 05, 2020

“Hello World”: The word which we hear when we start Programming

If you happened to learn some programming languages, then I am sure you are aware of what I am talking about. The “Hello World” program is like a tradition in the developer community. If you want to master any programming language, this should be the very first line of code we should be seeking for.

print("Hello World")

Python: A new Way to deal with Programming World was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

Recommender systems are machine learning algorithms that help customers discover new products and services. Every time you browse online for either shopping or watching content, a recommendation system is guiding you towards the potential item you might purchase or watch. They are so widely used now that many of us benefit from them without even knowing. We can’t possibly look through all the products or content on a website, this is where the recommendation system comes into play by giving us a better search experience, while also showing us many more products we might not discover otherwise.

In 2006 Netflix famously offered a prize to solve a simple problem that had been around for years all online. Commerce giants are fighting to get better at solving this same problem given observations of a user’s past behavior predict which other things that the same user will like, for example, based on a person’s viewing history will she like the matrix or star wars.

We can represent user preferences graphically as connections between people on one side and things on the other such as movies here, the strength of the connection is visually represented by the thickness of the lines so the problem is to fill out all of these missing connections we don’t yet know shown in red, but because of the messy unpredictable nature of people’s tastes, we can never perfectly predict what they will like because for one it involves a guess about the future based on something you’ve never seen and two the answer is always in flux because people’s tastes change over time. What we can do is try to estimate those values as best we can be using whatever data we have access to. So regardless of the context, we can describe the problem in the following mathematical form.

We use a matrix which is a collection of numbers organized into rows and columns where the rows are users and the columns are items. In this case movies and each cell in this matrix contains a number that describes how much each individual likes each movie for example we could use a scale from 0 to 5 where a 0 would represent I hate it 2 is neutral and 5 as I love it. At any given moment there will be some set of preference data for users and movies but it will be incomplete because users have only seen and rated a few movies each and so the problem boils down to trying to predict these missing values how should we make these predictions.

There are two general approaches to this problem the first is called :

Content filtering is to use the information we know about people in things as connective tissue for recommendations. It’s like asking a person questions about what kinds of movies she likes to figure out if she will like the matrix. It begins by labeling each person and movies with some known attributes or what we call features,

for example, action and comedy that means if a person likes comedy and hates action we can represent her as 3 0 each movie is also mapped to each feature in the same way, for example, that matrix has no comedy and lots of action so we could represent it as 0 4 and to determine whether someone will like a movie we need to multiply these factors together so we could represent the strength of the connection between the user and the matrix as 3 times 0 plus 0 times 4 equals 0 so our estimation is that she will hate the movie so to make our predictions we first need to gather this feature data for every user and movie to simplify let’s return to a matrix the representation we can store this information in two matrices. One defines the mapping between people and features the other defines the mapping between movies and futures and by multiplying these two matrices together using matrix multiplication we get an estimated strength of the connection between every person and movie and if we’d like we can normalize our data to make sure the scale stays between 0 and 4 by dividing all values by 8 and rounding to the nearest point 4 that’s one way to solve this problem is known as content filtering.

The problem is :

1. it’s overly simplistic
2. not very accurate

that’s because there are obviously more relevant features of a movie than just comedy or action. The obvious way to improve this is just to include more features if your call when Netflix started it did just this would ask new users to fill out a laundry list of questions about their preferences before presenting them with suggested movies and this leads to the problem of having to collect all of this preference data on users not only is it a burden for users, it’s also prone to failure as we aren’t always great at describing our own preferences. Sometimes we simply can’t explain why we like things we just do which brings us to another approach called

Collaborative filtering: The motivation for collaborative filtering comes from the idea that you will probably like things people with similar viewing habits. This idea was popularized in this 2009 paper by Cohen Bell and Wolinski. So to begin we can throw away the idea of dreaming up features used to connect people and movies. Instead, we flip things around and use the user preference data we do have to generate the features. For example, we might have this incomplete set of preference data and we will instead learn or discover the relevant features based on patterns in this data. This is done by simply reversing the problem.

We first perform an approximate factorization into two matrices and we can do this using a machine-learning approach. The job of the machine-learning algorithm is to guess values for those matrices which will match the existing data and the preference matrix as closely as possible. The simplest approach is to simply guess numbers over and over until you arrive at a set of numbers that predict the data with the lowest error overall. Once this estimation is finished we can multiply the matrices as before to fill in all of the missing values.

It’s important to note that we won’t know exactly what to label these discovered features. So we call them latent features because they arise out of the underlying patterns and the data. You can think of them as an average or weighted sum of the patterns in the data are not based on a human-defined feature such as comedy. That’s the key insight behind this method. With content filtering the features come from the human mind whereas with collaborative filtering the features are extracted directly from the patterns in the data. This will predict the data in the same way but more accurately.

It’s also important to notice that this kind of prediction is actually a form of compression because we can represent a huge matrix of preference data by two smaller matrices of feature data. Once our feature data is determined or learned we can throw away the original data we started with because we can simply multiply the features together to recreate that data. The reason this is possible is that the preference data we started with is not random but follows patterns that exist across people. If instead the preference data was filled with random numbers we would not be able to compress it accurately because there would be no patterns to extract.

This collaborative filtering approach has also been applied to a wide range of contexts such as evaluating policy outcomes before they are implemented in the real world known as synthetic control. For example, if we wanted to understand the effect of gun control or minimum wage increases in some city we could look at the effect in similar cities first which have implemented those policies to approximate the outcome using the same mechanism. So it is the patterns we share that allow this method to work. Recommendation systems based on the ideas are in widespread use today whether it’s movies music news articles or anything else you search for online the things that are recommended to you are based on patterns the machine has observed in other people who are similar to yourself.

But Collaborative Filtering comes with its own shortcomings. One such problem in such kinds of systems is the problem of Cold-Start.

Cold- Start represents in a literal sense a situation when the engine is cold and we were are unable to start it. The user or visitor cold start simply means that when a recommendation system meets a new user for the first time and thus there is a lack of history about her preferences. Getting to know your user is critical in creating a great user experience for them.

It is faced when we have

1. A new user.
2. A new product
3. A new recommender system is in place.

Remedies for the cold start problem is :

When we have a new user :

1. Start by recommending whatever is newer, cheaper, or popular. Popular items are highly likely to be liked by the user because it is popular and it is easier to build a profile on popular items.
2. Recommend demographically relevant products. For example, if it is a movie recommendation system you can recommend Punjabi songs to a user situated in Punjab.
3. Recommend product according to age, sex, or geography.
4. Use other sources of data like social network data.

When we have a new recommender system :

1. Design a Non-Recommender system that captures data to feed it later to the recommender system.
2. Get similar data from other sources.

Six Examples of Recommender Systems

Recommender systems work behind the scenes on many of the world’s most popular websites. E-commerce websites, for example, often use recommender systems to increase user engagement and drive purchases, but suggestions are highly dependent on the quality and quantity of data that freemium (free service to use/the user is the product) companies already have in place.

1. AMAZON

When we buy something or browse anything on Amazon, we see recommended products based on our likes or search results on the page.

As we can see above, I may have searched for products related to Sudha Murthy, but Amazon clusters it into segments based on other previous search results. A recommender system often biases the user’s opinion.

2. IMDB

When we rate a TV show or movie on IMDb it recommends other shows or movies based on important details like cast, genre, sub-genre, plot, and summary.

As we can see above, I was recommended to rate Frozen Planet because I’ve watched David Attenborough’s wildlife documentary series. In this case, IMDb suggested this to me based on the cast of the series.

3. FACEBOOK & INSTAGRAM

Facebook and Instagram use recommender systems on a wider scale for suggesting friends and stories in the newsfeed.

Often in the newsfeed section, the user is bombarded with articles similar to one another based on the likes and pages they follow. Some argue it creates an unconscious bias among the user, which may be bad or useful depending on how the user interprets something when they have only one side of the argument.

4. YOUTUBE

YouTube recommends videos based on previous clicks.

As we can see above, the first video is about Google. While I’ve seen the video, YouTube recommends it to me again.

5. GOOGLE

Whenever we are logged in and search on Google, it is stored in our history and recommends products based on our previous searches. Not surprisingly, more than 95% of the company’s revenue is generated through advertising.

From the above, we can see we were getting targeted with these advertisements based on recent search results.

6. ANOTHER GOOGLE SUBSIDIARY- GMAIL

Gmail’s Sponsored Ad

Gmail recommends advertisements based on my personal inbox and tags they may have generated.

Other examples of recommender systems at work include movies on Netflix, songs on Spotify, and profiles on Bumble.

To summarise:

In this article we understood how the Recommendation System works and the difference between Collaborative Filtering vs Content-Based Filtering and their working. Both methods have their own set of advantages and disadvantages. We defined the problem of a cold start and saw some ways to tackle it. In the end, we saw some use cases of recommendation systems in big tech giants. Recommendation systems are very widely used in the tech industry, there are many kinds of research going on to develop more features in the recommender. I'll be writing a second part for the codes(building recommender systems)and other technological advancements in this area of machine learning.

Happy learning!

Resources on Recommendation engines :

• Listing Embeddings in Search Ranking
• Home Embeddings for Similar Home Recommendations - Zillow Tech Hub

• Embeddings | Machine Learning | Google for Developers
• For Your Ears Only: Personalizing Spotify Home with Machine Learning
• [PDF] Pandora ' s Music Recommender | Semantic Scholar
• Embeddings@Twitter

Recommender Systems — The most valued application of machine learning. was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

Suvojit Hore - System Engineer - Infosys Research and Development | LinkedIn

What if I told you, with just one picture of any person, you can make them dance as you like? Yeah, sounds crazy! But now its possible with the latest advancements in Generative Adversarial Networks.

Liquid warping GAN is a unified framework for human motion imitation, appearance transfer and novel view synthesis. It uses a 3D body mesh recovery module to disentangle the pose and shape, which can not only model the joint location and rotation but also characterize the personalized body shape, thereby using it to understand and imitate human body motion and transfer this motion to another human in an image.

So, what are the inputs needed? Well, just 1 picture of the person who will dance, and a reference video of the dance by another person or robot (basically anyone with a well-defined body shape). That’s all!

Here is a combination of all 3 — input image, reference video and the output:

Crazy right?! Well here’s a brief explanation of how you can create your own, along with a link to the google colab notebook.

Colab Link:

Google Colaboratory

Follow the link and open the colab notebook.

Clone the iPerCore repo, install all the requirements, setup iPerCore and download all the assets. If you want to try it out with the sample photo and videos, download the samples and test any of the three types of motion translation — First one requires only 1 full frontal image of the person, Second one requires both the frontal and backside images of the person, and the third requires both along with the image of the background.

Now, to try it on your own photo and video, just upload them into the assets folder and link the photo and video in the 2nd last cell for custom input. Follow the link naming conventions mentioned in the notebook.

That's it! Just run the cell and sit back and relax. For your custom video, it will be saved inside the results folder and you can download it to your pc to view it.

Suvojit Hore - System Engineer - Infosys Research and Development | LinkedIn

Make any person dance with just 1 image! (Artificial Intelligence) was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.

INTRODUCTION

One of the major disadvantage a tree based algorithm face is its tendency to overfit the data points.

1. The Decision tree tends to split on the features that are most important for the model and features that hold slightly less value are left as it is.
2. Allowing a decision tree to split till a granular degree, is a prime behavior of this model that makes it prone to learning every point extremely well — to the point of perfect classification on a training set — ie: overfitting.

Now before getting into how Ensembling technique overcomes this drawback of Decision Tree, let us first define what are these Emsembling techniques.

Ensemble techniques are meta algorithms that combine several machine learning techniques into one supervised model in order to decrease variance through bagging, bias through boosting, or improve predictions through stacking.

Ensemble methods can be divided into two Sub-categories:

• Sequential ensemble methods are a way in which data is trained on multiple weak learners(stumps) sequentially and the input of each successive learner is the residual of previous learner.(e.g. AdaBoost,CatBoost,GBM).
  The overall performance can also be boosted by weighing previously mislabeled examples with higher weight.
• Parallel ensemble methods where multiple weak learners work in parallel to come to a strong prediction either by taking average or by voting classifiers.(e.g. Random Forest).
  The basic motivation of parallel methods is that the error can be reduced dramatically by averaging.

Why are Ensembling methods so popular:

• Performance: An ensemble can make better predictions and achieve better performance than any single base model.
• Robustness: An ensemble reduces the variance of the predictions and model performance.
• Stability:We use Bagging when there is low bias — high variance and we use boosting when there is high bias — low variance.

Bagging

Bagging, also known as bootstrap aggregation, is an ensemble technique that is often used to reduce variance within a dataset. The technique is useful for both regression and classification method.Bagging is used with multiple decision trees, where it raises the stability of models by improving accuracy and reducing variance, which eliminates the problem of overfitting.Let us understand this with most common bagging algorithm called Random Forest.

Random Forest takes its name in a quite literal sense. Like a forest consists of multiple trees, similarly, the Raondom forest algorithm uses multiple decision trees in a bagging fashion.This in turn reduces the variance by taking an average of the predicted values of all the trees.

Bagging has two subcategories Bootstrap and Aggregation.

Bootstrap is a sampling method that works on a principle of sampling with replacement. Let's say we have M rows and N features. We will sample out some data which has m rows and n features, given M>m and N>n. After sampling out we replace the space created in the original data with the same values, we do this to completely randomize our sample.We get large number of such samples and feed them to individual decision trees.Another advantage of sampling is that all features get equal opportunity to get split upon in a decision tree.

Aggregation is a way of combining the predictions that multiple decision trees have made. Aggregation methods are different for classification and regression. In the case of regression, an average is taken of all the outputs predicted by the individual classifiers, this is known as soft voting. For classification problems, the class with the highest majority of votes is accepted; this is known as hard voting.

Boosting

Boosting is an ensembling technique specifically used when the model has high bias and low variance unlike bagging this technique works sequentially, it takes a weak learner and converts it into a strong learner by successively modeling on the residuals of the previous learner. Let us understand it in layman's terms.

1. The model first takes a weak learner preferably a stump(one split decision tree)we feed the data into it.
2. The model underfits on the data, leaving residuals.
3. Now because the model was unable to capture all the features of the data,there are still multiple features present in the residual(error margins)
4. This residual is given to the next weak learner as input which again is able to capture some features from it.
5. The process goes on till there are no features left in the residuals and the cost function is reached its minima.
6. The final value is calculated as the sum of predictions of all the weak learners.

Some examples of boosting techniques are AdaBoost and Gradient Boosting Machine(GBM), a brief description of their work is given below :

Adaboost or adaptive boosting is a technique that is used as an ensemble method in machine learning. It is known as adaptive boosting because the weights are re-assigned at each instance, and higher weights are assigned to incorrectly classified instances.

It works on the principle where learners are added sequentially. Except for the first, each subsequent learner is grown from previously added learners. In simple words, weak learners are converted into strong learners by modeling on the residuals.

Adaboost algorithm also works on the same principle as boosting, but there is a slight difference in working. In boosting algorithms, only the incorrectly classified data points are sent to the next base learner. While in Adaboost, both records are allowed to pass.Though there is an increase in the weight of wrongly classified records and a decrease in the weight of correctly classified records.

To get deeper knowledge of Adaboost and see its working in a stepwise example please refer to the link :https://www.mygreatlearning.com/blog/adaboost-algorithm/

Gradient Boosting Machine(GBM) is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees.

Input requirements for GBM :

1.A Loss Function to optimize.

2.A weak learner to make predictions (Generally Decision tree).

3.An additive model to add base learners to minimize the cost function.

Working: Gradient Boosting relies on the intuition that the best possible next model when combined with the previous models, minimizes the overall prediction errors.

Just like AdaBoost, GBM works by sequentially adding base learners to an ensemble, each one correcting its predecessor. However, instead of changing the weights for every incorrect classified observation after every iteration like AdaBoost, Gradient Boosting method tries to fit the new base learner to the residual errors made by the previous predictor.

Some important features of Gradient Boosting Machine :

1)Gradient Boosting is one of the boosting algorithms that is used to minimize bias error of the model.

2)We can tune the n_estimator of the gradient boosting algorithm. However, if we do not mention the value of n_estimator, the default value is taken as 100. (n_estimator is the number of decision trees)

3)When it is used as a regressor, the cost function is Mean Square Error (MSE) and when it is used as a classifier then Log loss is taken as a cost function.

For further information on Gradient Boosting Machine please refer to the link: https://www.analyticsvidhya.com/blog/2021/04/how-the-gradient-boosting-algorithm-works/

Now to answer the question with which we started this blog. How do Ensembling techniques overcome the drawbacks of Decision trees?

For decision trees, ensemble methods are one of the only efficient means of performing regularization on the model.

While using decision trees we face a conundrum: in order to model a complex set of data, we often need many levels of the tree , but as the tree grows deeper, it becomes prone to overfitting.

That’s because every time you add a new level to your tree, you’re adding a new predicate in the sequence.The chain becomes so specific that it’s very likely to apply very few training examples for each leaf. Also, the number of leaves grows exponentially with the number of levels, each successive level gets significantly more prone to overfitting.

So instead of using one tree for example with depth 50, we might use 10 trees each of depth 5 in a random forest. Each tree comes to its own prediction, and then voting methods is used for the final prediction where each tree gets a vote. That way our model becomes more robust and less overtly specific.

Advantages of using Ensembling technique

1. It acts as a regularising parameter on overfitting models.
2. it increases stability of our final model
3. it increases the accuracy of our model drastically

Disadvantages of using Ensembling technique

1. It is considered as a black box model and therefore in a tradeoff between accuracy and interpretability it becomes very difficult to interpret the model which is a vital part of any real-world solution.
2. Also, ensemble techniques cost more to create, train, and deploy. The ROI(Return of Investment)of an ensemble approach should be considered while keeping its computational cost in mind.

Conclusion

Apart from the methods discussed in this article, it is common to use ensembles in deep learning by training diverse and accurate classifiers. Diversity can be achieved by varying model architectures, hyperparameter tuning, and training techniques.

Ensemble methods have been thriving in setting record performance on big datasets and are among the winners of Kaggle competitions. But at the same time, we need to keep in mind the high computational cost and low interpretability of these models.

Recommended reading :

• Ensemble Learning | Ensemble Techniques
• Ensemble Learning to Improve Machine Learning Results
• Otto Group Product Classification Challenge
• 1.11. Ensemble methods

Ensembling methods and why they are most preferred among Machine Learning practitioners. was originally published in AlmaBetter on Medium, where people are continuing the conversation by highlighting and responding to this story.