First, let’s start with what is the cost function. The cost function basicly measures the performance of a machine learning model for given data. Let’s talk about linear regression beacuse we know something about it. In the previous article, we found our w and b values by trying. But it is not a good way to find these values. It needs too much time and effort. At this point, the cost function helps us.

How does the cost function work?

If you look at this formula, it may seem like it is not possible to understand it. However, if we start forming this formula from the beginning, you will absolutely understand its logic. First, let’s have a look at a linear regression example.

We know that linear regression model is a kind of linear function. We also know that the function behaves differently when we give different w and b values. As you can see in the graph we have inputs as x values and the outputs as y values. Each dot in the graph represent a single data (x, y) from the training set. In the formula ‘y’ represents the outputs in the training set. Also we have ‘y-hat’ in the formula and it represents the model function outputs with respect to the w and b values that we give. You can also rewrite ‘y-hat’ as f(x[i]) which also equals w * x[i] + b (i’th element of x). So as you can guess, we are subtracting the training set’s output values from the model function’s prediction outputs. We can visualize it like below. We actually measure the difference between the prediction and the output value, and take the square of this difference. You may ask why do we take the square of it. Actually, we have to do it for proper results. It has a mathematical proof but no need to explain it here. You can visit this link if you want to learn it right now.

We obtained our difference but we have to do it for all of the training set values. So we use sigma operation (for loop in coding) to do this task. It starts from i = 1 and do the same operation for all of the values in the training set. We learnt that m represents the number of data and we do this operation until we reach the last (m’th) data. Great, we learnt what does this notation mean :

We summed all of the differences between the training set outputs and the model function’s outputs. But we have to obtain a mean value to have ‘cost’. You can think this ‘cost’ as an error. So, we have to divide this value by the number of elements for which we calculated the error rate (basically the number of data = m). But wait, we have a problem. In the formula, we divide it by 2 times m. Also here, you can check this link to know why. It is basically for more proper and better results. Finally, we have reached the form of the cost function.

As you can see, we have done some operations and obtained a ‘cost’ or we can say an error. We have done it for a single set of w and b. As you remember, ‘y-hat’ represents the output of the model function and we created this function by giving the values w and b. So, this cost function calculated the error or ‘cost’ for a single function with constant w and b values, which means we can obtain different ‘costs’ or errors by giving different w and b values. At the end, if we want to choose the best model function, we can choose the one which the cost function gives us the smallest error with it. And here how the cost function helps us to find the best model function in linear regression.

Conclusion

We finally managed to learn how does the cost function work. We have different ‘costs’ or errors for each model function with different w and b values. By the help of the cost function, we can choose the best fitting line or the best model function. But we can’t try all of the w and b values by hand so we have to use another tool to help us, the gradient descent. In the next article, we are going to talk about the gradient descent. We will discuss what is the relation between the cost function and the gradient descent, how does it choose the w and b values automatically and finally how does it find the best w and b values for the model function.

Thank you so much for reading. If I could give you some information about the cost function, it will make me very happy. You can e-mail me if you have see any problem in the article or give me a feedback. Have a good day!

In this graph, we know that each red dot represent a single data in the dataset. In linear regression, we are trying to find the best line which goes through between these red dots perfectly. This line actually represents the model that we trained. After we train that model and find the perfect line, it is also a linear function, we can give any input to the model and get a prediction. First let’s start from the begginning and learn some terms about linear regression.

Terminology

Training Set : Training set is the dataset that is used to train the model. In Figure 1, each red dot represents a single data in the training set. In this example we have years of experience as inputs and salaries as ouputs.

The model : The model is used to predict some output values for each input we give to the model. In linear regression, we are going to give the training set to the linear machine learning algorithm and it will find the perfect fitting line, model, for us.

How To Create The Model / The Best Fitting Line

We learned the in linear regression, our model function is also a linear function. You know, linear functions are represented as f(x) = wx + b. Both w and b are the constants and when they are changed, the behaviour of our function also changes. Here are some examples;

• If we assign w as 3 and b as 2, we will have the function in Figure 2.

• If we assign w as 2 and b as 2, we will have the function in Figure 3.

• And here are some other examples in a single Figure.

As you can see, we can obtain different types of linear functions by changing the values of w and b. Also in linear regression, our algorithm will try to find the best w and b values for the training set. Later we can give any input to that model for prediction.

How To Implement To Python Code

Firstly, we have to implement the necessary modules :

import numpy as np
import matplotlib.pyplot as plt

Then, we have to define our training set. x_train values represents the experience in the sector and y_train values represents the salary annually. In addition we are going to use the variable m to represent the number of data in the training set :

x_train = np.array([5.0, 10.0])
y_train = np.array([180.0, 250.0])
m = x_train.shape[0]

Let’s plot the data :

plt.scatter(x_train, y_train, marker='x', c='r')
plt.title("Experiences - Salaries")
plt.ylabel('Salary (in 1000s of dollars)')
plt.xlabel('Experience (years)')
plt.show()

Here as you can see we plotted the data on a graph. Next we have to find the best fitting line for this data.

In a for loop, we can access all of the data seperately.

for i in range(m):
  print(x_train[i], y_train[i])

[ 5. 10.] [180. 250.]

Later, we can give some w and b values and investigate them in a graph. Let’s try w = 100 and b = 100 :

w = 100
b = 100

Now, we can create our model to try these values. We know that our function is f(x) = wx + b :

def create_model(x, w, b):
  m = x.shape[0]
  f_wb = np.zeros(m)
  print(f_wb)
  for i in range(m):
      f_wb[i] = w * x[i] + b
  print(f_wb)
  return f_wb

Now let’s call the create_model function and plot the output :

model = create_model(x_train, w, b)

plt.plot(x_train, model, c='b',label='Our Prediction')
plt.scatter(x_train, y_train, marker='x', c='r',label='Actual Values')
plt.title("Experiences - Salaries")
plt.ylabel('Salary (in 1000s of dollars)')
plt.xlabel('Experience (years)')
plt.legend()
plt.show()

As you can see, we couldn’t get the perfect line for this dataset. Suppose that w = 14 and b = 110. We are going to obtain this graph :

Here our line perfectly fit the data. In this example, we found our w and b by trying some values. Actually there is a much more easier way to find these values but we are going to talk about them later.

Now, we have a model which fits the data perfectly. Our model function is f(x) = 14x + 110 (w = 14, b = 110). Let’s give some experience values as inputs and get salaries as outputs.

f(6) = 14 * 6 + 110 = 194

f(7) = 14 * 7 + 110 = 208

f(8) = 14 * 8 + 110 = 222

As you can see, we can obtain some prediction results by the help of our model. In linear regression, we can get some predictions by following the steps above. But as I said, there is a much more better way to find w and b values. So in the next articles, we are going to talk about the Cost Function and the Gradient Descent.

Conclusion

Thank you so much for reading until here. If I could give you some knowledge about Linear Regression, it will make me very happy. If you find some mistakes or if you want to give me a feedback, you can e-mail me. Have a good day!

Unsupervised learning algorithms investigate the data and group them. Let me give you an example. Suppose that you have an advertisement company and you want to show your advertisements which the customers are interested in. So you have to group your customers by looking at the links they visited, the words they searched or the places they have been. As you can see, we have inputs but we don’t have any specific output. To do this job, we have to use unsupervised machine learning algorithms because we don’t have outputs in data. For this data we can use clustering machine learning algorithm. Clustering algorithm groups unlabeled examples. Let me visualise it.

As you can see we have three feature and each of them represents an input. They can be the links they visited, the place they have been or the words they searched. Clustering algorithm groups these customers by looking their interests and we can show the proper advertisements. But be careful that this machine learning model can not say the interests of any customer. It can only group them into clusters and it is the main difference between supervised and unsupervised learning.

There are also many real life examples such as Google News. Google News investigates thousands of news and clusters them everyday. It finds similar words in the news and groups them.

Unsupervised Learning Algorithms:

As you can guess, there many unsupervised machine learning algorithms in the world and we used clustering algorithm in the previous example. The characteristic feature of unsupervised machine learning algorithms is that data only comes with inputs but no output labels. Here is three the most common ones:

1- Clustering: Groups similar datapoints together.

2- Anomaly Detection: Find unusual data point. (For ex: Detect unusual progress in financial.)

3- Dimensionality Reduction: Compare data using fewer numbers.

Differences Between Supervised and Unsupervised Machine Learning

In supervised learning, the data have inputs and outputs both together. The supervised learning algorithm uses this data to train the model and then the model can predict anything. We have absolute outputs in supervised learning. However, in unsupervised learning, the data doesn’t have outputs. Unsupervised learning algorithm investigates the data and clusters them. As you can see in Figure 1, unsupervised learning algorithm groups the inputs into clusters while supervised learning algorithm trying to draw the best seperating line by looking at the outputs. After you finish training your model with supervised learning algorithm, if you give any input to the model, it can produce an output for you. But if you do the same thing with unsupervised learning, the model is not able to produce an output for you, it can only cluster it.

Thanks for reading until here. If you learned something new, that will make me very happy. If you want to give me a feedback or find something wrong, you can send me an email. Have a great day!

Nowadays, people try to compete with machine. We as humans can learn, research, guess and dream something. We know that machines are not capable of doing them but that does not mean they will never be. So, can the machines do the same thing like us?

Machine learning is a type of artificial intelligence focused on building computer systems that learn from data. We prepare the data and give it to the machines to learn from them. Then, machine learning algorithms are trained to find relationships and patterns in data.

There are the three most common types of machine learning and supervised learning is one of them:

1- Supervised Learning:

In supervised learning, machine learns the relation between the input and the output label. The characteristic of supervised learning is that we give the learning algorithm examples to learn from. We give some examples which include inputs and outputs and the learning algorithm learns from them. And after all, when we give only the input, the machine can guess what the output should be.

We can use supervised learning in spam filtering, speech recognation, machine translation, online advertising, self driving cars and visual inspection.

Let me give you an example about supervised learning to make it more clear. Suppose that we have a data which contains lots of car models, their motor volume, their release date and prices. We give this data to machine to learn from and machine learning algorithm are trained with the data. Then, we obtain a trained model which can guess the price of any given car. Let’s visualize it.

These red points represent our data. Our x-axis is the motor volume and y-axis is the prices of the cars. (Numbers in x and y-axis are only for visualization. They do not represent the real values.) And we are using linear regression (it is one of the supervised machine learning algorithm) to train the model. Linear regression algorithm trying to find the best linear function for the data. We can also say that it is trying to draw the best line between the data. When it finds the most accurate one, we can give some inputs to the model to produce some outputs. (We give motor volume and the machine guesses the price.)

We used linear regression for this data. But what is regression? Regression is useful when we are trying to predict a number from infinitely many possible outputs.

There are many supervised learning methods and classification is the other one. Classification is a supervised machine learning method where the model tries to predict the correct label of a given input data. The best way to understand the classification is giving an example.

Let’s say that we have a data which contains the temparature of a crocodile’s egg and the gender of that egg. We know that the temparature of the egg is very important for the gender of the crocodile. The model is trained with that data by using classification method. Realise that we have only two outputs, male and female. The temparature can change but the output can not be different.

As you can see we only have two possible option. (male and female) But it does not mean that we always have two outputs. It can be more that two possibilities. In addition, classification outputs can also be called as a class or category.

Classification models predict categories. Also categories don’t have to be a number. They can also be non-numeric. They can be an image and machine learning model can decide if it is a cat or dog. Moreover, there can be more than one input. For instance, we can also take the size of the eggs for this dataset.

Differences Between Regression and Classification:

The biggest difference from regression, we have obvious small number of possible outputs. As we said we have infinite possible outputs in regression. Also regression is trying to find the best line between each dots while classification is trying the draw a boundary between different outputs.

Thanks for reading. If you learned something new, it will make me very happy. If you want to ask any question or find something wrong, you can notify me via email. Have a good day!