What is activation function?
An activation function is a mathematical function used in artificial neural networks to introduce non-linearity into the network. Neural networks consist of layers of interconnected nodes (neurons), and each connection between neurons has an associated weight. The activation function determines whether and to what extent a neuron is activated and contributes to the output of the network.
from wikipedia,
The activation function of a node in an artificial neural network is a function that calculates the output of the node based on its individual inputs and their weights. Nontrivial problems can be solved using only a few nodes if the activation function is nonlinear
Simple way to understand:
look at the down pictures
Think of an activation function as a gatekeeper or a water filter for information flowing through a neuron in a neural network
when we gives input dirty water into water filter and it return us clean drinking water so water filter ass ,stones layers decided how to clean dirty water .
Get Jabir Jayan’s stories in your inbox
Join Medium for free to get updates from this writer.
sometimes outsiders attempt to enter school but school gatekeeper prevent them to enter into school.He decide who will enter or not
activation function working in neuron as a gatekeeper or a water filter which information will ‘pass’ or ‘fire ‘ to next neuron
now we will talking about Non-linearity
you might be thinking what is Non-linearity !
lets break down ,
linearity:
- look at the picture you have a straight line, like the path a car follows when it’s driving on a flat road with no curves. In a linear function, if you double the input, you’ll also double the output, and if you add some value to the input, you’ll also see the same change in the output
Non-Linearity:
Now, if we look at the roller coaster track with twists and turns. Non-linearity means that the relationship between the input and output isn’t a straight line. When you change the input, the output doesn’t change in a simple, predictable way. Instead, it might change in unexpected ways, like going up and down, or even staying the same while the input changes.
