The Sandwich Concept of Neural Network

As I embarked on several interview to learn neural network; one way i could actually break down my knowledge of Neural Network is to see a neural network was as a sandwich. A sandwich has an outer layer of the bread on one side and the input layer at the bottom. A good picture is seen below:

A Neural Sandwich Network

From the bottom looks very like the input layer, as its the first layer that holds the hidden layer (with vegetables and meat). This example is only exemplified because we cannot see the human brain, and its best illustrative to beginners. As we add more hidden layers of cheese/meat and vegetables our model neural sandwich grows.

Node to node growth in the neurons

Typically, this is how our sandwich looks from side with bread as the first input layer with cholesterol in it.

We see the processing here are arranged in layers as such the sandwich.

To better display our simple neural sandwich maker. we will apply codes in:

import numpy as np

def sigmoid(x):
    return 1/(1 + np.exp(-x))

def neural_network(X, y):
    learning_rate = 0.1
    W1 = np.random.rand(2, 4)
    W2 = np.random.rand(4, 1)

Sigmoid is a function that carries vector x and y. Think of it as the mayonnaise that hold your ingredient(s) for making the sandwich. It's like a skeleton frame that takes the first vector x to feed into each layer; much like the plate aids the ingredient of your sandwich before you place them in the bread.

As the hidden layer we have in the first bread picture has one layer, so does our example. And thus, our sandwich is a one layered sandwich = an output layer.

However, the weights that connect the input with layer1 ie the vegetable and meat interaction increase the weight of the bread; thus, more layers more weights between layers and we can have 2 layered sandwich = 2 hidden layers. But as mentioned, we are applying a single layer to our hidden layer for simplicity.

Hence, we initialize random weights. Where W1 are the weights (of the vegetables) that connect with the input (bread at the bottom) creating layer1. W2 is the weight of the meat that connect to the merged layer 1(with mayonnaise) creating a final output.

    for epoch in range(10000):
        layer1 = sigmoid(np.dot(X, W1))
        output = sigmoid(np.dot(layer1, W2))

        error = (y - output)

        delta2 = 2 * error * (output * (1 - output))
        delta1 = delta2.dot(W2.T) * (layer1 * (1 - layer1))
        W2 += learning_rate * layer1.T.dot(delta2)
        W1 += learning_rate * X.T.dot(delta1)

    return np.round(output).flatten()

Think of your epoch as putting your sandwich in a sandwich maker for 10,000 seconds to cook. The forward pass of

layer1 = sigmoid (X. W1); output = sigmoid (layer1. W2)

We compute the error of not placing our bread in position, or adding too much sandwich by taking derivatives of the error function of the actual bread before compression and after compression

error = (y - output)^2
derivative_error = 2 * (y - output)

We may turn over our bread to avoid stickiness; this process is backward propagation, and the compressed bread now has an updated weight to the network.

Get sumbo marcus’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

To do this, we compute how much the weights (in the bread) has to change:

delta2 -> changes that are applied to W2.

delta1-> changes that needs to be applied to W1. As we flip the bread over.

Once we flip over the bread, we can adjust the bread in position to correct the error from the original (forward instance). Therefore, we compute changes for W2 being turn over at the bottom using:

1. the derivative error
2. the derivative of the output using the sigmoid derivative (ie the layer and output seen below).

delta2 = 2 * error * (output * (1 - output))
delta1 = delta2.dot(W2.T) * (layer1 * (1 - layer1))

To compute the new weight gotten by flipping our bread. we have to compute the changes for W1 and the changes we computed for W2 (flipping action). Hence, we calculate the deltas and apply the learning rate (the time for the sandwich to be made i.e., time for placing the bread then flipping it) on the weight is multiplied to the layer and deltas.

we know we need shorter time, to flip the bread as the longer the bread stays the crunchier/crispy it becomes. Similarly, A smaller learning rate prevents overshooting.

Bon appetite: Gute Reise: Code Completed.

Reference:

ola-sumbo/my-travel-plan: Einige projekt mit udacity, das bild und arbeiten (github.com)

https://github.com/stellarnet/pybrain/blob/master/X%20%3D%20np.array(%5B%20%5B0%2C0%2C1%5D%2C%5B0%2C1%2C1%5D%2C%5

The Neural Networks Model — IBM Documentation

Sandwich — Search (bing.com)

2 layered Sandwich — Search (bing.com)

Photojoiner — Collage Editor