Neural Network Training — TensorFlow Implementation

4 min readJun 21, 2026

A recap of Andrew Ng course (Machine Learning Specialization) on Coursera

Course 2 Week 1

Inference in code

We will implement the following algorithm in tensorflow. We could set the parameters for each layer with known values just for the sake of the inference demonstration.

Another implementation for handwritten digit recognition

Data in TensorFlow

NumPy is a popular way to represent data in Python; however, Google also developed TensorFlow to represent data for building neural networks. Unfortunately, there are some inconsistencies between how data is represented in Numpy and TensorFlow. So we need to make sure we are doing things correctly when we implement these in code.

By convention, we call the 1 by 2 matrix a row vector and a 2 by 1 matrix a column vector. We have been using the 1D array to represent data in course 1, but in course 2 we represent our data in matrices because TensorFlow was designed to handle a large amount of data.

the output of a1 is a row vector, and we could think of a Tensor as a matrix for now. We can convert a tensor back to NumPy.

Building a neural network

Get Aimyo’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

We have created Dense layers and then sequentially get the inputs and outputs. However, we can call a Sequential function in tensorflow to string together all the Dense layers. The Sequential functions do a lot of things for us, so that we don’t need to execute all the steps on our own.

In real-world code, we won’t specify each layer, we do the following instead. With the model, we can do the training and inferencing in TensorFlow.

Note: Including the sigmoid activation in the final layer is not considered best practice. It would instead be accounted for in the loss which improves numerical stability. This will be described in more detail in a later post.

Visualization

After creating a sequential model with two layers in TensorFlow, we can look at the model summary.

tf.random.set_seed(1234)  # applied to achieve consistent results
model = Sequential(
    [
        tf.keras.Input(shape=(2,)),
        Dense(3, activation='sigmoid', name = 'layer1'),
        Dense(1, activation='sigmoid', name = 'layer2')
     ]
)

Note: The tf.keras.Input(shape=(2,)), specifies the expected shape of the input. This allows Tensorflow to size the weights and bias parameters at this point. This is useful when exploring Tensorflow models. This statement can be omitted in practice and Tensorflow will size the network parameters when the input data is specified in the model.fit statement.

Note: The weights 𝑊 should be of size (number of features in input, number of units in the layer) while the bias 𝑏 size should match the number of units in the layer. In the first layer with 3 units, we expect W to have a size of (2,3) and 𝑏 should have 3 elements. In the second layer with 1 unit, we expect W to have a size of (3,1) and 𝑏 should have 1 element.

After fitting the model, we can visualize the units/neurons in first layer to see what the model learns

Interestingly, the first unit in first layer has larger activation values when the temperature is too low and then the second unit focuses on the duration. Finally, the third unit looks at the combination of duration and temperature.

The second layer takes the layer 1 activation values as inputs which ranges from 0 to 1. The visualization is below, the second layer outputs higher values when the combination of the three values from the first layer are towards 0.