RNN Model and Its Implementation Using Python

RNN

Deep learning has transformed the way we handle sequential data such as text, speech, time-series, and financial forecasting. Among various architectures, the Recurrent Neural Network (RNN) is a foundational model for sequence-based tasks.

In this article, we’ll cover:

• What an RNN is and how it works
• Advantages and limitations of RNNs
• Step-by-step Python implementation of an RNN using TensorFlow/Keras
• Example on a text classification dataset

🔹 What is an RNN?

A Recurrent Neural Network (RNN) is a type of neural network designed to process sequential data. Unlike feedforward neural networks, RNNs retain a memory of previous inputs due to their recurrent connections.

At each time step, an RNN takes an input x_t and a hidden state h_{t-1} from the previous step, and outputs a new hidden state h_t.

Get Iqra Mahmood’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Mathematically:

ht​=f(Wxt​+Uht−1​+b)

where:

• W and U are weight matrices
• b is a bias term
• f is a non-linear activation (like tanh or ReLU)

This structure allows RNNs to retain context across sequences, making them useful for language modeling, sentiment analysis, and time-series forecasting.

🔹 Limitations of RNNs

Despite their power, vanilla RNNs face challenges:

• Vanishing gradients: Hard to learn long dependencies.
• Slow training: Sequential processing reduces parallelization.
• Better alternatives: LSTMs and GRUs solve many of these problems.

Still, learning vanilla RNNs is important because it builds the foundation for advanced architectures.

🔹 RNN Implementation in Python (Keras/TensorFlow)

Let’s implement a simple RNN for text classification using the IMDB dataset.

Step 1: Import Libraries

import numpy as np
from tensorflow.keras.datasets import imdb
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, SimpleRNN, Dense
from tensorflow.keras.preprocessing import sequence

Step 2: Load and Preprocess Data

# Load dataset (only top 10,000 words)
max_features = 10000
maxlen = 500  # cut reviews after 500 words

(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
# Pad sequences to ensure equal length
x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = sequence.pad_sequences(x_test, maxlen=maxlen)

Step 3: Build RNN Model

model = Sequential()
model.add(Embedding(max_features, 32))
model.add(SimpleRNN(32))  # Vanilla RNN Layer
model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

Step 4: Train the Model

history = model.fit(x_train, y_train,
                    epochs=5,
                    batch_size=64,
                    validation_split=0.2)

Step 5: Evaluate Performance

test_loss, test_acc = model.evaluate(x_test, y_test)
print(f"Test Accuracy: {test_acc:.2f}")

🔹 Results

On running the above code, you should get ~80% accuracy on sentiment classification. While not state-of-the-art, this demonstrates how RNNs capture sequential information.

🔹 Real-World Applications of RNNs

• Natural Language Processing (NLP): Sentiment analysis, text generation, machine translation
• Speech Recognition: Understanding spoken words in sequence
• Time-Series Forecasting: Stock market prediction, weather forecasting
• Healthcare: Predicting patient health outcomes from sequential medical records

🔹 Final Thoughts

The Recurrent Neural Network (RNN) is a powerful architecture for handling sequential data. While modern deep learning often favors LSTMs, GRUs, or Transformers, understanding RNNs is essential for anyone diving into deep learning with Python.

By implementing RNNs in Python with Keras/TensorFlow, you gain a practical understanding of how sequence models work.