<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pang K.H. on Medium]]></title>
        <description><![CDATA[Stories by Pang K.H. on Medium]]></description>
        <link>https://medium.com/@pangkh98?source=rss-c6e25dbb9e63------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*hyoTEYCo5fEqRAju.jpg</url>
            <title>Stories by Pang K.H. on Medium</title>
            <link>https://medium.com/@pangkh98?source=rss-c6e25dbb9e63------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:34:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pangkh98/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Multi-Step Multivariate Time-Series Forecasting using LSTM]]></title>
            <link>https://pangkh98.medium.com/multi-step-multivariate-time-series-forecasting-using-lstm-92c6d22cd9c2?source=rss-c6e25dbb9e63------2</link>
            <guid isPermaLink="false">https://medium.com/p/92c6d22cd9c2</guid>
            <category><![CDATA[lstm]]></category>
            <category><![CDATA[recurrent-neural-network]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[keras]]></category>
            <dc:creator><![CDATA[Pang K.H.]]></dc:creator>
            <pubDate>Fri, 04 Dec 2020 10:00:56 GMT</pubDate>
            <atom:updated>2021-01-23T07:58:32.705Z</atom:updated>
            <content:encoded><![CDATA[<p>A simple tutorial of developing LSTM model for Time-Series Forecasting</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3z3IKfKJHQP7ymi3lBHHgA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@julianhochgesang?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Julian Hochgesang</a> on <a href="https://unsplash.com/s/photos/future-time?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3><strong>Concept</strong></h3><p>If you remember the plot of one of the MCU movie series Captain America: The First Avenger, Zola’s Algorithm was created to predict an individual’s future by evaluating their personal information such as bank records, medical histories and etc. Well, if you forgotten…here’s the clip to revise :</p><blockquote><a href="https://www.youtube.com/watch?v=tmfLYkcgJD8&amp;ab_channel=GregBurton">https://www.youtube.com/watch?v=tmfLYkcgJD8&amp;ab_channel=GregBurton</a></blockquote><p>In short, the application of Zola’s Algorithm is to predict the future by looking back at the previous data, similar to today’s Recurrent Neural Network (RNN) where it is used to forecast the dependent variables based on previous independent variables.</p><p>In this article, I will build a Multivariate Multi-Step predictive model using LSTM algorithm running in Google Colab. If you wish to understand how LSTM work before heading to the coding part, here is an great article with .gif that explain everything :</p><p><a href="https://medium.com/mlreview/understanding-lstm-and-its-diagrams-37e2f46f1714">Understanding LSTM and its diagrams</a></p><h3><strong><em>What is </em>Time<em>-Series Forecasting ?</em></strong></h3><p>Time-Series forecasting basically means predicting future dependent variable (y) based on past independent variable (x).</p><h3>What is Multivariate Forecasting ?</h3><p>If the model predicts dependent variable (y) based on one independent variable (x), it is called <strong>univariate </strong>forecasting. For <strong>Multivariate </strong>forecasting, it simply means predicting dependent variable (y) based on more than one independent variable (x).</p><h3>What is Multi-step Forecasting ?</h3><p>If the model predicts a single value for next time-step, it is called <strong>one-step </strong>forecast. For <strong>Multi-step</strong> forecast, it means predicting few times-steps ahead.</p><h3>What is Multivariate Multi-step Time-Series Forecasting ?</h3><p>With all methods combined, the model in this article will predict multi-step ahead of dependent variable (y) based on the past two independent variables (x).</p><h3>Full code</h3><p>Click <a href="https://github.com/mikepang98/TimeSeriesLSTM.git">HERE </a>for full code and all the necessary files.</p><h3><strong>Begin.</strong></h3><p><strong>Libraries required</strong></p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>import pandas as pd<br>import statsmodels.api as sm<br>from pandas import DataFrame , concat<br>from sklearn.metrics import mean_absolute_error , mean_squared_error<br>%matplotlib inline<br>%config InlineBackend.figure_format = &#39;retina&#39;<br>from numpy import mean , concatenate<br>from math import sqrt<br>from pandas import read_csv<br>from sklearn.preprocessing import MinMaxScaler<br>from tensorflow.keras.models import Sequential<br>from tensorflow.keras.layers import Dense,LSTM,Activation<br>from numpy import array , hstack<br>from tensorflow import keras<br>import tensorflow as tf</pre><pre># if you&#39;re using google drive to store the dataset<br>from google.colab import drive<br>drive.mount(&#39;/content/drive&#39;)</pre><p><strong>Dataset <br></strong>In this project, the dataset contain the independent variables (x1 , x2) and the dependent variables are (y) as shown in the Table below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/217/1*3YQMtY4wLLCdt5wQF7ajNQ.png" /><figcaption>Dataset</figcaption></figure><p><strong>Visualize Data<br></strong>In this dataset, we have total of 28 batches of data containing 1 dependent variable(y) and 2 independent variables(x1,x2). Each variable in every batch contain 1258 data points. Forming 1258 * 28 = 35224 of data points for each variable.</p><pre>url = &#39;/yourDirectory/Dataset.csv&#39;<br>dataset = pd.read_csv(url)<br>x_1 = dataset[&#39;x1&#39;]<br>x_2 = dataset[&#39;x2&#39;]<br>y = dataset[&#39;y&#39;]</pre><pre>x_1 = x_1.values<br>x_2 = x_2.values<br>y = y.values</pre><pre>plt.figure(figsize=(30, 6))<br>plt.plot(x_1[:10064] , label=&#39;x1&#39;)<br>plt.plot(x_2[:10064] , label=&#39;x2&#39;)<br>plt.plot(y[:10064] , label=&#39;y&#39;)<br>plt.legend(loc=&#39;upper right&#39;)<br>plt.title(&quot;Dataset&quot; ,  fontsize=18)<br>plt.xlabel(&#39;Time step&#39; ,  fontsize=18)<br>plt.ylabel(&#39;Values&#39; , fontsize=18)<br>plt.legend()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WVDMse3Fro5p_ISqXuxRGw.png" /><figcaption>First 8 Batches of Data</figcaption></figure><p><strong>Training Data Pre-Processing <br></strong>In common practice, data are usually pre-processed before splitting into train and test set. Here, we will need to perform data transformation by normalizing each variable data using MinMaxScaler. The steps are as follows :</p><ol><li>reshape each variable to 2D array with the configuration :(number of data, 1) for scaling.</li><li>Specify a Scaler ranging from 0 to 1 and fit each data variable individually. <em>To Note</em> : the fit_transform() function comprises of both function fit() and transform(), where it scan the max and min value of the data and transform them between 0 and 1. If the data has different range, the transformation will be different. For more info : <a href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html">MinMaxScaler()</a></li><li>Stack all the variables horizontally.</li></ol><pre># Step 1 : convert to [rows, columns] structure<br>x_1 = x_1.reshape((len(x_1), 1))<br>x_2 = x_2.reshape((len(x_2), 1))<br>y = y.reshape((len(y), 1))</pre><pre>print (&quot;x_1.shape&quot; , x_1.shape) <br>print (&quot;x_2.shape&quot; , x_2.shape) <br>print (&quot;y.shape&quot; , y.shape)</pre><pre># Step 2 : normalization <br>scaler = MinMaxScaler(feature_range=(0, 1))<br>x_1_scaled = scaler.fit_transform(x_1)<br>x_2_scaled = scaler.fit_transform(x_2)<br>y_scaled = scaler.fit_transform(y)</pre><pre># Step 3 : horizontally stack columns<br>dataset_stacked = hstack((x_1_scaled, x_2_scaled, y_scaled))<br>print (&quot;dataset_stacked.shape&quot; , dataset_stacked.shape)</pre><p>We want our model to learn by relating the previous [x1,x2] data with the future [y] data. Therefore, we need to re-arrange our data accordingly by using a split sequences() function created by MachineLearningMastery. There are 2 key arguments we need to specify which are : <br>1. n_steps_in : Specify how much data we want to look back for prediction<br>2. n_step_out : Specify how much multi-step data we want to forecast</p><p>Example : given<strong> n_steps_in = 8</strong> and <strong>n_step_out = 9</strong>, the training data will be arrange as per figure below, by arranging 8 independent variable with the next 9 dependent variable (including current step).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1DgWr82FnOLPHpS0WFyT5A.png" /></figure><p>In this project, we will lookback on 60 (x1,x2) past data, to forecast future 30 multi-step ahead of data. Therefore, the returned X and y are time-step shifted according to the argument, the the size should be (35139, 60, 2) and (35139, 30).</p><blockquote>The returned array are arranged in [n_dataset, n_steps_in , n_features] and [n_dataset, n_steps_out]. Feel free to print it out to understand more by using this code : print(X[i], y[i])</blockquote><pre># split a multivariate sequence into samples<br>def split_sequences(sequences, n_steps_in, n_steps_out):<br> X, y = list(), list()<br> for i in range(len(sequences)):<br>  # find the end of this pattern<br>  end_ix = i + n_steps_in<br>  out_end_ix = end_ix + n_steps_out-1<br>  # check if we are beyond the dataset<br>  if out_end_ix &gt; len(sequences):<br>   break<br>  # gather input and output parts of the pattern<br>  seq_x, seq_y = sequences[i:end_ix, :-1], sequences[end_ix-1:out_end_ix, -1]<br>  X.append(seq_x)<br>  y.append(seq_y)<br> return array(X), array(y)</pre><pre># choose a number of time steps #change this accordingly<br>n_steps_in, n_steps_out = 60 , 30</pre><pre># covert into input/output<br>X, y = split_sequences(dataset_stacked, n_steps_in, n_steps_out)<br>print (&quot;X.shape&quot; , X.shape) <br>print (&quot;y.shape&quot; , y.shape)</pre><p><strong>Splitting Data<br></strong>As mentioned earlier, we have total of 28 batches of dependent and independent variables, now is the time to split them into 25 batches for training and the rest 3 batches for testing.</p><pre>split_point = 1258*25<br>train_X , train_y = X[:split_point, :] , y[:split_point, :]<br>test_X , test_y = X[split_point:, :] , y[split_point:, :]</pre><p>Finally, our testing and training data shape should be like this : <br>train_X.shape (31450, 60, 2) <em>[n_datasets,n_steps_in,n_features]</em><br>train_y.shape (31450, 30) <em>[n_datasets,n_steps_out]</em><br>test_X.shape (3689, 60, 2) <br>test_y.shape (3689, 30) <br>n_features = 2</p><p><strong>LSTM Model Setting<br></strong>Here, we will start to set up our LSTM model architecture by initializing the optimizer learning rate as well as number of layers in the network. The neural network consist of : 2 LSTM nodes with 50 hidden units, a dense layer which specify the model’s output based on n_steps_out (how many future data we want to forecast) and end with an activation function.</p><p>To Note : The <strong>input_shape</strong> of the first LSTM node specify the shape of input data for prediction in the evaluation phase. Meanwhile, the <strong>Dense Layer </strong>specify the output shape of the model based on <strong>n_steps_out</strong></p><pre>#optimizer learning rate<br>opt = keras.optimizers.Adam(learning_rate=0.0001)</pre><pre># define model<br>model = Sequential()<br>model.add(LSTM(50, activation=&#39;relu&#39;, return_sequences=True, input_shape=(n_steps_in, n_features)))<br>model.add(LSTM(50, activation=&#39;relu&#39;))<br>model.add(Dense(n_steps_out))<br>model.add(Activation(&#39;linear&#39;))<br>model.compile(loss=&#39;mse&#39; , optimizer=opt , metrics=[&#39;mse&#39;])</pre><p><strong>Training <br></strong>In training, we have 25 batches of dataset, therefore the steps_per_epoch will be 25. In each steps, it will take 1258 data points (1 batch) for training. With 25 steps per epoch, all 25 batches will be trained in each epoch. With 60 epochs, each batch will be trained 60 times. The whole training progress should be less than 15minutes with Google Colab.</p><pre># Fit network</pre><pre>history = model.fit(train_X , train_y , epochs=60 , steps_per_epoch=25 , verbose=1 ,validation_data=(test_X, test_y) ,shuffle=False)</pre><p><strong>Test Data Preparation<br></strong>Here, the 3 batches of test data are manually saved in different csv file to ease the evaluation process. First, read the first batch of test data, reshape and scale it.</p><pre># Test Data Batch 1 , Test Data Batch 2 , Test Data Batch 3<br>url_test = &#39;/directory/Test Data Batch 1.csv&#39;<br>dataset_test_ok = pd.read_csv(url_test)<br>dataset_test_ok.head()</pre><pre># read test data<br>x1_test = dataset_test_ok[&#39;x1&#39;].values<br>x2_test = dataset_test_ok[&#39;x2&#39;].values<br>y_test = dataset_test_ok[&#39;y&#39;].values # no need to scale</pre><pre># convert to [rows, columns] structure<br>x1_test = x1_test.reshape((len(x1_test), 1))<br>x2_test = x2_test.reshape((len(x2_test), 1))<br>y_test = y_test.reshape((len(y_test), 1))</pre><pre>x1_test_scaled = scaler.fit_transform(x1_test)<br>x2_test_scaled = scaler.fit_transform(x2_test)</pre><p><strong>Performance Evaluation<br></strong>Next, we will use the <strong>prep_data </strong>function to prepare test data and feed into our trained model to forecast the next 30 data. Besides, it also does prepare the past data for plotting purposes as well as groundtruth for validation. The argument — <strong>start</strong>, <strong>end </strong>and <strong>last </strong>will be meant by —</p><ol><li>start , end : To specify range of Test Data based on <strong>n_steps_in</strong>. can be any point in the test data batch.</li><li>last : last data of predicted value based on <strong>n_step_out</strong>.</li></ol><p>To use this function, just simply feed in <strong>x1_test_scaled</strong>, <strong>x2_test_scaled, y_test </strong>and specify the <strong>start </strong>argument. The <strong>end </strong>&amp;<strong> last </strong>will be calculated automatically.</p><pre>def prep_data(x1_test_scaled , x2_test_scaled , y_test , start , end , last):<br>    #prepare test data X<br>    dataset_test = hstack((x1_test_scaled, x2_test_scaled))<br>    dataset_test_X = dataset_test[start:end, :]<br>    print(&quot;dataset_test_X :&quot;,dataset_test_X.shape)<br>    test_X_new = dataset_test_X.reshape(1,dataset_test_X.shape[0],dataset_test_X.shape[1])<br>    print(&quot;test_X_new :&quot;,test_X_new.shape)</pre><pre>#prepare past and groundtruth<br>    past_data = y_test[:end , :]<br>    dataset_test_y = y_test[end:last , :]<br>    scaler1 = MinMaxScaler(feature_range=(0, 1))<br>    scaler1.fit(dataset_test_y)<br>    print(&quot;dataset_test_y :&quot;,dataset_test_y.shape)<br>    print(&quot;past_data :&quot;,past_data.shape)</pre><pre>#predictions<br>    y_pred = model.predict(test_X_new)<br>    y_pred_inv = scaler1.inverse_transform(y_pred)<br>    y_pred_inv = y_pred_inv.reshape(n_steps_out,1)<br>    y_pred_inv = y_pred_inv[:,0]<br>    print(&quot;y_pred :&quot;,y_pred.shape)<br>    print(&quot;y_pred_inv :&quot;,y_pred_inv.shape)<br>    <br>    return y_pred_inv , dataset_test_y , past_data</pre><pre>#start can be any point in the test data (1258)<br>start = 60<br>end = start + n_steps_in <br>last = end + n_steps_out <br>y_pred_inv , dataset_test_y , past_data = prep_data(x1_test_scaled , x2_test_scaled , y_test , start , end , last)</pre><p>Next, we will evaluate the forecasted results with the metrics MAE and RMSE by using the <strong>evaluate_prediction </strong>function. Just simply feed in the forecasted results and groundtruth from <strong>prep_data </strong>function.</p><pre># Calculate MAE and RMSE<br>def evaluate_prediction(predictions, actual, model_name , start , end):<br>    errors = predictions - actual<br>    mse = np.square(errors).mean()<br>    rmse = np.sqrt(mse)<br>    mae = np.abs(errors).mean()</pre><pre>    print(&quot;Test Data from {} to {}&quot;.format(start, end))<br>    print(&#39;Mean Absolute Error: {:.2f}&#39;.format(mae))<br>    print(&#39;Root Mean Square Error: {:.2f}&#39;.format(rmse))<br>    print(&#39;&#39;)<br>    print(&#39;&#39;)<br></pre><pre>evaluate_prediction(y_pred_inv , dataset_test_y, &#39;LSTM&#39; , start , end)</pre><p>Lastly, we will be plotting the graphs by using <strong>plot_multistep </strong>function. Just simly feed in the past_data , forecasted results and groundtruth</p><pre># Plot history and future<br>def plot_multistep(history, prediction1 , groundtruth , start , end):<br>    plt.figure(figsize=(20, 4))<br>    y_mean = mean(prediction1)<br>    range_history = len(history)<br>    range_future = list(range(range_history, range_history + len(prediction1)))<br>    plt.plot(np.arange(range_history), np.array(history), label=&#39;History&#39;)<br>    plt.plot(range_future, np.array(prediction1),label=&#39;Forecasted with LSTM&#39;)<br>    plt.plot(range_future, np.array(groundtruth),label=&#39;GroundTruth&#39;)<br>    plt.legend(loc=&#39;upper right&#39;)<br>    plt.title(&quot;Test Data from {} to {} , Mean = {:.2f}&quot;.format(start, end, y_mean) ,  fontsize=18)<br>    plt.xlabel(&#39;Time step&#39; ,  fontsize=18)<br>    plt.ylabel(&#39;y-value&#39; , fontsize=18)</pre><pre>plot_multistep(past_data , y_pred_inv , dataset_test_y , start , end)</pre><p><strong>Results </strong><br>For each batch of test data, 120 to 180 and 180 to 240 will be the testing data to be fed into the trained model. Since we have 3 batches of test data, total of 6 tests will be performed. But in this article, only one batch of result will be shown.</p><pre>for i in range(120,240,60):<br>    start = i<br>    end = start + n_steps_in <br>    last = end + n_steps_out <br>    y_pred_inv , dataset_test_y , past_data = prep_data(x1_test_scaled , x2_test_scaled , y_test , start , end , last) <br>    evaluate_prediction(y_pred_inv , dataset_test_y, &#39;LSTM&#39; , start , end) <br>    plot_multistep(past_data , y_pred_inv , dataset_test_y , start , end)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x33IRvQ4mkQqHZHFmIMFzA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5wBRagtJmV2Mgzu-GGoYCg.png" /><figcaption><strong>Graph of forecasted results</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*uotwYEWx3WExb2bCyA-VtA.png" /><figcaption><strong>RMSE &amp; MAE Result</strong></figcaption></figure><h3>Summary</h3><p>In this tutorial, a LSTM model is developed. It has the capability of forecasting 30 steps ahead data based on previous 60 data with 2 features.</p><p>This code is also capable of processing datasets with more than 2 features<br>Feel free to modify the <strong>n_steps_in </strong>and <strong>n_steps_out </strong>for different variation.</p><h3><strong>Personal Note</strong></h3><p>Hopefully this article is able to help in understanding LSTM as well as the workflow of developing a RNN model. Also, this is my first article on Medium, do point out my mistakes for personal development.</p><p>Feel free to connect with me : <a href="https://www.linkedin.com/in/pang-kah-hoe-726620144/">LinkedIn</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92c6d22cd9c2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>