Forecasting with ARIMA Using Python

Forecasting

Introduction

Some of the methods for doing forecasting are: Exponential Smoothing, Single Equation Regression, Simultaneous-equation Regression, Autoregressive Integrated Moving Average (ARIMA), Vector Autoregression (VAR).

ARIMA

I’ll focus on ARIMA and how to automate the parameters on Python on this story. You can find more of my data science journey here.

An ARIMA model is a class of statistical models for analyzing and forecasting time series data. It explicitly caters to a suite of standard structures in time series data, and as such provides a simple yet powerful method for making skillful time series forecasts.

• AR: Autoregression. A model that uses the dependent relationship between an observation and some number of lagged observations.
• I: Integrated. The use of differencing of raw observations (e.g. subtracting an observation from an observation at the previous time step) in order to make the time series stationary.
• MA: Moving Average. A model that uses the dependency between an observation and a residual error from a moving average model applied to lagged observations.

The parameters of the ARIMA model are defined as follows:
p: The number of lag observations included in the model, also called the lag order.
d: The number of times that the raw observations are differenced, also called the degree of differencing.
q: The size of the moving average window, also called the order of moving average.

You can implement these below:

p_values = [0, 1, 2, 4, 6, 8, 10]
d_values = range(0, 3) #this is the range for the values
q_values = range(0, 3) #this is the range for the values

A linear regression model is constructed including the specified number and type of terms, and the data is prepared by a degree of differencing in order to make it stationary, i.e. to remove trend and seasonal structures that negatively affect the regression model.

A value of 0 can be used for a parameter, which indicates to not use that element of the model. This way, the ARIMA model can be configured to perform the function of an ARMA model, and even a simple AR, I, or MA model.

An ARIMA model can be created using the statsmodels library as follows:

1. Define the model by calling ARIMA() and passing in the p, d, and q parameters.
2. The model is prepared on the training data by calling the fit() function.
3. Predictions can be made by calling the predict() function and specifying the index of the time or times to be predicted.

We can automate the process of evaluating a large number of hyper-parameters for the ARIMA model by using a grid search procedure. We can automate the process of training and evaluating ARIMA models on different combinations of model hyper-parameters. In machine learning this is called a grid search or model tuning.

Get Joseph Magiya’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The approach is broken down into two parts:
1. Evaluate an ARIMA model.
2. Evaluate sets of ARIMA parameters.

Prerequisites

• Warnings
• pandas
• statsmodels (This library provides the capability to fit an ARIMA model)
• sklearn (for the mean Square Error)

For the above import them into your script as follows:

import warnings
from pandas import Series
from statsmodels.tsa.arima_model import ARIMA
from sklearn.metrics import mean_squared_error

Evaluate an ARIMA Model

1. Split the dataset into training and test sets.
2. Walk the time steps in the test dataset.
   a. Train an ARIMA model.
   b. Make a one-step prediction.
   c. Store prediction; get and store actual observation.
3. Calculate error score for predictions compared to expected values.

def evaluate_arima_model(X, arima_order):
    # prepare training dataset
train_size = int(len(X) * 0.66)
    train, test = X[0:train_size], X[train_size:]
    history = [x for x in train]
    # make predictions
    predictions = list()
    for t in range(len(test)):
        model = ARIMA(history, order=arima_order)
        model_fit = model.fit(disp=0)
        yhat = model_fit.forecast()[0]
        predictions.append(yhat)
        history.append(test[t])
    # calculate out of sample error
    error = mean_squared_error(test, predictions)
    return error

Evaluate sets of ARIMA parameters.

Specify a grid of p, d, and q ARIMA parameters to iterate. A model is created for each parameter, and its performance is evaluated by calling the evaluate_arima_model() function described in the previous section.

There are two additional considerations.
1. The first is to ensure the input data are floating point values (as opposed to integers or strings), as this can cause the ARIMA procedure to fail.

2. The statsmodels ARIMA procedure internally uses numerical optimization procedures to find a set of coefficients for the model. These procedures can fail, which in turn can throw an exception. We must catch these exceptions and skip those configurations that cause a problem.

warnings.filterwarnings("ignore")

This happens more often than you would think

def evaluate_models(dataset, p_values, d_values, q_values):
    dataset = dataset.astype('float32')
    best_score, best_cfg = float("inf"), None
    for p in p_values:
        for d in d_values:
            for q in q_values:
                order = (p,d,q)
                try:
                    mse = evaluate_arima_model(dataset, order)
                    if mse < best_score:
                        best_score, best_cfg = mse, order
                    print('ARIMA%s MSE=%.3f' % (order,mse))
                except:
                    continue
    print('Best ARIMA%s MSE=%.3f' % (best_cfg, best_score))

The Least MSE is the best set of parameters.

Running it

evaluate_models(series.values, p_values, d_values, q_values)

For a detailed example have a look at this on my Git.

BONUS: Issues and Work-around

After running the query, I bumped into this error multiple times while the code was executing.

Intel MKL ERROR: Parameter 4 was incorrect on entry to DLASCL.

If you’ve encountered this as well use this link to update conda, have a look here.

Cheers!