Difference Equations – Gaurav Sharma's Blog

1. Difference Equations

1.1. Introduction

Time series analysis deals with a series of random variables.

1.2. First Order Difference Equations

We will study time indexed random variables ${y_t}$ .

Let ${y_t}$ be a linear function of ${y_{t-1}}$ and ${w_t}$ .

$\displaystyle y_t = \phi{y_t} + w_t \ \ \ \ \ (1)$

Equation 1 is a linear first-order difference equation. It is a first-order difference equation because ${y_t}$ only depends on ${y_{t-1}}$ and not on other previous ${y_t}$ s.

In this chapter, we treat ${w_t}$ as a deterministic number and later on we will analyse the effects of treating it as a random variable.

1.3. Solution by Recursive Substitution

The equations are:

$\displaystyle \begin{array}{rcl} y_0 = \phi{y_{-1}} + w_t \\ y_1 = \phi{y_0} + w_t \\ \vdots \\ y_{t-1} = \phi{y_{t-2}} + w_{t-1} \\ y_t = \phi{y_t} + w_t \end{array}$

By recursively substituting we obtain:

$\displaystyle y_t = \phi^{t+1}y_{-1} + \phi^tw_1 + \phi^{t-2}w_2 + \dotsc + \phi{w_{t-1}} + w_t \ \ \ \ \ (2)$

1.4. Dynamic Multipliers

We want to know the effect of increasing ${w_t}$ on ${y_{t+j}}$ . This can be obtained by differentiating equation 10 with respect to ${w_t}$ .

$\displaystyle \frac{\partial y_{t+j}}{\partial w_t} = \phi^{j} \ \ \ \ \ (3)$

2. pth-Order Difference Equations

We generalize the above dynamic system to let the value of y to depend on p of its own lags in addition to the current value of the input variable ${w_t}$ .

$\displaystyle y_t = \phi_{1}y_{-1} + \phi^tw_1 + \phi^{t-2}w_2 + \dotsc + \phi{w_{t-1}} + w_t \ \ \ \ \ (4)$

We will rewrite the above p-th order equation to a vector first order equation.

We define,

$\displaystyle \xi_t= \begin{bmatrix} y_t \\ y_{t - 1} \\ \vdots \\ y_{t - p + 1} \end{bmatrix},\quad \ \ \ \ \ (5)$

$\displaystyle F = \begin{bmatrix} \phi_{1} & \phi_{2} & \phi_{3} & \cdots & \phi_{p - 1} & \phi_{p} \\ 1 & 0 & 0 & \cdots & 0 & 0\\ 0 & 1 & 0 & \cdots & 0 & 0\\ \vdots & \vdots & \vdots & \cdots & \vdots & \vdots \\ 0 & 0 & 0 & \cdots & 0 & 0 \end{bmatrix},\quad \ \ \ \ \ (6)$

$\displaystyle v_t= \begin{bmatrix} w_t \\ 0 \\ 0 \\ \vdots \\ 0 \end{bmatrix},\quad \ \ \ \ \ (7)$

Then,

$\displaystyle \xi_t = F\xi_{t-1} + v_t \ \ \ \ \ (8)$

or,

$\displaystyle \begin{bmatrix} y_t \\ y_{t - 1} \\ y_{t - 3;} \\ \vdots \\ y_{t - p + 1} \end{bmatrix}\quad = \quad \begin{bmatrix} \phi_{1} & \phi_{2} & \phi_{3} & \cdots & \phi_{p - 1} & \phi_{p} \\ 1 & 0 & 0 & \cdots & 0 & 0\\ 0 & 1 & 0 & \cdots & 0 & 0\\ \vdots & \vdots & \vdots & \cdots & \vdots & \vdots \\ 0 & 0 & 0 & \cdots & 0 & 0 \end{bmatrix}\quad \begin{bmatrix} y_{t - 1} \\ y_{t - 2} \\ y_{t - 3} \\ \vdots \\ y_{t - p} \end{bmatrix}\quad + \quad \begin{bmatrix} w_t \\ 0 \\ 0 \\ \vdots \\ 0 \end{bmatrix}\quad \ \ \ \ \ (9)$

Following the approach taken for solving first order difference equation and applying it to the vector equation, we get,

$\displaystyle \xi_t = F^{t+1}\xi_{-1} + F^tv_0 + F^{t-1}v_1 + F^{t-2}v_2 + \dotsc + F{v_{t-1}} + v_t \ \ \ \ \ (10)$

3. General Solution of a p-th Order Difference Equation

If the eigenvalues of F matrix are distinct then we can write F as

$\displaystyle F = T\Lambda T^{-1} \ \ \ \ \ (11)$

$\displaystyle \Lambda = \begin{bmatrix} \lambda_{1} & 0 & 0 & \cdots & 0 \\ 0 & \lambda_{2} & 0 & \cdots & 0 \\ 0 & 0 & \lambda_{3} & \cdots & 0 \\ \vdots & \vdots & \vdots & \cdots & \vdots \\ 0 & 0 & 0 & \cdots & \lambda_{p} \end{bmatrix},\quad \ \ \ \ \ (12)$

where T is a non-singular matrix.

Thus,

$\displaystyle F^2 = T\Lambda^2 T^{-1} \ \ \ \ \ (13)$

and

$\displaystyle \Lambda^2 = \begin{bmatrix} \lambda_{1}^2 & 0 & 0 & \cdots & 0 \\ 0 & \lambda_{2}^2 & 0 & \cdots & 0 \\ 0 & 0 & \lambda_{3}^2 & \cdots & 0 \\ \vdots & \vdots & \vdots & \cdots & \vdots \\ 0 & 0 & 0 & \cdots & \lambda_{p}^2 \end{bmatrix},\quad \ \ \ \ \ (14)$

In general,

$\displaystyle F^n = T\Lambda^n T^{-1} \ \ \ \ \ (15)$

and

$\displaystyle \Lambda^n = \begin{bmatrix} \lambda_{1}^n & 0 & 0 & \cdots & 0 \\ 0 & \lambda_{2}^n & 0 & \cdots & 0 \\ 0 & 0 & \lambda_{3}^n & \cdots & 0 \\ \vdots & \vdots & \vdots & \cdots & \vdots \\ 0 & 0 & 0 & \cdots & \lambda_{p}^n \end{bmatrix},\quad \ \ \ \ \ (16)$