time series - What is a stationary process?

How do you explain what a stationary process is? In the first place, what is meant by process, and then what does the process have to be like so it can be called stationary?

Answer

A stationary process is one where the mean and variance don't change over time. This is technically "second order stationarity" or "weak stationarity", but it is also commonly the meaning when seen in literature.

In first order stationarity, the distribution of $(X_{t+1}, ..., X_{t+k})$ is the same as $(X_{1}, ..., X_{k})$ for all values of $(t, k)$.

You can see whether a series is stationary through it's autocorrelation function (ACF): $\rho_k = Corr(X_t, X_{t-k})$. When the ACF of the time series is slowly decreasing, this is an indication that the mean is not stationary; conversely, a stationary series should converge on zero quickly.

For instance, white noise is stationary, while a random walk is not. We can simulate these distributions easily in R (from a prior answer of mine):

op <- par(mfrow = c(2,2), mar = .5 + c(0,0,0,0))

N <- 500
# Simulate a Gaussian noise process
y1 <- rnorm(N)
# Turn it into integrated noise (a random walk)
y2 <- cumsum(y1)


plot(ts(y1), xlab="", ylab="", main="", axes=F); box()
plot(ts(y2), xlab="", ylab="", main="", axes=F); box()
acf(y1, xlab="", ylab="", main="", axes=F); box()
acf(y2, xlab="", ylab="", main="", axes=F); box()

par(op)

Which ends up looking somewhat like this:

If a time series varies over time, it is possible to make it stationary through a number of different techniques.