Why is \(cov(x_t, x_{t+k}) = \dfrac{\alpha^k \sigma^2}{1-\alpha^2}\)?
If \(\{x_t\}\) is a stable \(AR(1)\) process (which means that $||<1) can be written as:
\[\begin{align*}
(1-\alpha \mathbf{B}) x_t &= w_t \\
\implies x_t &= (1-\alpha \mathbf{B})^{-1} w_t \\
&= w_t + \alpha w_{t-1} + \alpha^2 w_{t-2} + \alpha^3 w_{t-3} + \cdots \\
&= \sum\limits_{i=0}^\infty \alpha^i w_{t-i}
\end{align*}\]
From this, we can deduce that the mean is
\[
E(x_t)
= E\left( \sum\limits_{i=0}^\infty \alpha^i w_{t-i} \right)
= \sum\limits_{i=0}^\infty \alpha^i E\left( w_{t-i} \right)
= 0
\]
The autocovariance is computed similarly as:
\[\begin{align*}
\gamma_k = cov(x_t, x_{t+k})
&= cov \left(
\sum\limits_{i=0}^\infty \alpha^i w_{t-i}, \\
\sum\limits_{j=0}^\infty \alpha^j w_{t+k-j} \right) \\
&= \sum\limits_{j=k+i} \alpha^i \alpha^j cov ( w_{t-i}, w_{t+k-j} ) \\
&= \alpha^k \sigma^2 \sum\limits_{i=0}^\infty \alpha^{2i} \\
&= \frac{\alpha^k \sigma^2}{1-\alpha^2}
\end{align*}\]
See Equations (2.15) and (4.2).
