Time Series (1)
Time Series (Introduction)
Characteristics of Time Series
The primary objective of time series analysis is to develop mathematical models that provide plausible descriptions for sample data with time correlations. In order to provide a statistical setting for describing the character of data that seemingly fluctuate in a random fashion over time, we assume a time series can be defined as a collection of random variables indexed according to the order they are obtained in time. In general, a collection of random variables
Example of Series:
White Noise: A collection of uncorrelated, independent and identically distributed random variables
with mean 0 and finite variance . A particular useful white noise is Gaussian white noise, that is
Moving Average: We might replace the white noise series
by a moving average that smooths the series: This introduces a smoother version of white noise series, reflecting the fact that the slower oscillations are more apparent and some of the faster oscillations are taken out.
Autoregressions: Suppose we consider the white noise series
as input and calculate the output using the second-order equation: For . We can see the periodic behavior of the series.
Autocorrelation and Cross-Correlation
A complete description of a time series with
In practice, the multidimensional distribution function cannot usually be written easily unless the random variables are jointly normal. It is an unwieldy tool for displaying and analyzing time series data. On the other hand, the marginal distribution functions:
or the corresponding marginal density functions:
And The Mean Function:
When they exist, are often informative for examining the marginal behavior of the series.
Autocovariance
The autocovariance measures the linear dependence between two points on the same series observed at different times:
- Very Smooth series exhibit autocovariance functions that stay large even when
and are far apart - Very Choppy series tend to have auto covariance functions that are nearly zero for large separations.
Autocorrelation
The ACF measures the linear predictability of the series at time
Cross-covariance and Cross-correlation
Often, we want to measure the predictability of another series (different components)
We can easily extend the idea to multivariate time series where each sample contains r attributes:
The extension of autocovariance is then:
Stationary Time Series
There may exist a sort of regularity over time in the behavior of a time series.
Strictly Stationarity
When
When
are identically distributed
are identically distributed
Thus, if the variance function of the process exists, we have autocovariance function of the series
Same conclusions for all possible values of
Weak Stationarity
Thus,
or
From above, we can clearly see that a strictly stationary, finite variance, time series is also stationary. The converse is not true unless there are further conditions.
Several Properties:
is non-negative definite, for all positive integers and vector :
Gaussian Process
If a Gaussian Process is weakly stationary then it is also strictly stationary.
Linear Process
A Linear process
The autocovariance function for linear process is:
Estimation of Correlation
If a time series is stationary, the mean function is constant, so that we can estimate it by the sample mean:
The sum runs over a restricted range because
The sample autocorrelation function has a sampling distribution that allows us to assess whether the data comes from a completely random or white series or whether correlations are statistically significant at some lags.
Based on the property, we obtain a rough method of assessing whether peaks in
Vector-Valued and Multidimensional Series
Consider the notion of a vector time series
And the
Since,
The sample autocovariance matrix is defined as:
Where
In many applied problems, an observed series may be indexed by more than one time alone. For example, the position in space of an experimental unit might be described by two coordinates. We may proceed in these cases by defining a multidimensional process (does not have multiple dependent variables as multivariate case)
Where
The multidimensional sample autocovariance function is defined as:
Where each summation has range
Where each summation has range
The multidimensional sample autocorrelation function follows:
EDA
In general, it is necessary for time series data to be stationary, so averaging lagged products over time will be a sensible thing to do (fixed mean). Hence, to achieve any meaningful statistical analysis of time series data, it will be crucial that the mean and the autocovariance functions satisfy the conditions of stationarity.
Trend
Detrend
The easiest form of nonstationarity to work with is the trend stationary model wherein the process has stationary behavior around a trend. We define this as:
Where
Differencing
Differencing can be used to produce a stationary time series. The first difference operator is a linear operator denoted as:
If
Where
One advantage of differencing over detrending to remove trend is that no parameters are estimated in the differencing operation. One disadvantage, however, is that differencing does not yield an estimate of the stationary process
Backshift Operator
Backshift Operator is linear. We can rewrite first difference as:
And second difference as:
Transformations
If a time series presents nonstationary as well as nonlinear behavior, transformations may be useful to equalize the variability over the length of a single series. A particular useful transformation is the log transformation:
Which tends to suppress larger fluctuations that occur over portions of the series where the underlying values are larger.
Smoothing
Smoothing is useful in discovering certain traits in a time series, such as long-term trend and seasonal components.
Moving Average Smoother
If
Where
Kernel Smoothing
Kernel smoothing is a moving average smoother that uses a weight function or kernel to average the observations:
Where
Lowess and Nearest Neighbor Regression
Another approach to smooth a time series is nearest neighbor regression. The technique is based on
Lowess is a method of smoothing that is complex but similar to nearest neighbor regression:
- A certain proportion of nearest neighbors to
are included in a weighting scheme, values closer to in time get more weight. - A robust weighted regression is used to predict
and obtain the smoothed estimate of .
Visualization
Lagged Scatterplot Matrices (Non-linearity and Lag correlation)
In the definition of the ACF, we are essentially interested in relations between
The plot displays values
on the vertical axis plotted against on the horizontal axis.The sample autocorrelations are displayed in the upper right-hand corner and superimposed on the sctterplots are locally weighted scatterplot smoothing lines (LOWESS) that can be used to help discover any nonlinearities.
Ref
Time Series Analysis and Its Applications With R Examples by Robert H.Shumway and David S.Stoffer
https://www.math-stat.unibe.ch/e237483/e237655/e243381/e281679/files281692/Chap13_ger.pdf