Time Series Analysis
Autocorrelation and Smoothing
Forecasting gives us tomorrow's information, today, ... but only if we're good at it.
--------------------------------------------------------------------------------------
There is no time like the present to get a watch.
----------------------------------------------
This section presents a selected portion of a critique of standard techniques for forecasting and suggestions prepared by David P. Reilly, Senior Vice-President of Automatic Forecasting Systems. I've added links. Experience with statistics will help to understand the material. Click here for the full text.
Time series analysis attempts to match the observable patterns in data to an underlying model or sequences of models. These models may be identical or different for distinct ranges of time. Even if the model form is identical, the model parameters may be different. [See, for example, my interpretation of modeling, and the modeling possibilities in playing the stock market.] It is possible and even probable that the variance of the series [data dispersion] may not be constant. If it is not constant then it may be related to the level of the series or simply evidence of major changes or might even stochastically evolve over time.
Time series has three major dimensions. The first and simplest case is a single endogenous (dependent) variable that may or may not be affected by unusual events. The second case extends the first by including input (causal) variables that may have a role in the prediction or modeling of this dependent variable. The third case allows for simultaneous modeling of multiple dependent series. ....
On this point I should say that you can look but not see, listen but not hear. I.e., Observation (seeing, hearing,...) requires a model by means of which understanding occurs. I submit that any data collection presumes a model of some sort, since you need rules to direct you to the data in the first place -- data does not come to you unbidden. The issue turns on what you mean by 'model.' Admittedly, the process can be "ugly".
Statistical methods can be very useful in summarizing information and very powerful in testing hypothesis. ... To proceed with the application of a statistical test, one has to be careful about validating the assumptions under which the test is valid. It is often possible to remedy the violation and then safer to proceed. One of the most often violated assumptions is that the observations are independent. Unfortunately, the real world operates in ignorance of many statistical assumptions, which can lead to problems in analysis. The good news is that these problems may be easily overcome, so long as they are recognized and dealt with correctly.
The assumption of independence of observations implies that the most recent data point contains no more information or value than any other data point, including the first one that was measured. In practice, this means that the most recent reading has no special effect on estimating the next reading or measurement. [Unlike with stock market data, for instance, where the more recent prices have more value.] In summary, it provides no information about the next reading.
If this assumption, i.e. independence of all readings, is true, this implies that the overall mean or average is the best estimate of future values. [Compare with
moving average of market prices.] If however there is some serial or autoprojective (autocorrelation) structure, the best estimate of the next reading will depend on the recently observed values. The exact form of this prediction is based on the observed correlation structure. Stock market prices are an example of an autocorrelated data set. Weather patterns move slowly, thus daily temperatures have been found to be reasonably described by an AR(2) model which implies that your temperature is a weighted average of the last two days temperature. Try it and see if doesn't work!
----------------------------------------------
This material was written by Robert A. Yaffee, a statistician in the Social Sciences Computing Group of NYU's Academic Computing Facility.
If a researcher interested in longitudinal analysis has a series of data points over a time span, there are several methods he may use to define the nature of this series.
Time-series analysis may be divided into five basic classifications: extrapolative, decomposition, Box-Jenkins, spectral, and dynamic regression models.On the basis of a model of a given time series, future values of a series may be generated as a forecast over a chosen time horizon. The researcher first graphs the data and then divides it into initial and validation datasets. After formulating a model on the basis of the initial dataset, he compares its predicted values to the actual values of the validation dataset. [You might compare this with the forecasting techniques used in
neural networks.] ...Extrapolative methods consist of a variety of exponential smoothing techniques. [See
exponential moving average.] First, there is simple exponential smoothing, with or without a constant (a baseline level for the series). Second, there is Holt exponential smoothing with a trend (a deterministic tendency over time) for long-term patterns. Third, there is Winters exponential smoothing, which involves a linear or quadratic trend with a multiplicative or additive seasonal (regular variation around the trend) component. There is also stepwise autoregressive exponential smoothing for more short-run fluctuations.Exponential smoothing methods are typically cheaper, easier to use, and need less data than the fifty or more equally spaced values over time required by the Box-Jenkins techniques. For these reasons, smoothing methods are often applied to production, sales, and inventory control where strong consideration is given to keeping costs down and profits up; however, if seasonal variation is present, typically at least two years of data are necessary for both exponential smoothing and Box-Jenkins analysis. Dynamic-regression models are particularly good for development of social-science theory, insofar as they can handle more independent variables (hypothesis of simple relationships) and interactions (hypotheses of joint relationships). Dynamic regression may not be as easy to develop and apply as the more mechanical exponential-smoothing methods. Although exponential-smoothing methods do not produce theoretical models, their mechanical forecasts under some circumstances may be more feasible, accurate, cheaper, and timely than those derived from the Box-Jenkins models, which is why they have to be considered by persons studying or applying forecasting techniques....
This discussion is consistent with the view that you should always have the freshest possible picture of the market, because expectations are always changing. The changing expectations lead to new buy/sell decisions, and this causes chart patterns to morph.
---------------------------------------