Effect of Dimension Reduction on Prediction Performance of Multivariate Nonlinear Time Series

  • Received : 2015.09.04
  • Accepted : 2015.09.18
  • Published : 2015.09.30


The dynamic system approach in time series has been used in many real problems. Based on Taken's embedding theorem, we can build the predictive function where input is the time delay coordinates vector which consists of the lagged values of the observed series and output is the future values of the observed series. Although the time delay coordinates vector from multivariate time series brings more information than the one from univariate time series, it can exhibit statistical redundancy which disturbs the performance of the prediction function. We apply dimension reduction techniques to solve this problem and analyze the effect of this approach for prediction. Our experiment uses delayed Lorenz series; least squares support vector regression approximates the predictive function. The result shows that linearly preserving projection improves the prediction performance.


State space reconstruction;delay Lorenz series;Least Squares Support Vector Regression


  1. Zhi-Yong, Y., Guang, Y., and Cun-Bing, D. (2011), Timedelay feedback control in a delayed dynamical chaos system and its applications, Chinese Physics B, 20(1), 010207.
  2. Belkin, M. and Niyogi, P. (2002), Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering, In: T. G. Dietterich, S. Becker, and Z. Ghahramani. (eds), Advance in Neural Information Processing Systems, MIT Press, 585-591.
  3. Cao, L., Mees, A., and Judd, K. (1998), Dynamics from multivariate time series, Physica D: Nonlinear Phenomena, 121(1), 75-88.
  4. Chen, D. and Han, W. (2013), Prediction of multivariate chaotic time series via radial basis function neural network, Complexity, 18(4), 55-66.
  5. Das, A. and Das, P. (2007), Chaotic analysis of the foreign exchange rates, Applied Mathematics and Computation, 185(1), 388-396.
  6. Dhanya, C. and Kumar, D. N. (2011), Multivariate nonlinear ensemble prediction of daily chaotic rainfall with climate inputs, Journal of Hydrology, 403(3), 292-306.
  7. Dudul, S. V. (2005), Prediction of a Lorenz chaotic attractor using two-layer perceptron neural network, Applied Soft Computing, 5(4), 333-355.
  8. Fraser, A. M. and Swinney, H. L. (1986), Independent coordinates for strange attractors from mutual information, Physical Review A, 33(2), 1134.
  9. Adeli, H., Ghosh-Dastidar, S., and Dadmehr, N. (2008), A spatio-temporal wavelet-chaos methodology for EEG-based diagnosis of Alzheimer's disease, Neuroscience Letters, 444(2), 190-194.
  10. Barnard, J. P., Aldrich, C., and Gerber, M. (2001), Embedding of multidimensional time-dependent observations, Physical Review E, 64(4), 046201.
  11. Gholipour, A., Araabi, B. N., and Lucas, C. (2006), Predicting chaotic time series using neural and neurofuzzy models: a comparative study, Neural Processing Letters, 24(3), 217-239.
  12. Grassberger, P. and Procaccia, I. (2004), Measuring the strangeness of strange attractors, In: B. R. Hunt, J. A. Kennedy, T.-Y. Li and H. E. Nusse. (eds.), The Theory of Chaotic Attractors, Springer, 170-189.
  13. Han, M. and Wang, Y. (2009), Analysis and modeling of multivariate chaotic time series based on neural network, Expert Systems with Applications, 36(2), 1280-1290.
  14. Harding, A. K., Shinbrot, T., and Cordes, J. M. (1990), A chaotic attractor in timing noise from the VELA pulsar?, The Astrophysical Journal, 353, 588-596.
  15. He, X. and Niyogi, P. (2004), Locality preserving projections, In: S. Thrun, L. K. Saul and B. Scholkopf. (eds), In Advances in Neural Information Processing Systems 16, The MIT Press, Cambridge, USA, MA, 153-160.
  16. He, X., Cai, D., Yan, S., and Zhang, H.-J. (2005), Neighborhood preserving embedding, Proceedings of the Tenth IEEE International Conference on the Computer Vision (ICCV), 2, 1208-1213.
  17. Hotelling, H. (1933), Analysis of a complex of statistical variables into principal components, Journal of Educational Psychology, 24(6), 417- 441.
  18. Kennel, M. B., Brown, R., and Abarbanel, H. D. (1992), Determining embedding dimension for phase-space reconstruction using a geometrical construction, Physical Review A, 45(6), 3403-3411.
  19. Lorenz, E. N. (1963), Deterministic nonperiodic flow, Journal of the Atmospheric Sciences, 20(2), 130-141.<0130:DNF>2.0.CO;2
  20. Lorenz, E. N. (1995), The essence of chaos, University of Washington Press.
  21. Mackey, M. C. and Glass, L. (1977), Oscillation and chaos in physiological control systems, Science, 197(4300), 287-289.
  22. Mei-Ying, Y. and Xiao-Dong, W. (2004), Chaotic time series prediction using least squares support vector machines, Chinese Physics, 13(4), 454-458.
  23. Monahan, A. H. (2000), Nonlinear principal component analysis by neural networks: Theory and application to the Lorenz system, Journal of Climate, 13(4), 821-835.<0821:NPCABN>2.0.CO;2
  24. Mukherjee, S., Osuna, E., and Girosi, F. (1997), Nonlinear prediction of chaotic time series using support vector machines, Proceedings of the 1997 IEEE Workshop of the Neural Networks for Signal Processing, 511-520.
  25. Roweis, S. T. and Saul, L. K. (2000), Nonlinear dimensionality reduction by locally linear embedding, Science, 290(5500), 2323-2326.
  26. Shang, P., Li, X., and Kamae, S. (2005), Chaotic analysis of traffic time series, Chaos, Solitons and Fractals, 25(1), 121-128.
  27. Su, L.-y. (2010), Prediction of multivariate chaotic time series with local polynomial fitting, Computers and Mathematics with Applications, 59(2), 737-744.
  28. Suykens, J. A., De Brabanter, J., Lukas, L., and Vandewalle, J. (2002), Weighted least squares support vector machines: robustness and sparse approximation, Neurocomputing, 48(1), 85-105.
  29. Takens, F. (1981), Detecting strange attractors in turbulence. In: D. A. Rand and L. S. Young (eds.), Dynamical Systems and Turbulence, Warwick 1980, Springer, Berlin, German, 366-381.
  30. Tenenbaum, J. B., De Silva, V., and Langford, J. C. (2000), A global geometric framework for nonlinear dimensionality reduction, Science, 290(5500), 2319-2323.
  31. Torgerson, W. S. (1952), Multidimensional scaling: I. Theory and method, Psychometrika, 17(4), 401-419.
  32. Van der Maaten, L. (2007), An introduction to dimensionality reduction using matlab, Report, 1201(07-07), 62.
  33. Vapnik, V. (2013), The nature of statistical learning theory, Springer Science and Business Media.
  34. Zhang, T., Yang, J., Zhao, D., and Ge, X. (2007), Linear local tangent space alignment and application to face recognition, Neurocomputing, 70(7), 1547-1553.
  35. Zhang, Z.-Y. and Zha, H.-Y. (2004), Principal manifolds and nonlinear dimensionality reduction via tangent space alignment, Journal of Shanghai University (English Edition), 8(4), 406-424.

Cited by

  1. Data fusion combined with echo state network for multivariate time series prediction in complex electromechanical system pp.1807-0302, 2018,


Supported by : National Research Foundation of Korea (NRF)