ScaleNet - Multiscale neural-network architecture for time series prediction

The effectiveness of a multiscale neural-network (NN) architecture for the time series prediction of nonlinear dynamic systems has been investigated. The prediction task is simplified by decomposing different scales of past windows into different scales of wavelets (local frequencies), and predicting the coefficients of each scale of wavelets by means of a separate multilayer perceptron NN. The short-term history (short past windows) is decomposed into the lower scales of wavelet coefficients (high frequencies) which are utilized for "detailed" analysis and prediction, while the long-term history (long past window) is decomposed into higher scales of wavelet coefficients (low frequencies) that are used for the analysis and prediction of slow trends in the time series. These coordinated scales of time and frequency provides an interpretation of the series structures, and more information about the history of the series, using fewer coefficients than other methods. The prediction's results concerning all the different scales of time and frequencies are combined by another "expert" perceptron NN which learns the weight of each scale in the goal-prediction of the original time series. Each network is trained by the backpropagation algorithm using the Levenberg-Marquadt method. The weights and biases are initialized by a new clustering algorithm of the temporal patterns of the time series, which improves the prediction results as compared to random initialization. Three main sets of data were analyzed: the sunspots' benchmark, fluctuations in a far-infrared laser and a nonlinear numerically generated series. Taking the ultimate goal to be the accuracy of the prediction, we found that the suggested multiscale architecture outperforms the corresponding single-scale architectures. The employment of improved learning methods for each of the ScaleNet networks can further improve the prediction results.