A Grey Model-Least Squares Support Vector Machine Method for Time Series Prediction

: In this study, the authors aim to solve the time series prediction problem through pre-predicting multiple influence factors of the target sequence. Focusing on two pre-prediction approaches of influence factors (i.e., factors driven approach and time driven approach), we propose a time series prediction method based on the least squares support vector machine and grey model (GM-LSSVM). This method could improve the prediction precision of the target time series by differentiating the data characteristics of influence factors. A case study is put forward to predict China's economy from the perspective of system innovation and technological innovation. We selected public statistics data from 2005 to 2014 from the national bureau. The numerical experiment results illustrate that the accuracy of the GM-LSSVM is able to reach 95%, which proves the effectiveness of our proposed method in practice.


INTRODUCTION
Increasing scenarios of artificial intelligence technologies are keeping studies on time series prediction at center stage [8,34]. Especially in the economic context, large amounts of economic time series data are produced every second, and various business scenarios like the economic growth prediction, economic policy making, and social development condition analysis, motivate the continuous improvement of prediction methods [35].
Referring to relevant research, Tab. 1 describes traditional prediction methods, including least squares support vector machine, multiple regression model, grey prediction model, BP neural network and combination models of the methods above [9][10][11][12]. It can be seen that compared to the multiple regression model and grey prediction model, both the least squares support vector machine (LSSVM) and BP neural network have advantages in predicting nonlinear data sequences. Although the multiple regression model could also apply mathematical expressions to alternate the correlation relationship with uncertainty between dependent and independent variables, further assumptions are needed for predicting nonlinear data sequences. Besides, the grey prediction model (GM) is able to predict the target data sequence with uncertainty through analyzing the correlation relationship between the development trends of system factors (influence factors of the target sequence).
Hence, this paper studies the time series prediction problem through pre-predicting the influence factors (of the target sequence). The main contributions are as follows. Firstly, this paper defines the pre-prediction approaches of influence factors (i.e, the factors driven approach and time driven approach). Secondly, according to the factors driven approach, a prediction method based on the LSSVM and GM (GM-LSSVM) is proposed. Thirdly, a case study on predicting China's economic growth is put forward using the statistics data among 2005 to 2014 from the national bureau. The influence factors of economic growth are also established through the grey correlation analysis theory. The numerical experiment results show that our proposed method GM-LSSVM has high precision in predicting the economic growth in practice. Use nonlinear mapping algorithm to transform the linearly inseparable samples in low-dimensional input space into linear separable ones in a highdimensional feature space, in order to construct classifiers.
Fit nonlinear samples well, have strong generalization ability, and do not rely on the distribution type of samples.

Multiple regression model
Use mathematical expressions to alternate the correlation relationship with uncertainty between dependent and independent variables.
Assumptions are needed for nonlinear problems.

Grey prediction model (GM)
Through analyzing the correlation relationship between the development trend of system factors, generate data sequences with strong regularity, and then establish and solve the differential equation model to complete the whole prediction process.
Be able to predict systems with uncertainty, but have more errors when doing medium or long-term prediction on data sequences which include abnormal points.

BP neural network
Through learning and storing a large number of input -output model mapping, use back propagation to constantly adjust the network weights and threshold, in order to minimize the sum of square error.
Fit nonlinear samples well with strong fault tolerance, but have local minimization problems and low convergence speed.
The remainder of the paper is organized as follows. In Section 2, after presenting the previous work related to our research (including the least squares support vector machine and grey model), we propose the prediction method GM-LSSVM. In Section 3, we conduct our experiment via the public dataset from 2005 to 2014 on the National Bureau of Statistics Data. The case study on predicting China's economic growth is also discussed. Finally, the paper is concluded in Section 4.

TIME SERIES PREDICTION BASED ON THE LSSVM AND GM 2.1 Pre-prediction Approaches for Time Series Prediction
According to the relationship between influence factors and the target sequence, the pre-prediction of target sequence could be divided into two different approaches (see Fig. 1): (1) Time driven approach (see Fig. 1a). The time driven approach utilizes the pre-predicted value of the target sequence at every historical time period to predict the target value. (2) Factors driven approach (see Fig. 1b). The factors driven approach utilizes the pre-predicted value of all the influence factors at the target time period to predict the target value.
In Fig. 1a, Y is the target object, Xi (i = 1,2, …, n) is the influence factor of Y, and the shadow areas on the left side represent the known data under the historical time period (1,2, …, t), while the white areas on the right side represent the unknown data under the target time period t + 1, thus y t + 1 is the target prediction value. The time driven approach will firstly pre-predict the target sequence at every historical time period (y 1 , y 2 , y 3 , …, y t ) using the influence factors (x i1 , x i2 , x i3 , …, x it ,) (i = 1, 2, …, n), and then predict the target value y t + 1 using the pre-prediction results (y 1 , y 2 , y 3 , …, y t ). If the real data of target object Y at every historical time period is available, that is (y 1 , y 2 , y 3 , …, y t ) is known data, the time driven approach could directly obtain the final result y t+1 through omitting the prepredicting process.

Time Series Prediction Method Bases on the GM-LSSVM
Since there is much uncertainty in the collection of time series data among practical business scenarios, especially the economic data [14,15], the least squares support vector machine (LSSVM) and grey prediction model (GM) are applied for the time series prediction (see Tab. 1).
The support vector machine model can well fit the nonlinear relationship between economic growth and its influence factors, through combining the structural risk minimization principle with prior risk and incredible risk,. Compared with other prediction methods, LSSVM has incomparable advantages on solving the finite sample, high dimension, local minimum and nonlinear problems [16,17]. Moreover, the system which is consisted of economic growth and its influence factors is a grey system with great uncertainty, so it is difficult and unnecessary to find all the elements effecting economic growth. GM series model is the basic model of grey prediction theory, and takes the uncertainty system with little data as the research object. Especially the average GM (1, 1) model proposed by Deng. J has a wide range of application [18,19].
Thus, focusing on the factors driven approach (see Section 2.1), this paper proposes the time series prediction method based on least squares support vector machine and grey prediction model (GM-LSSVM). The steps are as follows: Input: 1) The time series of influence factors Xi(t), (i = 1, 2, …, n; t = 1, 2, …, M); 2) The time series of target object Y(t), (t = 1, 2, …, M); 3) The division proportion of training set and testing set μ.
Step 1: Calculate the prediction value î X (t + Δt) if the influence factor X i (t) after Δt period of time prediction method.
i. Set the original sequence of influence factors as ii. Establish the differential equation ( [ ] ( ) . Get the prediction model through calculating the differential equation above via parameter , following Eq.
ii. Construct the regression function following Eq. (5): where, Moreover, the purpose of the transformation above is to map the input space R n to the high-dimensional feature space H via the nonlinear mapping function ( ) ϕ′ * , and construct the optimal decision function among H. In (5), ε is the adjustment coefficient, θ is the deviation value and R ∈ , λ r is the Lagrange operators and r R λ ∈ .
iii. Select the kernel function. The kernel function and kernel parameter determine the prediction precision. The commonly used kernel functions are RBF kernel function, polynomial kernel function, and linear kernel function etc. a) RBF kernel function where, σ 2 is the width of the kernel function. b) Polynomial kernel function This paper selects the RBF function as the kernel function with strong generalization ability.
Step 3: Predict the target value after Δt period of time via the trained model and the prediction results of influence factors.
From the steps of GM-LSSVM method, it can be seen that the index system of influence factors has a great importance for the accuracy of prediction results. The ideal index system should be closely related to the target object, rather than find all the influence factors. For example, in the economic context, it is impossible to find all the influence factors of economic growth. Thus, this research will establish the influence factors of economic growth based on grey correlation analysis theory.
Julong Deng published the grey control system which marks the start of grey system theory [20]. Grey correlation analysis model establishes the correlation degree between elements through the similarity of curve geometry. According to the quantitative analysis, grey correlation degree can be calculated after comparing the geometrical relationship among time series. The comparative data series with the larger grey correlation degree is more similar to the reference data series on the developing direction and speed, and has the closer relationship with the reference sequence. The sample size of grey correlation analysis medal can be less than four. What's more, it can also be used for irregular data without the inconsistence between qualitative analysis results and quantitative results [21,22].
The steps of grey correlation analysis algorithm are as follows: Input: Time series; correlation coefficient ρ(ρ = 0.5). Output: Correlation degree.
Step 1: Set the analysis series. The reference series and comparative series should be set artificially. Reference series reflects the characteristic of the system behavior. Comparative series affects the behavior of the system. For as a mother sequence, and the rest of the series are subsequences.
Step 2: Initialize the original data. Because various system data may have different dimensions, it is difficult to get the correct conclusion when comparing with each other. So the grey correlation analysis algorithm generally must carry on the dimensionless processing of original data via Eq. (9).
Step 3: Calculate the absolute difference between each subsequence and mother sequence via Eq. (10).
Step 5: Calculate correlation degree. Because the correlation coefficient is the correlation value between comparative series and reference series in each moment, every point in the curve, so its number is more than one. The information is too scattered to compare. Thus, it is necessary to concentrate the correlation coefficient in every moment, each point in the curve, to a value through calculating its average. The average is named as correlation degree between comparative series and reference series. The correlation formula is as follows: where, t = 1, 2, …, n; i = 1, 2, …, s; j = 1, 2, …, t.
Step 6: analyze correlation degree. According to the order of correlation degree, the larger correlation degree is, the greater the index affects the mother series. Last, get the final results through analyzing the mean and variance of the correlation degree.

Experiment Design and Data Preparation
In this section, a case study on the economic growth prediction is put forward to verify the accuracy and effectiveness of the proposed method GM-LSSVM.
Economic growth usually refers to the continuous increase of output per capita (or income per capita) among a time span, which is the focus of all countries [1,2,33]. Through qualitative analysis, it can be found that economic growth is influenced by system innovation, technological innovation and industrial structure adjustment. The correlation of them is shown in Fig. 2 [3]. System innovation provides the development environment for technological innovation, while technological innovation leads to the changes of system and rules. What's more, the results of system innovation and technological innovation will contribute to the adjustment of industrial structure. Because the industrial structure is stable in the short term, its effect on both others is relatively small [4-7, 31, 32]. So this paper mainly focuses on the influence of system and technological innovation on the economic growth.
Since economic growth is mainly affected by system and technological innovation, appropriate indicators should be selected to measure the economic growth, system innovation and technological innovation. According to the data released on the national bureau of statistics and related research, the following indexes in Tab. 2 are chosen as our initial index system [13,23,24]. This paper uses GDP to describe economic growth. GDP cannot only measure the overall national output and income scale, but also economic fluctuations and economic cycle state on the whole [25].
Number of published papers (X11), published works (X 12 ) and authorized patents (X 13 ) reflect the national technological innovation level directly, while the high-tech products exports(X 14 ) and technical market turnover(X 15 ) reflect the developing progress of the technology innovation indirectly.
Taxes (X 21 ), Money supply (X 22 ), CPI (X 23 ), Foreign exchange reserve (X 24 ) and RMB exchange rate (X 25 ) are the indexes which are completely controlled by the state system; conversely the results of system innovation will ultimately reflect on these five indexes. So we chose the five indexes above as indicators describing system innovation.  13 Number of authorized patents X 14 The high-tech products exports (10 8 RMB) X 15 Technical market turnover (10 8 RMB) X 21 Taxes (10 8 RMB)

Figure 2 Three aspects that influence economic growth
Describing system innovation(X 2 ) X 22 Money supply (10 8 RMB) X 23 CPI X 24 Foreign exchange reserve (10 8 RMB) X 25 RMB exchange rate Tab. 2 contains 1 target object and 10 influence factors. There are two problems in the indicators selection process: (1) if we directly conduct the prediction with this 11 indicators, it will lead to large amounts of calculation, which affects the efficiency of LSSVM method [26]; (2) the selection of indicators is subjective, which can ensure the selected indexes and all the influence factors of economic growth to belong to a sufficient relationship rather than necessary. Thus, it is significant to keep the chosen influence factors having a close relationship to economic growth with less number via reasonable streamlining the index system. The grey correlation analysis method in section 2.3 is able to accomplish this demand through seeking relevance to replace integrity.
According to the national bureau of statistics data, we got the original data of the 11 indexes above around 10 years (2005 -2014) shown in Tab. 3.
It can be seen that in Tab. 3 there are great differences among the range of 11 indexes, so we should conduct the data preprocessing job first based on grey correlation analysis method and the initialization result is shown in Fig.  3. Fig. 3 shows that each index overall has a rising trend after preprocessing, but the rising rate and amplitude are quite different.
After preprocessing, taking Y (GDP) as the mother sequence and other influence factors as subsequences, calculate the correlation degree between GDP and each influence factor in Tab. 4 via the results of absolute difference and correlation coefficient.

Figure 3
The preprocessing results Fig. 4 shows the sorting results of correlation degree in Tab. 4. It can be found that from the left to the right there are two obvious drops among the correlation curves on the whole, which can divide influence factors into four levels: Level 1 consists of X 22 , X 14 with E(R x22 , R x14 ) = 0.9; Level 2 consists of X 21 , X 15 , X 13 with E(R x21 , R x15 , R x13 ) = 0.8; Level 3 consists of X 11 with E(R x11 ) = 0.7; Level 4 consists of X 24 , X 12 , X 23 , X 25 with E(R x24 , R x12 , R x23 , R x25 ) = 0.6. Since the average correlation degrees of Level 3 and 4 are too small, remove them from the index system. Moreover, the correlation degrees of X 15 , X 13 are quite similar in Level 2, only keep X 15 with larger degree. Above all, the final index system filtered by grey correlation analysis method only contains Money supply (X 22 ), the high-tech products exports (X 14 ), Taxes (X 21 ), and technical market turnover (X 15 ).
Tab. 5 shows the mean and variance of contribution degrees between system and technological innovation via Tab. 2. It can be seen that the mean of two-group contribution degree is similar, which means the influence of system innovation or technological innovation to economic growth is close. However, the variance of system innovation is much larger than the technological innovation. Thus, the results show that the contribution level of technological innovation to the economic growth is relatively stable, while reasonable system innovation will bring a surge in economic growth.  According to step 1, calculate the value of 4 influence factors based on the GM (1, 1) prediction method via the original data in previous seven years. For verification purpose，this experiment only completes the prediction for the next three years. Tab. 6 shows the value of influence factors calculated by the GM (1, 1), and uses the relative error ((prediction value − actual value)/actual value) to measure the accuracy of prediction results. We can figure out that all the average relative error is beyond 0.05, and the average relative errors of system innovation indexes X22 and X 21 are obviously higher than technological innovation indexes X 14 and X 15 . This is mainly because the uncertainty of innovation itself is big, and the subjectivity of system innovation is relatively stronger than technological innovation, which further verifies that only using the time series prediction model to predict data with strong uncertainty makes it hard to get ideal forecast results. According to step 2, train the LSSVM model by the original data of target object and its influence factors in previous seven years, and test the trained model by the data in last 3 years. The results are shown in Fig. 6, where Y exp represents the actual value of GDP, while Y pre represents the prediction value of GDP by LSSVM trained model. The relative error of trained LSSVM model is 0.014, which has reached good performance. In order to test the accuracy of prediction results, this paper applies these error measurements which are widely used: relative error  According to the error analysis in Tab. 8, the performance of GM-LSSVM on three error measurement is better than GM (1, 1), and the improvement of accuracy ((error of GM (1, 1)-error of GM-LSSVM)/error of GM (1, 1)) is all over 50%. What's more, combining the results in Tab. 6, the relative error of the final target object predicted by GM-LSSVM is 0.05, which is smaller than all the influence factors.
The high precision of the GM-LSSVM model is mainly caused by the following reasons: (1) GM (1, 1) adopts the ERM principle (empirical risk minimization) when doing parameters estimation, which leads to the low performance on the medium and long term prediction process with abnormal points. However, LSSVM is based on the SRM principle (structural risk minimization), which contributes to the feature of good nonlinear fitting, strong generalization ability, and independence of the sample distribution etc. So, GM-LSSVM combines two methods together by seeking compromise between complexity and generalization, and obtains better prediction results [28].  1) is to make the trend feature of time series more apparent through accumulating the original data, which improves the prediction efficiency [29,30]. Thus, for the prediction problem, that target object time series and influence factors time series both have big leaps, separately dealing with the trend feature of individual time series and then reasoning out the whole value is better than directly dealing with the trend feature of the whole time series in the final prediction effect. (The definition of data leap characteristics and optimal prediction performance law of individuals with leaps will be presented in detail through another paper.)

CONCLUSIONS
According to the prediction problem of economic growth, this paper achieves the following results: (1) put forward the GM-LSSVM prediction model, which integrated the input-output prediction model and the time series prediction model together. GM-LSSVM overcomes the difficulty that LSSVM must know the value of influence factors first in order to predict the target object in a certain period of time. At the same time, the experiment results show that compared with GM (1, 1), the improvement of prediction accuracy on GM-LSSVM is more than 50%. (2) simplify the index system of influence factors through grey correlation analysis method, and gain the final prediction results of China's economic growth, whose precision has reached 95%.
The future research will mainly focus on increasing the accuracy of the GM-LSSVM prediction model. Although GM-LSSVM has achieved good prediction results on multiple error performance, there is still some room to improve. Moreover, more comparison experiments on the real economic data for different time periods will also be conducted using the GM-LSSVM to study the tendency of economic development.