A Study on the Prediction Model of BP Neural Network quasi―Newton Method

开篇：润墨网以专业的文秘视角，为您筛选了一篇A Study on the Prediction Model of BP Neural Network quasi―Newton Method范文，如需获取更多写作素材，在线客服老师一对一协助。欢迎您的阅读与分享！

Abstract. Based on the main factors affecting the scale of higher education， this paper sets up a suitable prediction model for the development of the higher education scale. Using the actual data to verify the model， and compare it with the traditional statistical models. It is proved that BFGS* of bp neural network has higher accuracy and stronger utility.

Key words： BFGS， BP neural network， prediction， the scale of Higher Education.

1. Introduction

Since the reform and opening up in China， the higher education has made the outcome attract people's attention. Nowadays， the enrolled students scale has been ranked first all around the world， which has entered the stage of mass higher education. From 1978 to 2006， the scale of students increased from 1.323 billion to 23.742 billion， which has an increase of 17.9 times. As the higher education in cultivating a large number of talents for the society， it also provides many opportunities for job training， job-transfer training， short-term occupation technical training and continuing education for the society that made important contributions for the development of national quality improvement of the population， social economy. The fast development of China's social， economic， gave birth to many new occupation， new jobs to those professional and technical personnel， management personnel and all kinds of personnel demand. Showing a strong momentum of growth and providing a broad space for the development of higher education. Therefore， focus on the reasonable growth of education scale anticipation will become not only the key development factors， but an important policy basis of population quality and social economic development.

On the other hand， psychologist W. S. McCulloch and mathematician W. Pitts proposed M-P model since 1943， which artificial neural network research has experienced from rise to depression. And in recent years， the research of artificial neural network is developing to a mature stage， and its contribution has great development on neural network studies. The present researches focus on network weight learning algorithm， error function， network structure and its related convergence and stability. Moreover， the theory has proved that for any closed interval， a continuous function can be used BP neural network with one hidden layer to approximation， and a three layer BP neural network can be applied to nonlinear system identification. People in general learning algorithm of BP network is widely used in the power of the steepest descent method. Although the algorithm has global convergence， it is only linear convergence. So it has a slow convergence speed and a longer time of network training application. Some people applied the conjugate gradient method to exam BP network weight training， but this algorithm for the search direction is generated in the iterative process， which is at best approximate conjugate direction. It can improve the network training speed， but its result is not ideal. Therefore， this paper utilizes BFGS as a function of BP neural network training function， which can solve the problem effectively.

2. Neural network model

With the increase of the science and technology in the economic construction and social informatization， the spotlight of enterprise development strategic has already shifted to the cultivation of the core competitiveness gradually. So that enterprises pay more attention to the sustainable development of the allocation of human resources development， the enterprise marketing and production operation， the high degree， practical ability and innovative thinking personnel continue to increase. With the change of the social demand for talents training in Colleges and universities， where must put the record of formal schooling， discipline and the upgrading of the industrial structure， the development of science and technology and the change of employment structure up to co-ordinate. Therefore， to construct an appropriate nonlinear function face lots of difficulty. Aiming at this problem， this paper uses the BP neural network to forecast the scale of higher education.

It is clearly that the multilayer feed-forward neural network based on error back propagation （BP neural network） is the most widely used neural network model， and it is the most successful one. That’s because it has many advantages， for example， the parallel processing， distributed storage and fault tolerance of the superiority of the neural network model are structurally， self-learning， self-organizing and self-adaptive.

3. The empirical analysis

This paper establishes a nonlinear mapping model of education scale structure with three layers BP neural network. The inputs of the model are the numbers of national college students from 2004 to 2008， and the target vector is the numbers of the national college students in 2009. Through repeated training the neural network， it ultimately determines the number of hidden layer is nodes six， the neural network topology structure as shown in figure 1. （All the data come from Zhonghong database）

It is going to use Matlab software as programming tool. The setting parameters are， the maximum number of training is 1000 times， the learning rate is 0.05， it shows the iterative training process is 2000， training requirements for the longitude is 0.0001. Other values are by default matlab neural network toolbox value.

Using the number of higher education students in table 1 from 2004 to 2008 as input vector， with the number of higher education students in 2009 as the target vector. It is used for training the neural network and the BFGS as training function， then obtain the predictive value， compared with the actual value， the result of training error as shown in table 2.

From table 2， it is obvious that using a quasi-newton method as the BP neural network training function， which can be used to predict the scale of China's higher education more effectively. Additionally， it predicts the most area of our country values also more accurate. Only Tibet and Qinghai have larger prediction error. This is mainly because of higher education in Tibet and Qinghai is still in a relatively backward stage， the scale of higher education did not acquire enough development. So the neural network can not make an accurate prediction of the. And for the other provinces， the neural network can be used to make a better prediction. Therefore， it is proposes that the use of quasi-Newton method as the BP neural network training function can be used to predict the education in china successfully.

4. Performance analysis of neural network prediction

As shown in Figure 2， there are total 15 times iterative in neural network， when the iteration to the ninth step， the best check values intersect with the best value， as shown in the circle. From the figure that， when the neural network iterative to the fifteenth step， training value， testing value and verification value converges to the optimal value， therefore， the system iteration to fifteenth step can obtain the best results.

Figure 3 shows that when the neural network iterative to fifteenth step， the gradient convergence goes to the minimum value， and the checksum value also after the 6 checks achieve the best value， the step also converges to the minimum value.

As shown in Figure 4， the neural network constructs all input vectors as training value of training the network. Then remove part of data validation， and take another part of the data for testing. As shown in Figure 4， the training results， the test results and the results are superior. For the final outcome， the distribution of data even in fitting line on both sides， it shows that use quasi-Newton method as the BP neural network training function can forecast the scale of higher education in China， and the prediction results can reflect the actual situation perceptibly.

5. Conclusion

Based on BFGS of BP neural network to predict the scale of higher education in China， it uses its self-learning of BP neural network to analyze. During the training process， the weight value has to be modified frequently， so that the network actual output vector is close to the expected output value. Finally， through the example analysis of neural network weight matrix， the weights and thresholds of the network with the target vector and constantly revised， which can get the predictive value， the error of the predicted value and the target vector in an acceptable range. So， it is going to draw a conclusion that is by using BFGS of BP neural network to predict the scale of higher education can make more accurate evaluation of the educational industry.

References

[1] Wang Yongyi， Xujian. Neural network control [M]. Beijing： Mechanical Industry Press， 1999： 14-14

[2] Hechi- Nielsen R . Theory of t he back propagate ion neural net -work [J]. Proc of IJCNN， 1989（11）： 593- 603

[3] Gross berg S A . Nonlinear neural n et work： Principles， machines and architectures [J]. Neural Net works， 1998（1）：15-57

[4] Hunt K J. Sbarbaro D. Zbikowshi R et al. Neural networks for control systems-----A survey. Automatic， 1992， 28（6）： 1083-1111

[5] Chen S， BILLING S A . Neural network for nonlinear dynamic system modeling and identification [J]. Int Journal of Control， 1992， 56（ 2） ：319- 346.

[6] Malik M. The early restart algorithm [J]. Neural Computation， MIT， 2000（ 12） ： 1303- 1312.

[7] Chang K S， ABELG. A universal neural net with guaranteed convergence to zero system error [J]. IEEE Trans. Signal Processing， 1992， 40（ 12） ： 3022- 3031.

[8] He Shudong. On the research of multilayer feed forward neural network structure of [J]. Control theory and applications， 1998 （4）： 313- 318

[9] Zhang GuoPing. Anatomy of G-Combined Forecasting [J]. Forecast， 1988（5）：24-27

[10] Yuan Yaxiang. Nonlinear programming algorithm [M] Shanghai： Shanghai science and Technology Press， 1992