Open Access Open Access  Restricted Access Subscription or Fee Access

Higher Order Neural Networks Learning by Extended Kalman Filter

Agya Mishra, R.N. Yadav, D.K. Trivedi

Abstract


The Extended Kalman filter (EKF) is well known as a state estimation method for a nonlinear system and has been used to train a multilayered neural network (MNN) by augmenting the state with unknown connecting weights. The EKF Neural networks training algorithm is superior to standard back-propagation algorithm, but it is also known that Higher Order Neural Networks (HONN) have better performance than a standard multilayer perceptron networks. In this paper, more robust new learning algorithm for a Higher Order Neural Networks (HONN) based on EKF is proposed. The algorithm is an EKF coupled with HONN model, and has a new algorithm used to approximate the uncertainty of the system extreme nonlinearities. HONN consists of Generalized mean Neuron model (GMN).The GMN consists of an aggregation function which is based on generalized mean of all the inputs applied to it. EKF-HONN model is used for non linear state estimation, and to determine the weights and error covariance, the one step ahead prediction. Simulation results show that the proposed new algorithm is quite effective and can be implemented in the field of adaptive filtering nonlinear state estimation and predictions in place of traditional methods.

Keywords


Higher Order Neural Networks, Generalized Mean-Neuron, Function Approximation, and Extended Kalman Filter, Nonlinear Filtering, Online Training.

Full Text:

PDF

References


Ronghui Zhan and Jianwei Wan - Neural Network-Aided Adaptive Unscented Kalman Filter for Nonlinear State Estimation, ieee signal processing letters, vol. 13, no. 7,july 2006, pp 445-448.

R.N. Yadav ,Prem K. Kalra, Joseph John-, Neural network learning with generalized-mean based neuron model, Soft computing, Springer-Verlag vol-10 no-3,pp 257-263 :Feb 2006.

Agya Mishra, R.N. yadav,D. K. Trivedi-noise cancellor based on Generalized neural network, IEEE inter. Conference on Advanced communication technologies proceedings ICACT 2007,S.Korea pp- pp- 1998-2000,feb.2007.

Simon Haykin -Kalman Filtering and Neural Networks, John Wiley & Sons, Inc.2001

A. Yadav, D. Mishra, R.N. Yadav, S. Ray, P.K.Kalra-Time series prediction with single integrate-and–fire neuron, Applied soft computing, science direct, Elsevier, pp,245-252,2006

Gintaras V. Puskorius and Lee A. Feldkamp-Parameter based kalman filter training theory and implementation ,Kalman filtering and neural network, S. Haykin, wiley-iterscience pub,pp23-68,2001.

John Sum,Chi-Sing Leung, Gilbert,H. Young and Wing-kay Kan-On the Kalman Filtering Method in Neural network training and pruning, IEEE transuction on neural networks vol-10,no.1, pp 161-166,January 1999.

Felix Heimes, Extended kalman filter neural network Training: Experimental results and algorithm Improvements,0-7803-4778-1/98,IEEE,pp 1639-1644,1998.

Youji Iiguni, Hideaki Sakai and Hidekatsu Tokumaru,- A real-time learning algorithm for multilayered neural network based on the extended kalman filter,IEEE Transaction on signal processing vol.40,no4,pp 959-966,April 1992.

Ronald J. Williams,- Training Recurrent Networks using the Extended Kalman Filter,0-7803-0559-0/92,IEEE, IV,pp 241-246, 1992.

W.K.T. Fukuda and S.G. Tzafestas, Learning algorithm of layered neural networks via extended kalman filters, Int.J.Syst.Sci vol22, no.4, pp 753-768, 1991.

S.Singhal and L.Wu, Training Multilayered Perceptron with the Extended Kalman algorithm, in D.S. Tourelzky(ed) ,Advances in Neural Information Processing system I,pp133-140,1989.

KDeergha , Rao, M.N.S.Swamy, E.I.Plotkin -complex ekf neural network for adaptive equalization , IEEE International Symposium on Circuits and Systems ISCAS 2000, , 28-31, Geneva, Switzerland, pp-11-349-352, May 2000.

Herbert Jaeger A tutorial on training recurring neural networks, covering BPPT,RTRL,EKF and the „echo state network approach‟, International university of Bermen,pp 17-33,2005.

R. N. Lobbia, S. C. Stubberud, and M. W. Owen, “Adaptive extended Kalman filter using artificial neural networks,” Int. J. Smart Eng. Syst. Des., vol. 1, pp. 207–221, 1998.

Iulian B. Ciocoiu ,RBF network training using a dual extended kalman filter, Neurocomputing 48, Elsevier,pp-602-622,2002.

Dan Simon, Training radial basis neural networks with the extended kalman filter, Neurocomputing 48, Elsevier, pp 455-475,2002.

A. Alessandri, M. Cuneo, S. Pagnan, M. Sanguineti, On the convergence of EKF-based parameters optimization for neural Networks, proceeding IEEE conference on decision and control USA, pp 6181-6186, December 2003.

Greg Welch, Gary Bishop, An introduction to kalman filter ,Department of computer science university of N. carolina at Chapel Hill, pp 1-16,2006

Michael C. Nechyba and Yangsheng Xu, Cascade Neural Networks with Node-Decoupled Extended Kalman Filtering, IEEE,pp 214-218, Pitsburgh, 1997.

Chuan Wang and Jose C. Principe, Training Neural Networks with Additive Noise in the Desired Signal, tutorials 1999.

Herbert Jaeger A tutorial on training recurrent neural networks, covering BPPT , RTRL, EKF and the echo state network approach international university Bremen ,2005.

L.A Feldkamp and G.V. Puskorius, A signal processing frame work based on dynamic neural networks with application to problems in adaptation, filtering and classification, proceedings of the IEEE,86,pp 2259-2277,1998.


Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.