Context sensitive neural network by overlapped systems

Abstract

The use of context dependency in neural networks is an important issue in many cognitive situations. In this report we introduce a novel context dependent neural network model based on overlapped multi-neural network structures. We present a detailed study about contextual features and some of its applications in neural networks. We also present some different strategies for applying overlapping in neural networks. The generalization ability of a neural network is mainly influenced by three factors, the number and performance of the learning data samples, the complexity of the learning algorithm employed, and the network size. Neural network overlapping is one of the practical techniques of achieving a better generalization and recognition rate. This is due to its ability of decreasing the number of free weights of a neural network and providing less complexity of the neural network function. For this purpose overlapped neural networks have been used in feed-forward neural networks (MFNN) , self organizing maps (SOM) and in shared weight neural networks (SWNN). Overlapped neural networks also have the ability of performing a function localization over the neural network feature space. Among the feature space of any problem, three different types of features (from the relevance point of view ) can be distinguished: primary, contextual, and irrelevant features. Researches in the contextual features are mainly concerned with two issues. Identifying such contextual features, and managing them. We are presenting the strategy of identifying these context-sensitive features and five basic strategies for managing them. We are also presenting a context sensitive model for overcoming the slow convergence problems, and a context dependent (cd) neuron model that is considered a generalization of the traditional neuron model. We introduce a novel approach for problems regardless of sufficiency or accuracy of their historical observations or lab simulation data. Our approach is based on imposing a context of problem performance metrics into networks and gaining the enhancement towards its satisfactory state. We use an overlapped system of back propagation neural networks for our purpose. A main neural network is responsible for mapping input and output relation while a regulatory neural network evaluates the performance metrics satisfaction. We provide special training and testing algorithms for the overlapped system that guarantees a synchronized solution for both neural networks. An example of traffic control problem is simulated. The result of simulation shows a great enhancement of the solution using our approach.