• Journal of the Korean Institute of Telematics and Electronics B
• /
• v.28B no.11
• /
• pp.852-858
• /
• 1991

An adaptive hierarchical classifier(AHCL) for Korean character recognition using a neural net is designed. This classifier has two neural nets: USACL (Unsupervised Adaptive Classifier) and SACL (Supervised Adaptive Classifier). USACL has the input layer and the output layer. The input layer and the output layer are fully connected. The nodes in the output layer are generated by the unsupervised and nearest neighbor learning rule during learning. SACL has the input layer, the hidden layer and the output layer. The input layer and the hidden layer arefully connected, and the hidden layer and the output layer are partially connected. The nodes in the SACL are generated by the supervised and nearest neighbor learning rule during learning. USACL has pre-attentive effect, which perform partial search instead of full search during SACL classification to enhance processing speed. The input of USACL and SACL is a directional edge feature with a directional receptive field. In order to test the performance of the AHCL, various multi-font printed Hangul characters are used in learning and testing, and its processing its speed and and classification rate are compared with the conventional LVQ(Learning Vector Quantizer) which has the nearest neighbor learning rule.

• Proceedings of the KAIS Fall Conference
• /
• 2003.11a
• /
• pp.84-87
• /
• 2003

This paper proposes the enhanced REF network, which arbitrates learning rate and momentum dynamically by using the fuzzy system, to arbitrate the connected weight effectively between the middle layer of REF network and the output layer of REF network. ART2 is applied to as the learning structure between the input layer and the middle layer and the proposed auto-turning method of arbitrating the learning rate as the method of arbitrating the connected weight between the middle layer and the output layer. The enhancement of proposed method in terms of learning speed and convergence is verified as a result of comparing it with the conventional delta-bar-delta algorithm and the REF network on the basis of the ART2 to evaluate the efficiency of learning of the proposed method.

• Journal of the Korean Institute of Telematics and Electronics B
• /
• v.31B no.8
• /
• pp.211-218
• /
• 1994

In this paper we develope a hybrid learning rule to improve the robustness of multi-layer Perceptions. In most neural networks the activation of a neuron is deternined by a nonlinear transformation of the weighted sum of inputs to the neurons. Investigating the behaviour of activations of hidden layer neurons a new learning algorithm is developed for improved robustness for multi-layer Perceptrons. Unlike other methods which reduce the network complexity by putting restrictions on synaptic weights our method based on error-backpropagation increases the complexity of the underlying proplem by imposing it saturation requirement on hidden layer neurons. We also found that the additional gradient-descent term for the requirement corresponds to the Hebbian rule and our algorithm incorporates the Hebbian learning rule into the error back-propagation rule. Computer simulation demonstrates fast learning convergence as well as improved robustness for classification and hetero-association of patterns.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.7 no.5
• /
• pp.1044-1051
• /
• 2003

This paper surveys the EBP(Error Back Propagation) learning, the Cross Entropy function and the LBL(Layer By Layer) learning, which are used for learning the MLP(Multi Layer Perceptrons). We compare the merits and demerits of each learning method in the handwritten digit recognition. Although the speed of EBP learning is slower than other learning methods in the initial learning process, its generalization capability is better. Also, the speed of Cross Entropy function that makes up for the weak points of EBP learning is faster than that of EBP learning. But its generalization capability is worse because the error signal of the output layer trains the target vector linearly. The speed of LBL learning is the fastest speed among the other learning methods in the initial learning process. However, it can't train for more after a certain time, it has the lowest generalization capability. Therefore, this paper proposes the standard of selecting the learning method when we apply the MLP.

• Journal of the Korean Institute of Telematics and Electronics B
• /
• v.28B no.12
• /
• pp.27-37
• /
• 1991

In this paper, the novel PLD (Preceding Layer Driven) MLP (Multi Layer Perceptron) neural network model and its learning algorithm is described. This learning algorithm is different from the conventional. This integer weights and hard limit function are used for synaptic weight values and activation function, respectively. The entire learning process is performed by layer-by-layer method. the number of layers can be varied with difficulty of training data. Since the synaptic weight values are integers, the synapse circuit can be easily implemented with CMOS. PLD MLP neural network was applied to English Characters, arbitrary waveform generation and spiral problem.

• Journal of the Korea Society of Computer and Information
• /
• v.21 no.1
• /
• pp.1-8
• /
• 2016

Convolutional neural network (CNN) consists of a few pairs of both convolution layer and subsampling layer. Thus it has more hidden layers than multi-layer perceptron. With the increased layers, the size of convolution mask ultimately determines the total number of weights in CNN because the mask is shared among input images. It also is an important learning factor which makes or breaks CNN's learning. Therefore, this paper proposes the best method to choose the convolution size and the number of layers for learning CNN successfully. Through our face recognition with vast learning examples, we found that the best size of convolution mask is 5 by 5 and 7 by 7, regardless of the number of layers. In addition, the CNN with two pairs of both convolution and subsampling layer is found to make the best performance as if the multi-layer perceptron having two hidden layers does.

• Journal of the Korea Society of Computer and Information
• /
• v.9 no.4 s.32
• /
• pp.33-39
• /
• 2004

In this paper. we design the learning algorithm of LVQ which is used Forward Counter Propagation Networks to improve classification performance of LVQ networks. The weights of Forward Counter Propagation Networks which is between input layer and cluster layer can be learned to determine initial reference vectors by using SOM algorithm and to learn reference vectors by using LVQ algorithm. Finally. pattern vectors is classified into subclasses by neurons which is being in the cluster layer, and the weights of Forward Counter Propagation Networks which is between cluster layer and output layer is learned to classify the classified subclass, which is enclosed a class. Also. kr the number of classes is determined, the number of neurons which is being in the input layer, cluster layer and output layer can be determined. To prove the performance of the proposed learning algorithm. the simulation is performed by using training vectors and test vectors that ate Fisher's Iris data, and classification performance of the proposed learning method is compared with ones of the conventional LVQ, and it was a confirmation that the proposed learning method is more successful classification than the conventional classification.

• 제어로봇시스템학회:학술대회논문집
• /
• 1990.10a
• /
• pp.36-40
• /
• 1990

Conventional back propagation learning is generally characterized by slow and rather inaccurate learning which makes it difficult to use in control applications. A new multilayer perception architecture and its learning algorithm is proposed that consists of a Kohonen front layer followed by a back propagation network. The Kohonen layer selects a subset of the hidden layer neurons for local tuning. This architecture has been tested on the inverse kinematic solution of robot manipulator while demonstrating its fast and accurate learning capabilities.

• The Transactions of the Korean Institute of Electrical Engineers
• /
• v.40 no.2
• /
• pp.224-233
• /
• 1991

The multilayer expanson of single layer NN (Neural Network) was needed to solve the linear seperability problem as shown by the classic example using the XOR function. The EBP (Error Back Propagation ) learning rule is often used in multilayer Neural Networks, but it is not without its faults: 1)D.Rimmelhart expanded the Delta Rule but there is a problem in obtaining Ca from the linear combination of the Weight matrix N between the hidden layer and the output layer and H, wich is the result of another linear combination between the input pattern and the Weight matrix M between the input layer and the hidden layer. 2) Even if using the difference between Ca and Da to adjust the values of the Weight matrix N between the hidden layer and the output layer may be valid is correct, but using the same value to adjust the Weight matrixd M between the input layer and the hidden layer is wrong. Recognitron III was proposed to solve these faults. According to simulation results, since Recognitron III does not learn the three layer NN itself, but divides it into several single layer NNs and learns these with learning patterns, the learning time is 32.5 to 72.2 time faster than EBP NN one. The number of patterns learned in a EBP NN with n input and output cells and n+1 hidden cells are 2**n, but n in Recognitron III of the same size. [5] In the case of pattern generalization, however, EBP NN is less than Recognitron III.

• Journal of the Korea Society of Computer and Information
• /
• v.16 no.8
• /
• pp.39-47
• /
• 2011

Ergezinger's method, one of the layer-by-layer algorithms used for multilyer perceptrons, consists of an output node and can make premature saturations in the output's weight because of using linear least squared method in the output layer. These saturations are obstacles to learning time and covergence. Therefore, this paper expands Ergezinger's method to be able to use an output vector instead of an output node and introduces a learning rate to improve learning time and convergence. The learning rate is a variable rate that reflects the correlation coefficient between new weight and previous weight while updating hidden's weight. To compare the proposed method with Ergezinger's method, we tested iris recognition and nonlinear approximation. It was found that the proposed method showed better results than Ergezinger's method in learning convergence. In the CPU time considering correlation coefficient computation, the proposed method saved about 35% time than the previous method.