KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Research on a handwritten character recognition algorithm based on an extended nonlinear kernel residual network

  • Rao, Zheheng (Hubei Key Laboratory for High-efficiency Utilization of Solar Energy and Operation Control of Energy Storage System, Hubei University of technology) ;
  • Zeng, Chunyan (Hubei Key Laboratory for High-efficiency Utilization of Solar Energy and Operation Control of Energy Storage System, Hubei University of technology) ;
  • Wu, Minghu (Hubei Key Laboratory for High-efficiency Utilization of Solar Energy and Operation Control of Energy Storage System, Hubei University of technology) ;
  • Wang, Zhifeng (School of Educational Information Technology, Central China Normal University) ;
  • Zhao, Nan (Hubei Key Laboratory for High-efficiency Utilization of Solar Energy and Operation Control of Energy Storage System, Hubei University of technology) ;
  • Liu, Min (Hubei Key Laboratory for High-efficiency Utilization of Solar Energy and Operation Control of Energy Storage System, Hubei University of technology) ;
  • Wan, Xiangkui (Hubei Key Laboratory for High-efficiency Utilization of Solar Energy and Operation Control of Energy Storage System, Hubei University of technology)
  • Received : 2017.06.01
  • Accepted : 2017.09.16
  • Published : 2018.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

Although the accuracy of handwritten character recognition based on deep networks has been shown to be superior to that of the traditional method, the use of an overly deep network significantly increases time consumption during parameter training. For this reason, this paper took the training time and recognition accuracy into consideration and proposed a novel handwritten character recognition algorithm with newly designed network structure, which is based on an extended nonlinear kernel residual network. This network is a non-extremely deep network, and its main design is as follows:(1) Design of an unsupervised apriori algorithm for intra-class clustering, making the subsequent network training more pertinent; (2) presentation of an intermediate convolution model with a pre-processed width level of 2;(3) presentation of a composite residual structure that designs a multi-level quick link; and (4) addition of a Dropout layer after the parameter optimization. The algorithm shows superior results on MNIST and SVHN dataset, which are two character benchmark recognition datasets, and achieves better recognition accuracy and higher recognition efficiency than other deep structures with the same number of layers.

Keywords

References

  1. LeCun Y, Bengio Y, and Hinton G., "Deep learning," Nature, 521(7553), 436-444, 2015. https://doi.org/10.1038/nature14539
  2. Zhang X Y, Bengio Y, Liu C L., "Online and offline handwritten Chinese character recognition: A comprehensive study and new benchmark," Pattern Recognition, 61, 348-360, 2017. https://doi.org/10.1016/j.patcog.2016.08.005
  3. Liu C L, Yin F, Wang D H, et al., "Online and offline handwritten Chinese character recognition: benchmarking on new databases," Pattern Recognition, 46(1), 155-162, 2013. https://doi.org/10.1016/j.patcog.2012.06.021
  4. Foggia P, Percannella G, Vento M., "Graph matching and learning in pattern recognition in the last 10 years," International Journal of Pattern Recognition and Artificial Intelligence, 28(01), 2014.
  5. Zhang L, Tan J, Han D, et al., "From machine learning to deep learning: progress in machine intelligence for rational drug discovery," Drug Discovery Today, 1680-1685, 2017.
  6. Bottou L., "From machine learning to machine reasoning," Machine learning, 94(2), 133-149, 2014. Article (CrossRef Link). https://doi.org/10.1007/s10994-013-5335-x
  7. Yang J, Jiao Y, Xiong N, et al., "Fast Face Gender Recognition by Using Local Ternary Pattern and Extreme Learning Machine," KSII Transactions on Internet & Information Systems, 7(7), 2013.
  8. Zheng Y, Liu J, Liu H, et al., "Integrated Method for Text Detection in Natural Scene Images," KSII Transactions on Internet & Information Systems, 10(11), 2016.
  9. Ouellet S, Michaud F., "Enhanced automated body feature extraction from a 2D image using anthropomorphic measures for silhouette analysis," Expert Systems with Applications, 270-276, 2017.
  10. Yi C, Tian Y., "Scene text recognition in mobile applications by character descriptor and structure configuration," IEEE transactions on image processing, 23(7), 2972-2982, 2014. https://doi.org/10.1109/TIP.2014.2317980
  11. Yin X C, Yin X, Huang K, et al., "Accurate and robust text detection: A step-in for text retrieval in natural scene images," Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. ACM, 1091-1092, 2013.
  12. Luo H, Zhao F, Chen S, et al., "A Tree Regularized Classifier--Exploiting Hierarchical Structure Information in Feature Vector for Human Action Recognition," KSII Transactions on Internet & Information Systems, 11(3), 2017.
  13. Delaye A, Liu C L., "Contextual text/non-text stroke classification in online handwritten notes with conditional random fields," Pattern Recognition, 47(3), 959-968, 2014. https://doi.org/10.1016/j.patcog.2013.04.017
  14. Li W, Zhang T, Zhu Z, et al., "Detection of LSB Matching Revisited Using Pixel Difference Feature," KSII Transactions on Internet & Information Systems, 7(10), 2013.
  15. Mussarat Y, Muhammad S, Sajjad M, et al., "Content based image retrieval using combined features of shape, color and relevance feedback," KSII Transactions on internet and information systems, 7(12), 3149-3165, 2013. https://doi.org/10.3837/tiis.2013.12.011
  16. Shrivastava V K, Londhe N D, Sonawane R S, et al., "Exploring the color feature power for psoriasis risk stratification and classification: A data mining paradigm," Computers in biology and medicine, 65, 54-68, 2015.
  17. Kim S, Yu Z, Kil R M, et al., "Deep learning of support vector machines with class probability output networks," Neural Networks, 64, 19-28, 2015. https://doi.org/10.1016/j.neunet.2014.09.007
  18. Wong W K, Cheung D W, Kao B, et al., "Secure kNN computation on encrypted databases," Proceedings of the 2009 ACM SIGMOD International Conference on Management of data. ACM, 139-152, 2009.
  19. Aisha A, Muhammad S, Hussain S J, et al., "Face recognition invariant to partial occlusions," KSII Transactions on Internet and Information Systems, 8(7), 2496-2511, 2014. https://doi.org/10.3837/tiis.2014.07.017
  20. Chu J, Liang H, Tong Z, et al., "Slow Feature Analysis for Mitotic Event Recognition," KSII Transactions on Internet & Information Systems, 11(3), 1670-1683, 2017. https://doi.org/10.3837/tiis.2017.03.023
  21. He K, Zhang X, Ren S, et al., "Deep residual learning for image recognition," in Proc. of Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on, 27-30 June 2016.
  22. Huang G, Sun Y, Liu Z, et al., "Deep networks with stochastic depth," European Conference on Computer Vision. Springer International Publishing, 646-661, 2016.
  23. Luo X, Shen R, Hu J, et al., "A Deep Convolution Neural Network Model for Vehicle Recognition and Face Recognition," Procedia Computer Science, 107, 715-720, 2017. https://doi.org/10.1016/j.procs.2017.03.153
  24. Lee S J, Yun J P, Koo G, et al., "End-to-end recognition of slab identification numbers using a deep convolutional neural network," Knowledge-Based Systems, 132, 1-10, 2017. https://doi.org/10.1016/j.knosys.2017.06.017
  25. Kawano Y, Yanai K., "Food image recognition with deep convolutional features," in Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. ACM, 589-593, 2014.
  26. Girshick R., "Fast r-cnn," in Proc. of the IEEE International Conference on Computer Vision, 1440-1448, 2015.
  27. Kumar M, Jindal M K, Sharma R K., "Segmentation of isolated and touching characters in offline handwritten Gurmukhi script recognition," International Journal of Information Technology and Computer Science (IJITCS), 6(2), 58-63, 2014. https://doi.org/10.5815/ijitcs.2014.02.08
  28. Kamble P M, Hegadi R S., "Handwritten Marathi character recognition using R-HOG Feature," Procedia Computer Science, 45, 266-274, 2015. https://doi.org/10.1016/j.procs.2015.03.137
  29. Liang M, Hu X., "Recurrent convolutional neural network for object recognition," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 3367-3375, 2015.
  30. Schmidhuber J., "Deep learning in neural networks: An overview," Neural Networks, 61, 85-117, 2015. https://doi.org/10.1016/j.neunet.2014.09.003
  31. Zhang Y, Shi B M., "Improving pooling method for regularization of convolutional networks based on the failure probability density," Optik-International Journal for Light and Electron Optics, 145, 258-265, 2017.
  32. Affonso C, Rossi A L D, Vieira F H A, et al., "Deep learning for biological image classification," Expert Systems with Applications, 85, 114-122, 2017. https://doi.org/10.1016/j.eswa.2017.05.039
  33. LeCun Y, Bottou L, Bengio Y, et al., "Gradient-based learning applied to document recognition," in Proc. of the IEEE, 86(11), 2278-2324, 1998. https://doi.org/10.1109/5.726791

Cited by

  1. Maximum Likelihood Multi-innovation Stochastic Gradient Estimation for Multivariate Equation-error Systems vol.16, pp.5, 2018, https://doi.org/10.1007/s12555-017-0538-8
  2. Data Filtering Based Multi-innovation Gradient Identification Methods for Feedback Nonlinear Systems vol.16, pp.5, 2018, https://doi.org/10.1007/s12555-017-0596-y
  3. Recursive identification for multivariate autoregressive equation-error systems with autoregressive noise vol.49, pp.13, 2018, https://doi.org/10.1080/00207721.2018.1511873
  4. Particle Swarm Optimization Iterative Identification Algorithm and Gradient Iterative Identification Algorithm for Wiener Systems with Colored Noise vol.2018, pp.None, 2018, https://doi.org/10.1155/2018/7353171
  5. Adaptive Gradient-Based Iterative Algorithm for Multivariable Controlled Autoregressive Moving Average Systems Using the Data Filtering Technique vol.2018, pp.None, 2018, https://doi.org/10.1155/2018/9598307
  6. Iterative Identification Algorithms for Bilinear-in-parameter Systems by Using the Over-parameterization Model and the Decomposition vol.16, pp.6, 2018, https://doi.org/10.1007/s12555-017-0659-0
  7. Hierarchical Newton and least squares iterative estimation algorithm for dynamic systems by transfer functions based on the impulse responses vol.50, pp.1, 2018, https://doi.org/10.1080/00207721.2018.1544303
  8. Maximum Likelihood-based Multi-innovation Stochastic Gradient Method for Multivariable Systems vol.17, pp.3, 2019, https://doi.org/10.1007/s12555-018-0135-5
  9. Maximum likelihood-based recursive least-squares estimation for multivariable systems using the data filtering technique vol.50, pp.6, 2018, https://doi.org/10.1080/00207721.2019.1590664
  10. Parameter estimation for a special class of nonlinear systems by using the over-parameterisation method and the linear filter vol.50, pp.9, 2019, https://doi.org/10.1080/00207721.2019.1615576
  11. Decomposition-based Gradient Estimation Algorithms for Multivariable Equation-error Systems vol.17, pp.8, 2019, https://doi.org/10.1007/s12555-018-0875-2
  12. Iterative Identification of Discrete-Time Systems With Bilinear Forms in the Presence of Colored Noises Based on the Hierarchical Principle vol.14, pp.9, 2018, https://doi.org/10.1115/1.4044013