Sound event classification using deep neural network based transfer learning

깊은 신경망 기반의 전이학습을 이용한 사운드 이벤트 분류

Lim, Hyungjun;Kim, Myung Jong;Kim, Hoirin

  • Received : 2015.11.10
  • Accepted : 2015.12.22
  • Published : 2016.03.31


Deep neural network that effectively capture the characteristics of data has been widely used in various applications. However, the amount of sound database is often insufficient for learning the deep neural network properly, so resulting in overfitting problems. In this paper, we propose a transfer learning framework that can effectively train the deep neural network even with insufficient sound event data by employing rich speech or music data. A series of experimental results verify that proposed method performs significantly better than the baseline deep neural network that was trained only with small sound event data.


Sound event classification;Transfer learning;Deep neural network


  1. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition," IEEE Signal Process. Mag. 29, 82-97 (2012).
  2. G. E. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pre-trained deep neural networks for large vocabulary speech recognition," IEEE Trans. Audio, Speech, and Lang. Process. 20, 33-42 (2012).
  3. C. Weng, D. Yu, S. Watanabe, and B. H. F. Juang, "Recurrent deep neural networks for robust speech recognition," in Proc. IEEE ICASSP, 5532-5536 (2014).
  4. Y. Lei, N. Scheffer, L. Ferrer, and M. McLaren, "A novel scheme for speaker recognition using a phonetically-aware deep neural network," in Proc. IEEE ICASSP, 1695-1699 (2014).
  5. D. G. Romero and A. McCree, "Insight into deep neural networks for speaker recognition," in Proc. Interspeech, 1141-1145 (2015).
  6. S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Trans. Knowl. Data Eng. 22, 1345-1359 (2010).
  7. L. Deng and X. Li, "Machine learning paradigms for speech recognition: An overview," IEEE Trans. Audio, Speech, Lang. Process. 21, 1060-1089 (2013).
  8. A. Das and M. Hasegawa-Johnson, "Cross-lingual transfer learning during supervised training in low resource scenarios," in Proc. Interspeech, 3531-3535 (2015).
  9. J. T. Huang, J. Li, D. Yu, L. Deng, and Y. Gong, "Crosslanguage knowledge transfer using multilingual deep neural network with shared hidden layers," in Proc. IEEE ICASSP, 7304-7308 (2013).
  10. O. Gencoglu, T. Virtanen, and H. Huttunen, "Recognition of acoustic events using deep neural networks," in Proc. IEEE European Signal Process. Conf, 506-510 (2014).
  11. M. Espi, M. Fujimoto, K. Kinoshita, and T. Nakatani, "Feature extraction strategies in deep learning based acoustic event detection," in Proc. Interspeech, 2922-2926 (2015).
  12. S. Nakamura, K. Hiyane, F. Asano, T. Yamada, and T. Endo, "Data collection in real acoustical environments for sound scene understanding and hands-free speech recognition," in Proc. Eurospeech, 2255-2258 (1999).
  13. P. Price, W. M. Fisher, J. Bernstein, and D. S. Pallett, "The DARPA 1000-word resource management database for continuous speech recognition," in Proc. IEEE ICASSP, 651-654 (1988).
  14. G. Tzanetakis and P. Cook, "Musical genre classification of audio signals," IEEE Trans. Audio, Speech and Lang. Process. 10, 293-302 (2002).
  15. Y. Miao, "Kaldi+PDNN: building DNN-based ASR systems with Kaldi and PDNN," arXiv:1401.6984, (2014).
  16. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, "How transferable are features in deep neural networks?" in Proc. Neural Inform. Process. Syst., 3320-3328 (2014).