Optical Music Score Recognition System for Smart Mobile Devices Han, SeJin; Lee, GueeSang;
In this paper, we propose a smart system that can optically recognize a music score within a document and can play the music after recognition. Many historic handwritten documents have now been digitalized. Converting images of a music score within documents into digital files is particularly difficult and requires considerable resources because a music score consists of a 2D structure with both staff lines and symbols. The proposed system takes an input image using a mobile device equipped with a camera module, and the image is optimized via preprocessing. Binarization, music sheet correction, staff line recognition, vertical line detection, note recognition, and symbol recognition processing are then applied, and a music file is generated in an XML format. The Music XML file is recorded as digital information, and based on that file, we can modify the result, logically correct errors, and finally generate a MIDI file. Our system reduces misrecognition, and a wider range of music score can be recognized because we have implemented distortion correction and vertical line detection. We show that the proposed method is practical, and that is has potential for wide application through an experiment with a variety of music scores.
Music Recognition;Music OCR;Optical Music Score Recognition;
J. M. Yoo, N. D. Toan, D. J. Choi, H. R. Park, and G. S. Lee, "Advanced Binarization Method for Music Score Recognition sing Local Thresholds," CIT Workshops, 2008, pp. 417-420.
V. Q. Nhat and G. S. Lee, "Adaptive Line Fitting for Staff Detection in Handwritten Music Score Images," Proc. 8th ICUIMC, article no. 99, 2014.
A. Rebelo, I. Fujinaga, F. Paszkiewicz, A. R. S. Marcal, C. Guedes, and J. S. Cardoso, "Optical music recognition: state-of-the-art and open issues," International Journal of Multimedia Information Retrieval, vol. 1, issue. 3, 2012, pp.173-190.
D. Bainbridge and T Bell, "The Challenge of Optical Music Recognition," Computers and the Humanities, vol. 35, issue. 2, 2001, pp. 95-121.
K. H. Park, S. R. Oh, H. J. Son, J. M. Yoo, S. H. Kim, and G. S. Lee, "Decision-Tree Algorithm for Recognition of Music Score Images Obtained by Mobile Phone Camera," The Journal of the Korea Contents Association, vol. 8, no. 6, 2008, pp. 16-25.
Y. Y. Tang and C. Y. Suen, "Image transformation approach to nonlinear shape restoration," IEEE Trans. Systems. Man and Cybernetics, vol. 23, issue. 1, 1993, pp. 155-172.
T. Beran and T. Macek, "Recognition of Printed Music Score," MLDM'99, LNAI 1715, 1999, pp. 174-179.
A. Capela, J. S. Cardoso, A. Rebelo, and C. Guedes, "Integrated Recognition System for Music Scores," Proc. ICMC'2008, 2008.
A. Robelo, G. Capela, and J. S. Cardoso, "Optical recognition of music symbols: A comparative study," IJDAR, vol. 13, issue. 1, 2010, pp. 19-31.
Q. Q. Arshad, W. Z. Khan, and Z. Ihsan, "Overview of Algorithms and Techniques for Optical Music Recognition," CIIT Workshops on 4th CWRC, 2006.
K. B. Kim, W. J. Lee, and Y. W. Woo, "Automatic Recognition and Performance of Printed Musical Sheets Using Fuzzy ART," The Journal of the Korea Institute of Electronic Communication Sciences, vol. 6, no. 1, 2011, pp. 84-89.
G. S. Choudhury, T. Dilauro, M. Droettboom, I. Fujunaga, and K. Macmillan, "Strike Up the Score," D-Lib Magazing, vol. 7, no. 2. 2001.