Journal of the Semiconductor & Display Technology (반도체디스플레이기술학회지)

The Korean Society Of Semiconductor & Display Technology (한국반도체디스플레이기술학회)
권호 표지

Photo Retrieval System using Combination of Smart Sensor and Visual Descriptor

스마트 센서와 시각적 기술자를 결합한 사진 검색 시스템

  • Lee, Yong-Hwan (Department of Smart Mobile, Far East University) ;
  • Kim, Heung-Jun (Department of Computer Engineering, Gyeongnam National University of Science and Technology)
  • 이용환 (극동대학교 스마트모바일학과) ;
  • 김흥준 (경남과학기술대학교 컴퓨터공학과)
  • Received : 2014.06.02
  • Accepted : 2014.06.20
  • Published : 2014.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes an efficient photo retrieval system that automatically indexes for searching of relevant images, using a combination of geo-coded information, direction/location of image capture device and content-based visual features. A photo image is labeled with its GPS (Global Positioning System) coordinates and direction of the camera view at the moment of capture, and the label leads to generate a geo-spatial index with three core elements of latitude, longitude and viewing direction. Then, content-based visual features are extracted and combined with the geo-spatial information, for indexing and retrieving the photo images. For user's querying process, the proposed method adopts two steps as a progressive approach, filtering the relevant subset prior to use a content-based ranking function. To evaluate the performance of the proposed scheme, we assess the simulation performance in terms of average precision and F-score, using a natural photo collection. Comparing the proposed approach to retrieve using only visual features, an improvement of 20.8% was observed. The experimental results show that the proposed method exhibited a significant enhancement of around 7.2% in retrieval effectiveness, compared to previous work. These results reveal that a combination of context and content analysis is markedly more efficient and meaningful that using only visual feature for image search.

Keywords

References

  1. M. Li, Z. Chen, H. J. Zhang, "Statistical Correlation Analysis in Image Retrieval", Pattern Recognition, vol.35, issue.12, pp.2687-2693, 2002. https://doi.org/10.1016/S0031-3203(01)00249-7
  2. Website: http://en.wikipedia.org.
  3. N. Singhai, S. K. Shandilya, "A Survey on: Content Based Image Retrieval Systems", International Journal of Computer Applications, vol.4, no.2, pp.22-26, 2010.
  4. A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, R. Jain, "Content-based Image Retrieval at the End of the Early Years", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.22, no.12, pp.1349-1380, 2000. https://doi.org/10.1109/34.895972
  5. B. S. Manjunath, H. R. Ohm, V. V. Vasudevan, A. Yamada. "Color and Texture Descriptors", IEEE Transactions on Circuits and System for Video Technology, vol.11, no.6, pp.703-715, 2001. https://doi.org/10.1109/76.927424
  6. R. S. Torres, A. X. Falcao. "Content-based Image Retrieval: Theory and Applications", Brazilian Symposium on Computer Graphics and Image Processing, vol.13, no.2, pp.165-189, 2006.
  7. International Organization for Standards, "ISO/IEC TR 24800-1:2012 Information Technology -JPSearch - Part 1: System Framework and Components", http:// www.iso.org/iso/home/store/catalogue_ ics/ catalogue_detail_ics.htm?csnumber=60232.
  8. H. H. Wang, D. Mohamad, N.A. Ismail, "Semantic Gap in CBIR: Automatic Objects Spatial Relationships Semantic Extraction and Representation", International Journal of Image Processing, vol.4, issue.3, pp.192-204, 2010.
  9. A. Lakdashti, M. S. Moin, K. Badie, "A Novel Semantic-based Image Retrieval Method", International Conference on Advanced Communication Technology, pp.969-974, 2008.
  10. V. V. Kumar, N. G. Rao, A.L. N. Rao, "RTL: Reduced Texture Spectrum with Lag Value Based Image Retrieval for Medical Images", International Journal of Future Generation Communication and Networking, vol.2, no.4, pp.39-48, 2009.
  11. R. Jain, "Photo Retrieval: Multimedia's Chance to Solve a Real Problem for Real People", IEEE Multimedia, vol.14, no.3, pp.111-112, 2007.
  12. M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, P. Yanker, "Query by Image and Video Content: The QBIC System", IEEE Computer, pp.23-32, 1995.
  13. J. R. Smith, S. Chang, "Visually Searching the Web for Content", IEEE Multimedia, pp.12-20, 1997.
  14. T. Sikora, "The MPEG-7 Visual Standard for Content Description - An Overview", IEEE Transactions on Circuits and Systems for Video Technology, vol.11, no.6, pp.696-702, 2001. https://doi.org/10.1109/76.927422
  15. K. Velmurugan, Lt.Dr.S. Santhosh Baboo, "Content-based Image Retrieval using SURF and Colour Moments", Global Journal of Computer Science and Technology, vol.11, issue.10, 2011.
  16. M. S. Lew, "Next-Generation Web Searches for Visual Content", IEEE Computer, pp.46-53, 2000.
  17. Hae-Kwang Kim, "Image Retrieval Standard MPEG-7 CDVS on Mobile Environment", TTA Journal, vol.144, pp.35-38, 2012.
  18. K. Toyama, R. Logan, A. Roseway, P. Anandan, "Geographic Location Tags on Digital Images", ACM International Conference on Multimedia, pp.156-166, 2003.
  19. Y. Zheng, X. Xie, W. Y. Ma, "GeoLife2.0: A Location-based Social Networking Service", International Conference on Mobile Data Management: Systems, Services and Middleware, pp.357-358, 2009.
  20. Y. F. Chen, G. D. Fabbrizio, D. Gibbon, R. Jana, S. Jora, B. Renger, B. Wei, "GeoTracker: Geospatial and Temporal RSS Navigation", International Conference on World Wide Web, pp.41-50, 2007.
  21. A. Qamra, E. Y. Chang, "Scalable Landmark Recognition using EXTENT", Multimedia Tools and Applications, Vol. 38, Issue. 2, pp.187-208, 2008. https://doi.org/10.1007/s11042-007-0178-8
  22. L. Kennedy, M. Naaman, S. Ahern, R. Nair, T. Rattenbury, "How Flickr Helps us Make Sense of the World: Context and Content in Community-Contributed Media Collections", International Conference on Multimedia, pp.631-640, 2007.
  23. K. Tollmar, T. Yeh, T. Darrell, "IDeixis - Searching the Web with Mobile Images for Location-based Information", Mobile Human-Computer Interaction, LNCS Volume 3160, pp.288-299, 2004.
  24. M. F. Khan, S. Beg, F. Kashif, "Image Transference and Retrieval over SMS", International Journal of Computer Science Issues, Vol.8, Issue.5, No.2, pp.357-361, 2011.
  25. W. Bailer, H. Furntratt, P. Schallauer, G. Thallinger, W. Haas, "A C++ Library for Handling MPEG-7 Descriptions", Proceedings of ACM Multimedia Open Source Software Competition, pp.731-734, 2011.
  26. P. Geiger, R. Pryss, M. Schickler, M. Reichert, "Engineering an Advanced Location-based Augmented Reality Engine for Smart Mobile Devices", Technical Report, University of Ulm, pp.1-25, 2013.
  27. V. Castelli, L.D. Bergman, "Image Databases: Search and Retrieval of Digital Imagery", Wiley Inter-Science, 2002.
  28. Yong-Hwan Lee, Je-Ho Park, Youngseop Kim, "Object Identification and Localization for Image Recognition", Journal of the Semiconductor and Display Technology, vol.11, no.4, pp.49-55, 2012.