Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Recommendation System Using Big Data Processing Technique

빅 데이터 처리 기법을 적용한 추천 시스템에 관한 연구

  • Yun, So-Young (Department of Computer Engineering, Pukyong National University) ;
  • Youn, Sung-Dae (Department of Computer Engineering, Pukyong National University)
  • Received : 2017.01.27
  • Accepted : 2017.02.13
  • Published : 2017.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

With the development of network and IT technology, people are searching and purchasing items they want, not bounded by places. Therefore, there are various studies on how to solve the scalability problem due to the rapidly increasing data in the recommendation system. In this paper, we propose an item-based collaborative filtering method using Tag weight and a recommendation technique using MapReduce method, which is a distributed parallel processing method. In order to improve speed and efficiency, the proposed method classifies items into categories in the preprocessing and groups according to the number of nodes. In each distributed node, data is processed by going through Map-Reduce step 4 times. In order to recommend better items to users, item tag weight is used in the similarity calculation. The experiment result indicated that the proposed method has been more enhanced the appropriacy compared to item-based method, and run efficiently on the large amounts of data.

네트워크와 IT 기술의 발전으로 사용자들은 장소에 구애 받지 않고 어디서든 본인이 원하는 아이템을 검색하고 구매하고 있다. 이에 따라 추천시스템에서 급증하는 데이터로 인한 확장성 문제를 어떻게 해결할 것인가에 대한 연구들이 다양하게 진행되고 있다. 본 논문에서는 Tag 가중치를 적용한 아이템 기반 협업 필터링 기법과 분산 병렬 처리 방식인 MapReduce 방법을 적용한 추천 기법을 제안한다. 제안하는 기법은 속도 향상과 효율성을 위해 전처리 과정에서 아이템을 카테고리별로 분류하고 노드 수에 맞게 그룹지은 후 사용한다. 각 분산 노드에서 4번의 Map-Reduce 단계를 통해 데이터 처리를 진행하는데 사용자에게 더 나은 아이템을 추천하기 위해 유사도 계산에서 아이템 Tag 가중치를 사용한다. 마지막 Reduce 단계를 거쳐 출력된 예측값 중 상위 N개의 아이템을 추천에 사용한다. 실험을 통해 제안 하는 기법이 대량의 데이터를 효율적으로 처리하며 기존의 아이템 기반 기법보다 추천의 적합성도 향상되는 것을 확인하였다.

Keywords

References

  1. P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl, "Grouplens: an open architecture for collaborative filtering of Netnews," Proceedings of the ACM Conference on Computer Supported Cooperative Work, pp. 175-186, New York, NY, USA, Oct. 1994.
  2. G. Linden, B. Smith, and J. York, "Amazon.com recommendations: item-to-item collaborative filtering," IEEE Internet Computing, vol. 7, no. 1, pp. 76-80, January/February 2003. https://doi.org/10.1109/MIC.2003.1167344
  3. M. Papagelisa, D. Plexoosakis, "Qualitative analysis of user-based and item-based prediction algorithms for recommendation agents," ACM Transactions on Information Systems, 22, vol 1. pp.152-166, Sep. 2004.
  4. S. Meng, W. Dou, X. Zhang, and J. Chen, "KASR : A Keyword-Aware Service Recommendation Method on MapReduce for Big Data Applications," IEEE Transactions on Parallel and Distributed Systems, vol. 25, no.12, pp.3221-3231, Dec. 2014. https://doi.org/10.1109/TPDS.2013.2297117
  5. A. Stanescu, S. Nagar, and D. Caragea, "A Hybrid Recommender System : User Profiling from Keywords and Ratings," 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence(WI) and Intelligenr Agent Technologies(IAT), pp. 73-80, Dec.2013.
  6. Z. Zhao, M. Shang, "user-Based Collaborative-Filtering Recommendation Algorithms on Hadoop," 2010 Third International Conference on Knowledge Discovery and Data Mining, pp. 478-481, Jan. 2010.
  7. P. Ghuli, A. Ghosh, and R. Shettar, "A collaborative filtering recommendation engine in a distributed environment," 2014 International Conference on Contemporary Computing and Informatics(IC3I), pp. 568-574, Nov. 2014.
  8. Y. Shang, Z. Li, W. Qu, Y. Xu, Z. Song, and X. Zhou, "Scalable Collaborative Filtering Recommendation Algorithm with MapReduce," 2014 IEEE 12th International Conference on Dependable, Autonomic and Secure Computing, pp.103-108, Aug. 2014.
  9. F. Lu, L. Hong, and L. Changfeng, " The Improvement and implementation of distributed item-based collaborative filtering algorithm on Hadoop," Proceedings of the 34th Chinese Control Conference, pp.9078-9083, July 2015.
  10. H. Liang, Y. Xu, Y. Li, R. Nayak, and G. Shaw, "A Hybrid Recommender Systems based on Weighted Tags," in 10th SIAM International Conference on Data Mining(SDM 2010), Apr. 2011.
  11. S. Yun, S. Youn, "A Study on Recommender Technique Applying User Activity and Time Information," Journal of Korea Institute of Information and Communication Engineering, vol. 19, no. 3, pp. 543-551, Mar. 2015. https://doi.org/10.6109/jkiice.2015.19.3.543
  12. J. A. Konstan, B. N. Miller, D. Maltz, J. L. Herlocker, L. R. Gordon, and J. Riedl, "GroupLens: Applying Collaborative Filtering to Usenet News," Communications of the ACM, vol. 40, no. 3, pp. 77-87, Mar. 1997 https://doi.org/10.1145/245108.245126
  13. B. Sarwar, G. Karypis, J. Konstan, and J. Riedl, "Item-based Collaborative Filtering Recommendation Algorithms," Proceeding of the 10th International World Wide Web Conference, ACM Press, pp. 285-295, 2001.
  14. T. White, Hadoop: The Definitive Guide, Sebastopol. CA: O'REILLY, 2010.
  15. K. Han, Do it: Hadoop with big data, Seoul: Easys Publishing, 2013.
  16. U. Ramachandran, H. Venkateswaran, A. Sivasubramaniam, and A. Singla, "Issues in understanding the scalability of parallel systems," in Proceedings of the First International Workshop on Parallel Processing, pp.399-404. Dec. 1994.
  17. S. Ko, "A Recommender Agent using Association Item Trees," Journal of KIISE:Software and Applications, vol. 36, no. 4, pp. 298-305, Apr. 2009.