Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Detection The Behavior of Smartphone Users using Time-division Feature Fusion Convolutional Neural Network

시분할 특징 융합 합성곱 신경망을 이용한 스마트폰 사용자의 행동 검출

  • Shin, Hyun-Jun (Divsion of Convergence of Computer and Media, Mokwon University) ;
  • Kwak, Nae-Jung (Department of Cyber Security, Baejae University) ;
  • Song, Teuk-Seob (Divsion of Convergence of Computer and Media, Mokwon University)
  • Received : 2020.05.12
  • Accepted : 2020.06.08
  • Published : 2020.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Since the spread of smart phones, interest in wearable devices has increased and diversified, and is closely related to the lives of users, and has been used as a method for providing personalized services. In this paper, we propose a method to detect the user's behavior by applying information from a 3-axis acceleration sensor and a 3-axis gyro sensor embedded in a smartphone to a convolutional neural network. Human behavior differs according to the size and range of motion, starting and ending time, including the duration of the signal data constituting the motion. Therefore, there is a performance problem for accuracy when applied to a convolutional neural network as it is. Therefore, we proposed a Time-Division Feature Fusion Convolutional Neural Network (TDFFCNN) that learns the characteristics of the sensor data segmented over time. The proposed method outperformed other classifiers such as SVM, IBk, convolutional neural network, and long-term memory circulatory neural network.

스마트폰의 보급 이후 웨어러블 디바이스에 대한 관심이 높아지고 다양화되면서 사용자들의 생활에 밀접하게 연관되고 있으며, 개인화된 서비스를 제공하기 위한 방법으로 사용되고 있다. 본 논문에서는 스마트폰에 내장된 3축 가속도 센서와 3축 자이로 센서의 정보를 합성곱 신경망에 적용하여 사용자의 행동을 검출하는 방법을 제안한다. 인간의 행동은 동작의 크기와 범위에 따라서 동작을 구성하는 신호 데이터의 지속시간을 포함한 시작 시점과 끝나는 시점이 다르다. 이로 인해 합성곱 신경망에 그대로 적용하면 행동 인식 정확도에 대한 성능상의 문제가 있다. 따라서 센서 데이터를 시간의 구간에 따라 분할된 특징을 학습하는 시분할 특징 융합 합성곱 신경망(TDFFCNN: Time-Division Feature Fusion Convolutional Neural Network)을 제안하였다.

Keywords

References

  1. D. Roggen, G. Troester, P. Lukowicz, L. Ferscha, J. Millan, and R. Chavarriaga, "Opportunistic human activity and context recognition," IEEE Computer, vol. 46, no. 2, pp. 36-45, Feb. 2013. https://doi.org/10.1109/MC.2012.393
  2. R. A. Voicu, C. Dobre, L. Bajenaru, and R. I. Ciobanu, "Human physical activity recognition using smartphone sensors," Sensors, vol. 19, no. 3, pp. 458, Jan. 2019. https://doi.org/10.3390/s19030458
  3. S. M. Lee, H. Y. Joe, and S. M. Yun, "Machine learning analysis for human behavior recognition based on 3-axis acceleration sensor," Information and communication, vol. 33, no. 11, pp. 65-70, 2016.
  4. D. Anguita, A. Ghio, L. Oneto, X. Parra and J. L. Reyes-Ortiz, "A public domain dataset for human activity recognition using smartphones," in Proceeding of 21th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp. 24-26, Apr. 2013.
  5. M. Shoaib, "Human activity recognition using heterogeneous sensors," in Proceeding of ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 1-4, Sep. 2013.
  6. G. Chetty, M. White and F. Akther, "Smart phone based data mining for human activity recognition," Procedia Computer Science, vol. 46, pp. 1181-1187, 2015. https://doi.org/10.1016/j.procs.2015.01.031
  7. A. Stisen, H. Blunck, S. Bhattacharya, T. S. Prentow, M. B. Kjærgaard, A. Dey, T. Sonne, and M. M. Jensen, "Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recognition," in Proceeding of 13th ACM Conference on Embedded Networked Sensor Systems, pp. 127-140, Nov. 2015.
  8. C. A. Ronao and S. B. Cho, "Human activity recognition with smartphone sensors using deep learning neural networks," Expert Systems with Applications, vol. 59, pp. 235-244, 2016. https://doi.org/10.1016/j.eswa.2016.04.032
  9. W. Jiang, and Z. Yin. "Human activity recognition using wearable sensors by deep convolutional neural networks," in Proceeding of the 23rd ACM international conference on Multimedia, pp. 1307-1310, Oct. 2015.
  10. H. R. Cho, and S. M. Yoon, "Divide and conquer-based 1D CNN human activity recognition using test data sharpening," Sensors, vol. 18, no. 4, pp. 1055, Apr. 2018. https://doi.org/10.3390/s18041055