Journal of the Korean Society for Aviation and Aeronautics (한국항공운항학회지)

The Korean Society for Aviation and Aeronautics (한국항공운항학회)
권호 표지
DOI QR코드

DOI QR Code

Application of Human Machine Interface and Augmented Reality Technology to Flight Operation

인간-기계 인터페이스 및 증강현실 기술의 항공운항 분야 적용

  • 박형욱 (라이어슨대학교 항공우주공학과) ;
  • 정준 (라이어슨대학교 항공우주공학과) ;
  • 장조원 (한국항공대학교 항공운항학과) ;
  • 주성현 (한국생산기술연구원) ;
  • 황영하 (한국생산기술연구원)
  • Received : 2018.08.17
  • Accepted : 2019.02.11
  • Published : 2019.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

The primary objective of this paper is to introduce the application of Human-Machine Interface (HMI) and Augmented Reality (AR) technologies in flight operations. These include: self-check-in, baggage handling, airport security and surveillance, airport operations monitoring, In-Flight Entertainment and Connectivity (IFEC), cockpit design, and cabin crew support. This paper investigates the application status and development trends of HMI and AR technologies for airports and aircraft. These technologies can provide more efficient in-flight passenger service and experience by using AR devices. This paper also discusses the developments such as; the Integrated Control Application (ICA) for the IFEC interface, AR flight simulation training program using the fixed-based simulator, and the AR aircraft cabin interior concept test program. These applications present how HMI and AR techniques can be utilized in actual flight operations. The developed programs in this paper can be applied to their purpose within aircraft interiors and services to enhance efficiency, comfort, and experience.

Keywords

HHOHB8_2019_v27n2_54_f0001.png 이미지

Fig 1. Smart Bag Drop at Incheon Airport

HHOHB8_2019_v27n2_54_f0002.png 이미지

Fig 2. Airstar of Incheon Airport

HHOHB8_2019_v27n2_54_f0003.png 이미지

Fig 3. Ground Handler with AR Glasses[13]

HHOHB8_2019_v27n2_54_f0004.png 이미지

Fig 4. Non-contact fingerprint recognition[14]

HHOHB8_2019_v27n2_54_f0005.png 이미지

Fig 5. Air Traffic Control with HoloLens[18]

HHOHB8_2019_v27n2_54_f0006.png 이미지

Fig 6. Mock-up of Tempest Fighter, BAE[25]

HHOHB8_2019_v27n2_54_f0007.png 이미지

Fig 7. Strike II Helmet of BAE[26]

HHOHB8_2019_v27n2_54_f0008.png 이미지

Fig 8. AR Navigation (top) and Demonstration Screen (Bottom) of WAYRAY[27]

HHOHB8_2019_v27n2_54_f0009.png 이미지

Fig 9. Wireless and Portable IFE

HHOHB8_2019_v27n2_54_f0010.png 이미지

Fig 10. ActiVision-Smart Interactive Window

HHOHB8_2019_v27n2_54_f0011.png 이미지

Fig 11. Aircraft Cabin Research Facility of National Research Council of Canada [28]

HHOHB8_2019_v27n2_54_f0012.png 이미지

Fig 12. Passenger Information at HoloLens[29]

HHOHB8_2019_v27n2_54_f0013.png 이미지

Fig 13. Galley equipments

HHOHB8_2019_v27n2_54_f0014.png 이미지

Fig 14. Smart Galley of Diehl Aerospace [31]

HHOHB8_2019_v27n2_54_f0015.png 이미지

Fig 15. Menu of ICA

HHOHB8_2019_v27n2_54_f0016.png 이미지

Fig 16. Boing 747-400 Simulator

HHOHB8_2019_v27n2_54_f0017.png 이미지

Fig 17. Optical See-Through Goggle Concept[8]

HHOHB8_2019_v27n2_54_f0018.png 이미지

Fig 18. Airbus A320 Starting Procedure[32]

HHOHB8_2019_v27n2_54_f0019.png 이미지

Fig 19. Concept of HoloLens Simulator Training App

HHOHB8_2019_v27n2_54_f0020.png 이미지

Fig 20. Composition of Sim. Training App

HHOHB8_2019_v27n2_54_f0021.png 이미지

Fig 21. Simulator Training App

HHOHB8_2019_v27n2_54_f0022.png 이미지

Fig 22. Depth Sensor of HoloLens

HHOHB8_2019_v27n2_54_f0023.png 이미지

Fig 23. Spatial Mapping of HoloLens

HHOHB8_2019_v27n2_54_f0024.png 이미지

Fig 24. Aircraft Cabin Mock-up

HHOHB8_2019_v27n2_54_f0025.png 이미지

Fig 25. Inside of Aircraft Cabin Mock-up

HHOHB8_2019_v27n2_54_f0026.png 이미지

Fig 26. OLED Display at Aircraft Cabin Ceiling on AR Environment

HHOHB8_2019_v27n2_54_f0027.png 이미지

Fig 27. Smart Window on AR Environment

HHOHB8_2019_v27n2_54_f0028.png 이미지

Fig 28. Smart Cabin Divider on AR Environment

References

  1. Boeing, "Current Market Outlook: 2018-2037," 2018. (Available at: https://www.boeing.com/resources/boeingdotcom/commercial/market/commercial-market-outlook/assets/downloads/2018-cmo-09-11.pdf)
  2. 장조원, 정준, "AIX(항공기 인테리어 엑스포)를 통해 본 항공기 인테리어 산업(1/2)," 월간항공, 2018년 7월호, pp. 80-87.
  3. 장조원, 정준, "AIX(항공기 인테리어 엑스포)를 통해 본 항공기 인테리어 산업(2/2)," 월간항공, 2018년 8월호, pp. 80-85.
  4. Bjerregaard, L., "Top 10 Ultra-Cool Cabin Interior Products," MRO-Network, 04 Apr. 2018.(https://www.mro-network.com/mrolinks/top-10-ultra-cool-cabin-interior-products)
  5. 서동일, "IEEE P3079 표준화 동향," TTA 저널, Vol. 174, 2017, pp.39-45.
  6. Billinghurst, M., Clark, A. & Lee, G., "A Survey of Augmented Reality," Foundations and Trends(R) in Human-Computer Interaction, Vol. 8, No. 2-3, 2014, pp. 73-272. https://doi.org/10.1561/1100000049
  7. Liu, J., Mei, J., Zhang, X., Lu, X., Huang, J., "Augmented reality-based training system for hand rehabilitation," Multimedia Tools Applications, Vol. 76, Issue 13, 2017, pp.14847-14867. https://doi.org/10.1007/s11042-016-4067-x
  8. Sowmyasree, M., Durga, S., Sindhusha, P., "Augmented Reality," International Journal of Scientific & Engineering Research, Vol. 4, Issue 9, September 2013, pp. 1469-1474.
  9. Nicholl, R., "Airline Head-Up Display Systems: Human Factors Considerations," International Journal of Economics & Management Sciences, Vol. 4, Issue 5, 2015, pp.1-12.
  10. Incheon city, "Mar 18, 2018 at front of self Check-in counter in Incheon airport," Dreanstime, 14 March 2018.(https://www.dreamstime.com/mar-front-self-check-counter-incheon-airport-city-korea-image112557721)
  11. 국민일보, "여권.탑승권 없이 출국심사, AI.IoT 활용... 더 스마트해지는 인천공항," 17 June 2018.(http://m.kmib.co.kr/view.asp? arcid=0923966352)
  12. Korean Air, "Incheon International Airport Gets Smarter with Robotic Help," August 2018. (http://micro.koreanair.com/mainevent/lax/201808/edm/Incheon-Robot.html)
  13. EON Reality, "EON Reality Creates Next Generation Augmented Reality Solution with SATS for Airport Ramp Handling Services,"2017.(https://www.eonreality.com/press-releases/eon-reality-creates-next-generation-augmented-reality-solution-sats-airpor t-ramp-handling-services/)
  14. 인천 국제공항.(https://www.airport.kr/ap/en/index.do)
  15. Peddie, J., "Augmented Reality : Where We Will All Live,"1st Ed., Springer, 2017.
  16. Pinska, E. & Tijus, C., "Augmented Reality Technology for Control Tower Analysis of Applicability based on the Field Study," In Proceeding of 1st CEAS European Air and Space Conference, 2007, pp. 573-580.
  17. Hofmann, T., Konig, C., Bruder, R. & Bergner, J., "How to reduce workload - Augmented reality to ease the work of air traffic controllers," Work, Vol. 41, 2012, pp. 1168-1173. https://doi.org/10.3233/WOR-2012-0299-1168
  18. The Future of Work, "This Is What Happens When HoloLens Meets Air Traffic Control."(https://www.youtube.com/watch?v=uOIDfC_jNGc)
  19. "Reimagine airport control centers with HoloLens technology," SITA, 2016.(https://www.sita.aero/resources/type/videos/reimagine-airport-control-centers-with-hololens-technology)
  20. Mobile & Wearables, "Helsinki Airport exploring operational potential of Microsoft HoloLens," May 2017.(https://www.futuretravelexperience.com/2017/05/sita-lab-and-helsinki-airport-explore-potential-of-microsoft-hololens/)
  21. Foyle, D.C., Andre, A.D., Hooey, B.L., "Situation Awareness in an Augmented Reality Cockpit: Design, Viewpoints and Cognitive Glue," In Proceedings of the 11th International Conference on Human Computer Interaction. Las Vegas, NV., 2005.
  22. Aukstakalnis, S., "Practical Augmented Reality: A guide to the technologies, applications, and human factors for AR and VR. Crafordsville," Pearson Education Inc., 2017.
  23. Nicholl, R., "Airline Head-Up Display Systems: Human Factors Considerations," SSRN Electronic Journal, Vol. 4, Issue 5, 2014. (DOI:10.4172/2162-6359.1000248)
  24. Furht, B., "Handbook of Augmented Reality,"1st Ed., Springer New York 2011.
  25. Ian King, "Tempest jet takes UK engineering by storm," Sky News, 18 July 2018.(http://news.sky.com/story/tempestjet-takes-uk-engineering-by-storm-11440702)
  26. British Telecommunications PLC, "Future plane cockpit's could include virtual dials and readings that appear in front of the pilot's eyes," 14 September 2017.(http://home.bt.com/tech-gadgets/future-tech/bae-systems-mixed-reality-cockpit-aeroplane-11364212192683)
  27. WAYRAY (https://wayray.com/)
  28. National Research Council of Canada, "National Research Council of Canada introduces facility to improve air travellers' experience."(www.nrc-cnrc.gc.ca/eng/stories/2015/new_facility.html)
  29. Robertson, A., "Air New Zealand imagines flight attendants using HoloLens to read your emotions," The Verge, 24 May 2017. (https://www.theverge.com/2017/5/24/15 686740/microsoft-hololens-air-new-zealandaugmented-reality-airplane-emotion-reading)
  30. Bellamy III, W., "9 Companies Using Augmented Reality and Virtual Reality in Aviation," Avionics International, 24 August 2017.(https://www.aviationtoday.com/2017/08/24/9-companies-using-augmented-virtua l-reality-aviation/)
  31. DIEHL Aviation, "Smart Galley."(https://www.diehl.com/aviation/en/innovation/innovative-solutions/smart-galley/)
  32. Airbus, "A318/A319/A320/A321 Flight Crew Training Manual," 08 July 2008.