International Journal of Naval Architecture and Ocean Engineering

The Society of Naval Architects of Korea (대한조선학회)
권호 표지
DOI QR코드

DOI QR Code

Application of reinforcement learning to fire suppression system of an autonomous ship in irregular waves

  • Lee, Eun-Joo (Graduate School, Dept. of Naval Architecture & Ocean Engineering, Chungnam National University) ;
  • Ruy, Won-Sun (Department of Naval Architecture & Ocean Engineering, Chungnam National University) ;
  • Seo, Jeonghwa (Department of Naval Architecture & Ocean Engineering, Chungnam National University)
  • Received : 2020.06.07
  • Accepted : 2020.11.01
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In fire suppression, continuous delivery of water or foam to the fire source is essential. The present study concerns fire suppression in a ship under sea condition, by introducing reinforcement learning technique to aiming of fire extinguishing nozzle, which works in a ship compartment with six degrees of freedom movement by irregular waves. The physical modeling of the water jet and compartment motion was provided using Unity 3D engine. In the reinforcement learning, the change of the nozzle angle during the scenario was set as the action, while the reward is proportional to the ratio of the water particle delivered to the fire source area. The optimal control of nozzle aiming for continuous delivery of water jet could be derived. Various algorithms of reinforcement learning were tested to select the optimal one, the proximal policy optimization.

Keywords

Acknowledgement

This research was financially supported by the Institute of Civil Military Technology Cooperation funded by the Defense Acquisition Program Administration and Ministry of Trade, Industry and Energy of Korean government under grant No. UM19304RD3.

References

  1. Beck, J.E., Woolf, B.P., 1998. Using a learning agent with a student model. In: International Conference on Intelligent Tutoring Systems. Springer, pp. 6-15.
  2. Brent, R.P., 1973. Chapter 4: an Algorithm with Guaranteed Convergence for Finding a Zero of a Function. Algorithms for Minimization without Derivatives. PrenticeHall, Englewood Cliffs, NJ, ISBN 0-13-022335-2.
  3. de Vos, J., Hekkenberg, R., 2019. Assessment of the required subdivision index for autonomous ships based on equivalent safety. Assessment 17-18.
  4. DeFilippo, M., Robinette, P., Sacarny, M., Benjamin, M.R., 2019. The remote explorer IV: an autonomous vessel for oceanographic research. In: OCEANS 2019-Marseille, pp. 1-8.
  5. Department of Defense, 1986. Interface Standard for Shipboard Systems: Ship Motion and Attitude (Metric), DOD-STD-1399(NAVY) Section 301A. Naval Sea Systems Command, Washington, DC.
  6. Elkhart brass X-Stream, 2020. http://www.elkhartbrass.com/products/masterstream/x-stream/multimedia. (Accessed 13 March 2019).
  7. Fraize, J., Ekman, F., Fürth, M., Hoffenson, S., Chell, B., 2019. Multi-objective reliability-based design optimization of an autonomous sailing vessel. In: 29th International Ocean and Polar Engineering Conference. International Society of Offshore and Polar Engineers.
  8. IMO, 2018. Maritime safety committee (MSC) 100th session.
  9. Juliani, A., Berges, V.P., Vckay, E., Gao, Y., Henry, H., Mattar, M., Lange, D., 2018. Unity: A General Platform for Intelligent Agents arXiv preprint arXiv 1809.02627.
  10. Kim, J.B., Hwang, K.Y., An, C.H., Han, Y.H., 2019. Research trends on PID control and reinforcement learning control an drone. J. Adv. Technol. Res. 4 (1), 6-13. ISSN: 0974-6269.
  11. Ko, E.S., Lee, M.H., 2019. A study on the legal status of unmanned maritime systems. Korean J. Mil. Art Sci. 75, 89-118. https://doi.org/10.31066/KJMAS.2019.75.3.004
  12. Lee, G.I., 2019. Standardization trend of IEC for self-operating ships. In: Proceedings of the Korean Institute of Navigation and Port Research Conference. Korean Institute of Navigation and Port Research, pp. 222-224.
  13. Macklin, M., Müller, M., 2013. Position based fluids. ACM Trans. Graph. 32 (4), 1-12.
  14. McNeil, J.G., Lattimer, B.Y., 2017. Robotic fire suppression through autonomous feedback control. Fire Technol. 53, 1171-1199. https://doi.org/10.1007/s10694-016-0623-1
  15. Miyashita, T., Sugawa, O., Imamura, T., Kamiya, K., Kawaguchi, Y., 2014. Modeling and analysis of water discharge trajectory with large capacity monitor. Fire Saf. J. 63, 1-8. https://doi.org/10.1016/j.firesaf.2013.09.028
  16. ML-Agents, 2020. https://github.com/Unity-Technologies/ml-agents/. (Accessed 12 May 2020).
  17. Mnih, V., Puigdomenech, A., Badia, Mirza, M., Graves, A., Lillicrap, T.P., Harley, T., Silver, D., Kavukcuoglu, K., 2016. Asynchronous Methods for Deep Reinforcement Learning. ArXiv preprint arXiv:1602.01783.
  18. Obi user manual, 2020. URL. http://obi.virtualmethodstudio.com/tutorials/.
  19. Perez, T., 2006. Ship Motion Control: Course Keeping and Roll Stabilisation Using Rudder and Fins. Springer Science & Business Media.
  20. Sin, I.S., 2018. Special Report-Trends in standardization of autonomous transport ships. TTA J 36-42.
  21. SM-1250 SERIES REACH, 2020. URL. https://www.elkhartbrass.com/files/aa/downloads/performance/SM-1250%20SERIES%20REACH@HEIGHT.pdf.
  22. Tang, H., Yin, Y., Shen, H., 2019. A model for vessel trajectory prediction based on long short-term memory neural network. J. Mar. Eng. Technol. 1-10.
  23. Unity. https://docs.unity3d.com/Manual/index.html. (Accessed 13 January 2020).
  24. Yoon, M.O., 2019. Expansion of the application of autonomous firefighting facilities and fire suppression strategy improvement plan for the reduction of fire damage. J. Kor. Inst. Fire Sci. Eng.