International Journal of Internet, Broadcasting and Communication

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Comparative analysis of activation functions within reinforcement learning for autonomous vehicles merging onto highways

  • Dongcheul Lee (Department of Multimedia Engineering, Hannam University) ;
  • Janise McNair (Department of Electrical & Computer Engineering, University of Florida)
  • Received : 2023.12.15
  • Accepted : 2024.01.17
  • Published : 2024.02.29
Download PDF
⟨ Previous Next ⟩

Abstract

Deep reinforcement learning (RL) significantly influences autonomous vehicle development by optimizing decision-making and adaptation to complex driving environments through simulation-based training. In deep RL, an activation function is used, and various activation functions have been proposed, but their performance varies greatly depending on the application environment. Therefore, finding the optimal activation function according to the environment is important for effective learning. In this paper, we analyzed nine commonly used activation functions for RL to compare and evaluate which activation function is most effective when using deep RL for autonomous vehicles to learn highway merging. To do this, we built a performance evaluation environment and compared the average reward of each activation function. The results showed that the highest reward was achieved using Mish, and the lowest using SELU. The difference in reward between the two activation functions was 10.3%.

Keywords

References

  1. D. Lee, "Comparison of Reinforcement Learning Activation Functions to Maximize Rewards in Autonomous Highway Driving," The Journal of the Institute of Internet, Broadcasting and Communication (JIIBC), Vol. 22, No. 5, pp. 63-68, 2022. DOI: https://doi.org/10.7236/JIIBC.2022.22.5.63
  2. A. Alkhouly, A. Mohammed, and H. Hefny, "Improving the Performance of Deep Neural Networks Using Two Proposed Activation Functions," IEEE Access, Vol. 9, 2021. DOI: https://doi.org/10.1109/ACCESS.2021.3085855
  3. Z. Hu, J. Zhang, and Y. Ge, "Handling Vanishing Gradient Problem Using Artificial Derivative," IEEE Access, Vol. 9, 2021. DOI: https://doi.org/10.1109/ACCESS.2021.3054915
  4. S. Douglas and J. Yu, "Why RELU Units Sometimes Die: Analysis of Single-Unit Error Backpropagation in Neural Networks," 2018 52nd Asilomar Conference on Signals, Systems, and Computers, 2018. DOI: https://doi.org/10.1109/ACSSC.2018.8645556
  5. H. Zheng, Z. Yang, W. Liu, J. Liang and Y. Li, "Improving deep neural networks using softplus units," 2015 International Joint Conference on Neural Networks (IJCNN), 2015. DOI: https://doi.org/10.1109/IJCNN.2015.7280459
  6. J. Xu, Z. Li, B. Du, M. Zhang and J. Liu, "Reluplex made more practical: Leaky ReLU," 2020 IEEE Symposium on Computers and Communications (ISCC), 2020. DOI: https://doi.org/10.1109/ISCC50000.2020.9219587
  7. Y. Tsai, Y. Jheng and R. Tsaih, "The Cramming, Softening and Integrating Learning Algorithm with Parametric ReLU Activation Function for Binary Input/Output Problems," 2019 International Joint Conference on Neural Networks (IJCNN), 2019. DOI: https://doi.org/10.1109/IJCNN.2019.8852023
  8. J. Kang et al., "An Improved 3D Human Pose Estimation Model Based on Temporal Convolution with Gaussian Error Linear Units," 2022 8th International Conference on Virtual Reality (ICVR), 2022. DOI: https://doi.org/10.1109/ICVR55215.2022.9848068
  9. Z. Huang, T. Ng, L. Liu, H. Mason, X. Zhuang and D. Liu, "SNDCNN: Self-Normalizing Deep CNNs with Scaled Exponential Linear Units for Speech Recognition," ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020. DOI: https://doi.org/10.1109/ICASSP40776.2020.9053973
  10. J. Barron, "Continuously Differentiable Exponential Linear Units," arXiv preprint arXiv:1704.07483, 2017. DOI: https://doi.org/10.48550/arXiv.1704.07483
  11. S. Elfwing, E. Uchibe, and K. Doya, "Sigmoid-weighted linear units for neural network function approximation in reinforcement learning," Neural Networks, Vol. 107, 2018. DOI: https://doi.org/10.1016/j.neunet.2017.12.012
  12. D. Misra, "Mish: A self regularized non-monotonic activation function," arXiv preprint arXiv:1908.08681, 2019. DOI: https://doi.org/10.48550/arXiv.1908.08681
  13. E. Leurent, "An Environment for Autonomous Driving Decision-Making," GitHub repository: https://github.com/eleurent/highway-env, 2018.
  14. J. Schulman, et al., "Proximal policy optimization algorithms," arXiv preprint arXiv:1707.06347, 2017. DOI: https://doi.org/10.48550/arXiv.1707.06347
  15. C. Garbin, Z. Xingquan, and O. Marques, "Dropout vs. batch normalization: an empirical study of their impact to deep learning," Multimedia Tools and Applications, Vol. 79, 2020. DOI: https://doi.org/10.1007/s11042-019-08453-9