文献类型：期刊文献

英文题名：Exploring the performance impact of soft constraint integration on reinforcement learning-based autonomous vessel navigation: Experimental insights

作者：Jiang, Xin[1];Li, Jiawen[1,2,3,4];Huang, Zhenkai[1,3,4];Huang, Ji[1,3,4];Li, Ronghui[1,3,4]

机构：[1]Guangdong Ocean Univ, Naval Architecture & Shipping Coll, Zhanjiang, Peoples R China;[2]Hainan Vocat Univ Sci & Technol, Key Lab Int Shipping Dev & Property Digitizat Hain, Haikou, Peoples R China;[3]Tech Res Ctr Ship Intelligence & Safety Engn Guang, Zhanjiang, Peoples R China;[4]Guangdong Prov Key Lab Intelligent Equipment South, Zhanjiang, Peoples R China

年份：2024

卷号：16

外文期刊名：INTERNATIONAL JOURNAL OF NAVAL ARCHITECTURE AND OCEAN ENGINEERING

收录：SCI-EXPANDED(收录号：WOS:001294298300001)、、EI(收录号：20243116773112)、Scopus(收录号：2-s2.0-85199576624)、WOS

基金：This research was funded by the Young Innovative Talents Grants Program of Guangdong Province (Grant No. 2022KQNCX024) , the Ocean Young Talent Innovation Program of Zhanjiang City (Grant No. 2022E05002) , the National Natural Science Foundation of China (Grant No. 52171346) , the Natural Science Foundation of Guangdong Province, China (Grant No. 2021A1515012618) , the special projects of key fields of Universities in Guangdong Province (Grant No. 2023 ZDZX3003) , the program for scientific research start-up funds of Guangdong Ocean University, and the College Student Innovation Team of Guangdong Ocean University (Grant No. 202410566032) .

语种：英文

外文关键词：Deep reinforcement learning; Soft constraint; Autonomous ship; Reward function; Artificial intelligence

外文摘要：Reinforcement learning has shown promise in enabling autonomous ship navigation, allowing vessels to adapt and make informed decisions in complex marine environments. However, the integration of soft constraints and their impact on performance in RL-based autonomous vessel navigation research remain understudied. This research addresses this gap by investigating the implications of soft constraints in the context of the risk-averse ship navigation problem. Four distinct soft constraint functions are proposed, which are integrated with two widely used RL algorithms, resulting in the creation of eight risk-averse autonomous vessel navigation models. To ensure a comprehensive evaluation of their performance, comparative analyses are conducted across seven virtual digital channel environments. Additionally, a novel metric, known as Large Helm Momentum (LHM), is introduced to quantify the smoothness of autonomous vessel navigation. Through thorough experimentation, key considerations for the design of soft constraint functions in the domain of autonomous ship navigation are identified. A comprehensive understanding of how different soft constraint functions influence autonomous driving behavior has been achieved. Key considerations for designing soft constraint functions in the domain of autonomous ship navigation have also been identified. Five principles, namely the constraint association principle, dominance of hard constraints, reward-balance principle, mapping requirement principle, and iterative improvement principle, are proposed to optimize the design of soft constraint functions for autonomous ship navigation, providing valuable guidance and insights.