In the complex and variable deep-sea environment,the compensation control of ship motion ensures the safety and efficiency of equipment installation and transportation in offshore wind farms.However,the ship motion po...In the complex and variable deep-sea environment,the compensation control of ship motion ensures the safety and efficiency of equipment installation and transportation in offshore wind farms.However,the ship motion posture compensation control system is severely affected by uncertainties,which significantly impact the accuracy of compensation control.In this paper,we propose a ship three-degree-of-freedom(3-DoF)motion posture stabilization control method based on the DTW-LSTM-MATD3 algorithm.We use the multi-agent twin delayed deep deterministic policy gradient(MATD3)to control a platform with six electric cylinders to achieve stable control.However,owing to random noise affecting the ship’s motion posture,we use a dynamic time warping(DTW)algorithm to distinguish between high-frequency noise and low-frequency tracking signals.Further,we embed a long short-term memory(LSTM)network into the MATD3 network to better align the Critic network’s training with the true Q-value.We use a combined reward function to enhance the agent’s exploration capability in complex dynamic environments.Finally,verification was conducted under sixth-level,abrupt sea conditions with high-frequency noise,as well as under real abrupt sea conditions,and a generalization test was also carried out.Simulation results show that the proposed DTW-LSTM-MATD3 method has great compensation control ability.展开更多
Migratory birds depend on the perception of atmospheric updraft for long-distance flight.To realize more efficient autonomous soaring in an unpowered glider,different strategies for using potential sensorimotor cues t...Migratory birds depend on the perception of atmospheric updraft for long-distance flight.To realize more efficient autonomous soaring in an unpowered glider,different strategies for using potential sensorimotor cues to achieve autonomous soaring efficiency were compared and optimized.A simulation framework of autonomous soaring for an unpowered glider was developed based on a reinforcement learning algorithm.The framework was composed of three models:an updraft environment model,the glider's dynamics and control model,and a reinforcement learning agent,which learns to harvest more energy in flight.Based on the simulation,effects of different combinations of 12 potential sensorimotor cues on soaring efficiency were studied.Firstly,the absence of one particular sensorimotor cue and the use of only a single valid cue in autonomous soaring were analyzed.The results showed that the vertical airflow velocity gradient(aw)and the wing-tip updraft velocity difference(τ)have advantages over the other cues.Secondly,strategies combining aw orτwith other cues were analyzed to achieve more effective autonomous soaring,and seven potentially effective combinations of sensorimotor cues were identified.The final results showed that,among the tested combinations,the combination of vertical airflow velocity(Vw)andτ,enables the most efficient autonomous soaring.This study identified a highly effective sensorimotor cue strategy to guide an intelligent glider to achieve long-distance autonomous soaring flight.展开更多
基金supported by the National Natural Science Foundation of China(No.52105466).
摘要In the complex and variable deep-sea environment,the compensation control of ship motion ensures the safety and efficiency of equipment installation and transportation in offshore wind farms.However,the ship motion posture compensation control system is severely affected by uncertainties,which significantly impact the accuracy of compensation control.In this paper,we propose a ship three-degree-of-freedom(3-DoF)motion posture stabilization control method based on the DTW-LSTM-MATD3 algorithm.We use the multi-agent twin delayed deep deterministic policy gradient(MATD3)to control a platform with six electric cylinders to achieve stable control.However,owing to random noise affecting the ship’s motion posture,we use a dynamic time warping(DTW)algorithm to distinguish between high-frequency noise and low-frequency tracking signals.Further,we embed a long short-term memory(LSTM)network into the MATD3 network to better align the Critic network’s training with the true Q-value.We use a combined reward function to enhance the agent’s exploration capability in complex dynamic environments.Finally,verification was conducted under sixth-level,abrupt sea conditions with high-frequency noise,as well as under real abrupt sea conditions,and a generalization test was also carried out.Simulation results show that the proposed DTW-LSTM-MATD3 method has great compensation control ability.
基金supported by the National Natural Science Foundation of China(Nos.12202384 and U2241274)the Leading Talent Project for Scientific and Technological Innovation in Zhejiang Province(No.2023R5220)the Specialized Research Projects of Huanjiang Laboratory,China。
摘要Migratory birds depend on the perception of atmospheric updraft for long-distance flight.To realize more efficient autonomous soaring in an unpowered glider,different strategies for using potential sensorimotor cues to achieve autonomous soaring efficiency were compared and optimized.A simulation framework of autonomous soaring for an unpowered glider was developed based on a reinforcement learning algorithm.The framework was composed of three models:an updraft environment model,the glider's dynamics and control model,and a reinforcement learning agent,which learns to harvest more energy in flight.Based on the simulation,effects of different combinations of 12 potential sensorimotor cues on soaring efficiency were studied.Firstly,the absence of one particular sensorimotor cue and the use of only a single valid cue in autonomous soaring were analyzed.The results showed that the vertical airflow velocity gradient(aw)and the wing-tip updraft velocity difference(τ)have advantages over the other cues.Secondly,strategies combining aw orτwith other cues were analyzed to achieve more effective autonomous soaring,and seven potentially effective combinations of sensorimotor cues were identified.The final results showed that,among the tested combinations,the combination of vertical airflow velocity(Vw)andτ,enables the most efficient autonomous soaring.This study identified a highly effective sensorimotor cue strategy to guide an intelligent glider to achieve long-distance autonomous soaring flight.