PUBLISHED PAPERS
2025 |
PPO-Based Autonomous Swarm Navigation for USVs: A Digital Twin Approach to Defend Brazil’s Maritime Domain Proceedings Article André Siqueira Ruela; Marcelo Silva Souza Resumo | Links | BibTeX | Tags: multi-agent reinforcement learning, proximal policy optimization, unmanned surface vehicles @inproceedings{Ruela2025PpoBased,This paper investigates the scalability of Proximal Policy Optimization (PPO) for coordinating unmanned surface vehicles (USVs) in complex asymmetric warfare scenarios. We extend prior single-agent marine navigation research by establishing a configurable multi-agent reinforcement learning framework, capable of supporting more than 5 USVs, systematically evaluating performance across increasing agent populations (1-5 USVs) and environmental complexity levels (0-20 dynamic obstacles). Our digital twin environment integrates Crest Ocean System and Dynamic Water Physics 2 to simulate realistic maritime conditions. Experimental results reveal nuanced trade-offs: while Curriculum Learning (CL) significantly accelerates initial training, the baseline PPO achieves higher asymptotic success rates during the training phase. However, in final evaluations on complex multi-agent scenarios, PPO combined with CL demonstrates superior generalization and robustness (e.g., 89.66% mission success in 5-agent/20-obstacle configurations). This work provides critical insights into the interplay of training efficiency, asymptotic performance, and generalization in complex multi-agent Deep Reinforcement Learning (DRL) tasks for autonomous swarm systems. |