Multi-Modal Perception and Fusion for Maritime Autonomy: A Survey

With the rapid progress of deep learning and the increasing availability of maritime sensing and communication technologies, autonomous ships are emerging as a pivotal direction for the future of intelligent marine operations. In autonomous maritime systems, sensors are the foundation of environmental perception, and their cooperative performance directly affects the safety and reliability of navigation. This survey focuses on recent advances in multi-modal perception and fusion strategies for maritime autonomy. The sensing modalities considered include radar, EO/IR cameras, LiDAR (for near-field perception), sonar, INS, AIS, and satellite-based observations. We analyze the strengths and limitations of these sensors and highlight the necessity of multi-modal fusion under complex maritime conditions, such as adverse weather, dynamic sea states, and non-cooperative targets. Based on recent studies, fusion approaches are categorized into early fusion, feature-level fusion and decision-level fusion, with applications in tasks such as vessel detection, obstacle avoidance, and trajectory prediction. Finally, we discuss the current limitations of multimodal fusion in maritime autonomy—such as asynchronous data streams, sensor misalignment, and limited public datasets—and suggest future research directions toward robust, scalable, and real-time fusion frameworks for autonomous ship operations.