Department of Computer, Control and Management Engineering

This paper explores the smart exploitation of multi-modal communication capabilities of underwater nodes to enable reliable and swift underwater networking. Following a model-based reinforcement learning approach, we define a framework allowing senders to select the best forwarding relay for its data jointly with the best communication device to reach that relay. The choice is also driven by the quality of the communication to neighboring nodes over time, thus allowing nodes to adapt to the highly adverse and swiftly varying conditions of the underwater channel. The resulting forwarding method allows applications to choose among different classes of soft Quality of Service (QoS), favoring, for instance, reliable routes to the destination, or seeking faster packet delivery. We name our forwarding method MARLIN-Q, for Multi-modAl Reinforcement Learning-based RoutINg with soft QoS. We evaluate the performance of MARLIN-Q in varying networking scenarios where nodes communicate through two acoustic modems with widely different characteristics. MARLIN-Q is compared to state-of-the-art forwarding protocols, including a channel-aware solution, and a machine learning-based solution. Our results show that a smartly learned selection of relay and modem is key to obtain a packet delivery ratio that is twice as much that of other protocols, while maintaining low latency and energy consumption.