The 2nd Workshop on Maritime Computer Vision (MaCVi) 2024 addresses maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicles (USV). Three challenges categories are considered: (i) UAV-based Maritime Object Tracking with Re-ideruification, (ii) USV-based Maritime Obstacle Segmentation and Detection, (iii) USV-based Maritime Boat Tracking. The USV-based Maritime Obstacle Segmentation and Detection features three sub-challenges, including a new embedded challenge addressing efficicent inference on real-world embedded devices. This report offers a comprehensive overview of the findings from the challenges. We provide both statistical and qualitative analyses, evaluating trends from over 195 submissions. All datasets, evaluation code, and the leaderboard are available to the public at https://macvi.org/workshop/macvi24.

The process of fish cage inspections, which is a necessary maintenance task at any fish farm, be it small-scale or industrial, is a task that has the potential to be fully automated. Replacing trained divers who perform regular inspections with autonomous marine vehicles would lower the costs of manpower and remove the risks associated with humans performing underwater inspections. Achieving such a level of autonomy implies developing an image processing algorithm that is capable of estimating the state of biofouling buildup. The aim of this work is to propose a complete solution for automating the said inspection process; from developing an autonomous control algorithm for an ROV, to automatically segmenting images of fish cages, and accurately estimating the state of biofouling. The first part is achieved by modifying a commercially available ROV with an acoustic SBL positioning system and developing a closed-loop control system. The second part is realized by implementing a proposed biofouling estimation framework, which relies on AI to perform image segmentation, and by processing images using established computer vision methods to obtain a rough estimate of the distance of the ROV from the fish cage. This also involved developing a labeling tool in order to create a dataset of images for the neural network performing the semantic segmentation to be trained on. The experimental results show the viability of using an ROV fitted with an acoustic transponder for autonomous missions, and demonstrate the biofouling estimation framework’s ability to provide accurate assessments, alongside satisfactory distance estimation capabilities. In conclusion, the achieved biofouling estimation accuracy showcases clear potential for use in the aquaculture industry.

In the interest of both enabling long-term autonomous monitoring of at-risk marine environments and raising awareness and capabilities among citizens, a heterogeneous system of marine robots was developed, integrated, and deployed on a mission in the Adriatic Sea. This paper details a use-case scenario for a team of marine robotic agents for the purpose of cooperative marine litter detection and mapping, while also including interested citizens in the loop and allowing them to serve as operators. Two Autonomous Surface Vehicles (ASVs), a Remotely Operated Vehicle (ROV), and a Smart Buoy were deployed in a real marine environment to demonstrate the cooperative abilities of this system.

This paper presents an overview of advances in estimation of the biofouling state of fish cages as a part of the HEKTOR (Heterogeneous Autonomous Robotic System in Viticulture and Mariculture) project. Firstly, the developed framework for biofouling estimation is shown and explained in brief. A method using k-means clustering for labeling images of fish cages is outlined. It is followed by results of machine learning approaches for automatic inferring of semantic meaning of pixels in an image trained on a recorded dataset using said outlined method. Furthermore, a brief overview and results of contour detection on images classified by trained machine learning models are given. Moreover, a method of feature-based monocular camera distance estimation constrained by assumptions of the viewing angle is presented. All mentioned methods and algorithms fit together in order to produce an estimation of how biofouled the observed net is. The successfulness of the estimation depends on the viewing conditions while filming the cages. In good conditions the results are satisfactory and could be used by industrial fisheries in place of human labour. All the developed algorithms are fast enough so that the entire process from start to finish takes less than 1 second.

Aquaculture net pens inspection and monitoring are important to ensure net stability and fish health in the fish farms. Remotely operated vehicles (ROVs) offer a low-cost and sophisticated solution for the regular inspection of the underwater fish net pens due to their ability of visual sensing and autonomy in a challenging and dynamic aquaculture environment. In this paper, we report the integration of an ROV with a visual servoing scheme for regular inspection and tracking of the net pens. We propose a vision-based positioning scheme that consists of an object detector, a pose generator, and a closed-loop controller. The system employs a modular approach that first utilizes two easily identifiable parallel ropes attached to the net for image processing through traditional computer vision methods. Second, the reference positions of the ROV relative to the net plane are extracted on the basis of a vision triangulation method. Third, a closed-loop control law is employed to instruct the vehicle to traverse from top to bottom along the net plane to inspect its status. The proposed vision-based scheme has been implemented and tested both through simulations and field experiments. The extensive experimental results have allowed the assessment of the performance of the scheme that resulted satisfactorily and can supplement the traditional aquaculture net pens inspection and tracking systems.

There are activities in viticulture and mariculture that require extreme physical endurance from human workers, making them prime candidates for automation and robotization. This paper presents a novel, practical, heterogeneous, autonomous robotic system divided into two main parts, each dealing with respective scenarios in viticulture and mariculture. The robotic components and the subsystems that enable collaboration were developed as part of the ongoing HEKTOR project, and each specific scenario is presented. In viticulture, this includes vineyard surveillance, spraying and suckering with an all-terrain mobile manipulator (ATMM) and a lightweight autonomous aerial robot (LAAR) that can be used in very steep vineyards where other mechanization fails. In mariculture, scenarios include coordinated aerial and subsurface monitoring of fish net pens using the LAAR, an autonomous surface vehicle (ASV), and a remotely operated underwater vehicle (ROV). All robotic components communicate and coordinate their actions through the Robot Operating System (ROS). Field tests demonstrate the great capabilities of the HEKTOR system for the fully autonomous execution of very strenuous and hazardous work in viticulture and mariculture, while meeting the necessary conditions for the required quality and quantity of the work performed.

This paper deals with the development of control software for a remotely operated vehicle (ROV) as part of an automated fish cage inspection system in aquaculture. The ROV navigates autonomously around the fish cages streaming video to the topside computer that runs the algorithms for vertical rope detection. The topside computer sends back the velocity references to the ROV in real-time in order to successfully complete the inspection task. These images are also used to determine the state of net biofouling using a pre-trained convolutional neural network to help determine which nets are in need of cleaning. Robot Operating System (ROS) framework is developed to enable the topside computer to access the video stream from the ROV, process the images, and send back velocity references that would result in the complete inspection of fish cages. The inspection task is planned by following the recognizable rope segments of the outer structure of the fish cage downwards by controlling the vehicle's yaw, heave, and depth.