This paper addresses the use of deep learning techniques in 3D point cloud labeling of environment representations for the task of a semantic visual localization of mobile robots. In contrast to standard problems resolved with Convolutional Neural Networks (CNNs), the paper deals with applying CNNs to segment point clouds that are, unlike images, unordered and unstructured. The used point clouds contain laser measurements of 3D positions (x,y,z) as well as captured RGB camera images from the scanned scene to colorize the point cloud (RGB values). The main focus of the paper is on implementation and evaluation of a hand-crafted convolution layer and the ConvPoint CNN architecture that introduces continuous convolutions for point cloud processing. The solution was implemented in the Python programming language using the PyTorch deep learning framework.

The scientific discipline of Computer Vision (CV) is a fast developing branch of Machine Learning (ML). It addresses various tasks important for robotics, medicine, autonomous driving, surveillance, security or scene understanding. The development of sensor technologies enabled wide usage of 3D sensors, and therefore, it increased the interest of the CV research community in creating methods for 3D sensor data. This paper outlines seven CV tasks with 3D point cloud data, state-of-the-art techniques, and datasets. Additionally, we identify key challenges.

This paper treats the problem of 3D outdoor environment mapping using images acquired by Unmanned Aerial Vehicle (UAV). The main focus is on the generation of 3D model for large scale environments. In order to perform 3D model reconstruction and mapping from 2D aerial images we employed a Structure from Motion (SfM) based approach. The obtained results using this approach for different scenarios, the rubble field and village, are presented. The generated UAV 3D point cloud data are compared with the ground truth using the least square method, where the ground truth represents a reference model with high accuracy geodetic precision. The comparison of the 3D environment models with the rubble field and village scenarios and the ground truth data is also given.

The paper addresses the problem of detecting pedestrians using three dimensional data acquired by an autonomous mobile robot equipped with an on-board 3D laser scanner. Previous works in this field have dealt with various approaches for combining 2D and 3D range data features for the use in pedestrian classification. In this paper we propose an image processing pipeline for generating a depth image from point clouds data and then localizing object candidates from the depth image. It involves the image segmentation, feature extraction and human classification processes within unstructured dynamic environments. Three different approaches for the detection of pedestrians, vehicles and cyclists using only 3D range data were employed as a part of this system. We train and test the classifiers in an open environment, with presence of multiple pedestrians, cyclists and vehicles, using only point cloud data. The effectiveness and robustness of the proposed system are verified through experiments with real data. This system is also capable to deal with a real-time framerate (10Hz) with high accuracy.

The paper deals with the mathematical modeling and control of an unmanned aerial vehicle (UAV), called octocopter, based on a linear quadratic regulator (LQR). The complex multivariable and nonlinear UAV model is linearized and represented in the state space form. Optimal LQR control system, which is composed of combination the altitude (UAV height - ${z}$ position) and attitude (orientation) controllers, was first designed. Then, this system is extended to provide an additional control of the translation movement in ${x}$ and ${y}$ directions. The proposed LQR control structure is capable of controlling the UAV for all position and orientation coordinates while tracking desired 3D trajectory. Simulation studies are performed on the UAV model where the designed LQR controller has been compared with previously developed PD controller.

The paper focuses on the design and development of a two-wheeled and self-balancing robot as well as its control. The problem is equivalent to the inverted pendulum principle of balancing robots. Dynamic model based PD controller and empirical controller were designed. These controllers are used in closed loop system to provide the robot balance, even when robot is slightly pushed, which normally causes it to fall. The equations of robot motion were derived using Lagrangian and mapped to a transfer function in the complex s-domain. The controller parameters are initially tuned using PID Tuner in Simulink. Then, the zero-order-hold discretization method was applied to implement this control on the Arduino microcontroller. Furthermore, the controller parameters are additionally adjusted through experiments in order to exhibit better control performance. Moreover, the effect of an unexpected disturbance on the robot was taken into consideration. The effectiveness of the designed controllers was verified experimentally.

This paper introduces a novel control approach for Doubly-Fed Induction Generator (DFIG) operating in island mode based on the cascaded control structure with disturbance estimation. The control of the DFIG is a challenging task due to its inherent nonlinearity, fast dynamics, and unpredictable disturbances acting on the system. The proposed control structure involves a nominal controller for plant and disturbance observer (DOB) in each of the inner and outer control loop. The first-order disturbance observers are designed to estimate the time-varying and unknown disturbances. With disturbance estimation, the nominal linear dynamics is obtained in both loops. This enables the same approach for designing controllers for the inner and outer loop which significantly simplifies implementation. The controllers are designed based on the demanded error dynamics and ensure stable operation of the system, while proposed DOBs estimate disturbances including external load. Finally, the effectiveness and quality of the proposed control structure were verified through numerical simulations in terms of external disturbances rejection and closed-loop tracking performance.

With the aim of increasing the efficiency of maintenance and fuel usage in airplanes, structural health monitoring (SHM) of critical composite structures is increasingly expected and required. The optimized usage of this concept is subject of intensive work in the framework of the EU COST Action CA18203 “Optimising Design for Inspection” (ODIN). In this context, a thorough review of a broad range of energy harvesting (EH) technologies to be potentially used as power sources for the acoustic emission and guided wave propagation sensors of the considered SHM systems, as well as for the respective data elaboration and wireless communication modules, is provided in this work. EH devices based on the usage of kinetic energy, thermal gradients, solar radiation, airflow, and other viable energy sources, proposed so far in the literature, are thus described with a critical review of the respective specific power levels, of their potential placement on airplanes, as well as the consequently necessary power management architectures. The guidelines provided for the selection of the most appropriate EH and power management technologies create the preconditions to develop a new class of autonomous sensor nodes for the in-process, non-destructive SHM of airplane components.

This paper focuses on the design of an adaptive controller for read/write (R/W) head servo system in a hard disk drive (HDD). In this control system a reference model-based adaptive loop is added to the inner feedback loop with the PID controller. The parameters of the PID controller are computed using PID tuner within Matlab/Simulink. The normalized MIT rule is then used to tune the controller parameters in order to reduce an error between the reference model and actual system. The main objective is to provide accurate and fast positioning of the R/W head and to ensure a fast response of the arm actuator. The performance of the designed system is evaluated through simulations taking into account an influence of changes in the system parameters as well as the effect of external disturbances acting.

The paper deals with the moving object tracking in dynamic environments, which is one of the most important problems in the field of computer vision. Over the last decade, an intensive work has been extensively done to create smart, autonomous vehicles that provide very precise and fast algorithms for the object detection and tracking. Our paper elaborates and demonstrates how it can be possible to monitor the trajectory of moving objects with high precision using sensor data, where the detection has been previously done. The standard Kalman Filter is described as an introduction to the Extended Kalman Filter (EKF) which was used for the algorithm implementation. Therefore, a problem of choosing model equations is also described, as well as the KITTI dataset used for the object detection. The main contribution of this paper includes an algorithm for the trajectory tracking that is capable to predict the position of moving objects. This algorithm is verified by experiments using realistic dataset.

The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors’ measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds’ alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments.

This paper presents a design of the mobile robot motion framework using slightly modified Vector Field Histogram (VFH) algorithm. The VFH algorithm provides both local motion planning and obstacle avoidance based on on-board sensors measurements. This framework utilizes Robot Operating System (ROS) for the software implementation and simulation purposes. The effectiveness of the proposed framework was verified in both static and dynamic unknown environments. The obtained simulation results indicate a proficiency of the VFH algorithm in navigating the mobile robot from the start to the goal position avoiding collision with obstacles. Trajectories obtained using the designed framework are smooth and without oscillations even in the presence of moving obstacles.

This paper addresses the problem of 3D registration of outdoor environments combining heterogeneous datasets acquired from unmanned aerial (UAV) and ground (UGV) vehicles. In order to solve this problem, we introduced a novel Scale Invariant Registration Method (SIRM) for semi-automated registration of 3D point clouds. The method is capable of coping with an arbitrary scale difference between the point clouds, without any information about their initial position and orientation. Furthermore, the SIRM does not require having a good initial overlap between two heterogeneous datasets. Our method strikes an elegant balance between the existing fully automated 3D registration systems (which often fail in the case of heterogeneous datasets and harsh outdoor environments) and fully manual registration approaches (which are labour-intensive). The experimental validation of the proposed 3D heterogeneous registration system was performed on large-scale datasets representing unstructured and harsh outdoor environments, demonstrating the potential and benefits of the proposed 3D registration system in real-world environments.

This paper treats a path planning problem for the mobile robot with differential constraints using modified RRT (Rapidly exploring random tree) algorithm based on Dubin’s curves. In this paper, the planning problem is considered as a problem of finding a feasible path between the initial and goal point in a static environment with obstacles. Modifications on basic RRT algorithm are necessary due to differential constraints of non-holonomic car-like robot. Dubin’s vehicle is selected as robot model, which represents special case of Simple car that moves only forward with constant velocity and can only make left and right turns. Algorithm is implemented in ROS (Robot Operating System) and tested through realistic simulations in 2D environment consisted of free space and obstacles. The simulation setup was conducted using a 2D Stage simulator in ROS and RViz tool for visualization. Pioneer 3AT robot model was used as Dubin’s car for simulation purposes.