This article is dedicated to Eduardo D. Sontag on the occasion of his 70th birthday. We build upon fundamental stability concepts developed by Sontag, such as input-to-state stability and its related properties, to study a relevant application in industrial internet of things, namely estimation for wireless networked control systems. Particularly, we study emulation-based state estimation for nonlinear plants that communicate with a remote observer over a shared wireless network subject to packet losses. To reduce bandwidth usage, a stochastic communication protocol is employed to determine which node should be given access to the network. Each node has a different successful transmission probability. We describe the overall closed-loop system as a stochastic hybrid model, which allows us to capture the behaviour both between and at transmission instants, whilst covering network features such as random transmission instants, packet losses and stochastic scheduling. We then provide sufficient conditions on the transmission rate that guarantee an input-to-state stability property (in expectation) for the corresponding estimation error system. We illustrate our results in the design of circle criterion observers.

Transmit power control is one of the most important issues in wireless networks, where nodes typically operate on limited battery power. Reducing communicating power consumption is essential for both economic and ecologic reasons. In fact, transmitting at unnecessarily high power not only reduces node lifetime, but also introduces excessive interference and electromagnetic pollution. Existing work in the wireless community mostly focus on designing transmit power policies by taking into account communication aspects like quality of service or network capacity. Wireless networked control systems (WNCSs), on the other hand, have different and specific needs such as stability, which require transmit power policies adapted to the control context. Transmit power design in the control community has recently attracted much attention, and available works mostly consider linear systems or specific classes of non-linear systems with a single-link view of the system. In this paper, we propose a framework for the design of stabilising transmit power levels that applies to much larger classes of non-linear plants, controllers, and multi-link setting. By exploiting the fact that channel success probabilities are related to transmit power in a non-linear fashion, we first derive closed-loop stability conditions that relate channel probabilities with transmission rate. Next, we combine these results together with well-known and realistic interference models to provide a design methodology for stabilising transmit power in non-linear and multi-link WNCSs.

We present a new algorithm called policy iteration plus (PI+) for the optimal control of nonlinear deterministic discrete-time plants with general cost functions. PI+ builds upon classical policy iteration and has the distinctive feature to enforce recursive feasibility under mild conditions, in the sense that the minimization problems solved at each iteration are guaranteed to admit a solution. While recursive feasibility is a desired property, it appears that existing results on the policy iteration algorithm fail to ensure it in general, contrary to PI+. We also establish the recursive stability of PI+: the policies generated at each iteration ensure a stability property for the closed-loop system. We prove our results under more general conditions than those currently available for policy iteration, by notably covering set stability. Finally, we present characterizations of near-optimality bounds for PI+ and prove the uniform convergence of the value functions generated by PI+ to the optimal value function. We believe that these results would benefit the burgeoning literature on approximate dynamic programming and reinforcement learning, where recursive feasibility is typically assumed without a clear method for verifying it and where recursive stability is essential for safe operation of the system.

This paper investigates recursive feasibility, recursive robust stability and near-optimality properties of policy iteration (PI). For this purpose, we consider deterministic nonlinear discrete-time systems whose inputs are generated by PI for undiscounted cost functions. We first assume that PI is recursively feasible, in the sense that the optimization problems solved at each iteration admit a solution. In this case, we provide novel conditions to establish recursive robust stability properties for a general attractor, meaning that the policies generated at each iteration ensure a robust \KL-stability property with respect to a general state measure. We then derive novel explicit bounds on the mismatch between the (suboptimal) value function returned by PI at each iteration and the optimal one. Afterwards, motivated by a counter-example that shows that PI may fail to be recursively feasible, we modify PI so that recursive feasibility is guaranteed a priori under mild conditions. This modified algorithm, called PI+, is shown to preserve the recursive robust stability when the attractor is compact. Additionally, PI+ enjoys the same near-optimality properties as its PI counterpart under the same assumptions. Therefore, PI+ is an attractive tool for generating near-optimal stabilizing control of deterministic discrete-time nonlinear systems.

We present stability conditions for deterministic time-varying nonlinear discrete-time systems whose inputs aim to minimize an infinite-horizon time-dependent cost. Global asymptotic and exponential stability properties for general attractors are established. This work covers and generalizes the related results on discounted optimal control problems to more general systems and cost functions.

We investigate the scenario where a perturbed nonlinear system transmits its output measurements to a remote observer via a packet-based communication network. The sensors are grouped into N nodes and each of these nodes decides when its measured data is transmitted over the network independently. The objective is to design both the observer and the local transmission policies in order to obtain accurate state estimates, while only sporadically using the communication network. In particular, given a general nonlinear observer designed in continuous-time satisfying an input-to-state stability property, we explain how to systematically design a dynamic event-triggering rule for each sensor node that avoids the use of a copy of the observer, thereby keeping local calculation simple. We prove the practical convergence property of the estimation error to the origin and we show that there exists a uniform strictly positive minimum inter-event time for each local triggering rule under mild conditions on the plant. The efficiency of the proposed techniques is illustrated on a numerical case study of a flexible robotic arm.

Various methods are nowadays available to design observers for broad classes of systems. Nevertheless, the question of the tuning of the observer to achieve satisfactory estimation performance remains largely open. This paper presents a general supervisory design framework for online tuning of the observer gains with the aim of achieving various trade-offs between robustness and speed of convergence. We assume that a robust nominal observer has been designed for a general nonlinear system and the goal is to improve its performance. We present for this purpose a novel hybrid multi-observer, which consists of the nominal one and a bank of additional observer-like systems, that are collectively referred to as modes and that differ from the nominal observer only in their output injection gains. We then evaluate on-line the estimation cost of each mode of the multi-observer and, based on these costs, we select one of them at each time instant. Two different strategies are proposed. In the first one, initial conditions of the modes are reset each time the algorithm switches between different modes. In the second one, the initial conditions are not reset. We prove a convergence property for the hybrid estimation scheme and we illustrate the efficiency of the approach in improving the performance of a given nominal high-gain observer on a numerical example.

We propose a certainty-equivalence scheme for adaptive control of scalar linear systems subject to additive, i.i.d. Gaussian disturbances and bounded control input constraints, without requiring prior knowledge of the bounds of the system parameters, nor the control direction. Assuming that the system is at-worst marginally stable, mean square boundedness of the closed-loop system states is proven. Lastly, numerical examples are presented to illustrate our results.

– This paper considers controlled scalar systems relying on a lossy wireless feedback channel. In contrast with the existing literature, the focus is not on the system controller but on the wireless transmit power controller that is implemented at the system side for reporting the state to the controller. Such a problem may be of interest, e.g. , for the remote control of drones, where communication costs may have to be considered. Determining the power control policy that minimizes the combination of the dynamical system cost and the wireless transmission energy is shown to be a non-trivial optimization problem. It turns out that the recursive structure of the problem can be exploited to determine the optimal power control policy. As illustrated in the numerical performance analysis, in the scenario of a dynamics without perturbations, the optimal power control policy consists in decreasing the transmit power at the right pace. This allows a significant performance gain compared to conventional policies such as the full transmit power policy or the open-loop policy.

This paper considers the problem of online optimization where the objective function is time-varying. In particular, we extend coordinate descent type algorithms to the online case, where the objective function varies after a finite number of iterations of the algorithm. Instead of solving the problem exactly at each time step, we only apply a finite number of iterations at each time step. Commonly used notions of regret are used to measure the performance of the online algorithm. Moreover, coordinate descent algorithms with different updating rules are considered, including both deterministic and stochastic rules that are developed in the literature of classical offline optimization. A thorough regret analysis is given for each case. Finally, numerical simulations are provided to illustrate the theoretical results.

We study emulation-based state estimation for non-linear plants that communicate with a remote observer over a shared wireless network subject to packet losses. To reduce bandwidth usage, a stochastic communication protocol is employed to determine which node should be given access to the network. We describe the overall wireless system as a hybrid model, which allows us to capture the behaviour both between and at transmission instants, whilst covering network features such as random transmission instants, packet losses, and stochastic scheduling. Under this setting, we provide sufficient conditions on the transmission rate that guarantee an input-to-state stability property for the corresponding estimation error system. We illustrate our results with an example of Lipschitz non-linear plants.

This paper presents two schemes to jointly estimate parameters and states of discrete-time nonlinear systems in the presence of bounded disturbances and noise. The parameters are assumed to belong to a known compact set. Both schemes are based on sampling the parameter space and designing a state observer for each sample. A supervisor selects one of these observers at each time instant to produce the parameter and state estimates. In the first scheme, the parameter and state estimates are guaranteed to converge within a certain margin of their true values in finite time, assuming that a sufficiently large number of observers is used and a persistence of excitation condition is satisfied in addition to other observer design conditions. This convergence margin is constituted by a part that can be chosen arbitrarily small by the user and a part that is determined by the noise levels. The second scheme exploits the convergence properties of the parameter estimate to perform subsequent zoom-ins on the parameter subspace to achieve stricter margins for a given number of observers. The strengths of both schemes are demonstrated using a numerical example.

To investigate solutions of (near-)optimal control problems, we extend and exploit a notion of homogeneity recently proposed in the literature for discrete-time systems. Assuming the plant dynamics is homogeneous, we first derive a scaling property of its solutions along rays provided the sequence of inputs is suitably modified. We then consider homogeneous cost functions and reveal how the optimal value function scales along rays. This result can be used to construct (near-)optimal inputs on the whole state space by only solving the original problem on a given compact manifold of a smaller dimension. Compared to the related works of the literature, we impose no conditions on the homogeneity degrees. We demonstrate the strength of this new result by presenting a new approximate scheme for value iteration, which is one of the pillars of dynamic programming. The new algorithm provides guaranteed lower and upper estimates of the true value function at any iteration and has several appealing features in terms of reduced computation. A numerical case study is provided to illustrate the proposed algorithm.