In this article, we analyze the stability properties of stochastic linear systems in closed loop with an optimal policy that minimizes a discounted quadratic cost in expectation. In particular, the linear system is perturbed by both additive and multiplicative stochastic disturbances. We provide conditions under which mean-square boundedness, mean-square stability, and recurrence properties hold for the closed-loop system. We distinguish two cases, when these properties are verified for any value of the discount factor sufficiently close to 1, or when they hold for a fixed value of the discount factor in which case tighter conditions are derived, as illustrated in an example. The analysis exploits properties of the optimal value function, as well as a detectability property of the system with respect to the stage cost, to construct a Lyapunov function for the stochastic linear quadratic regulator problem.

This paper studies the emulation-based stabilization of nonlinear networked control systems with two time scales. We address the challenge of using a single communication channel for transmitting both fast and slow variables between the plant and the controller. A novel dual clock mechanism is proposed to schedule transmissions for this purpose. The system is modeled as a hybrid singularly perturbed dynamical system, and singular perturbation analysis is employed to determine individual maximum allowable transmission intervals for both fast and slow variables, ensuring semi-global practical asymptotic stability. Enhanced stability guarantees are also provided under stronger assumptions. The efficacy of the proposed method is illustrated through a numerical example.

We investigate the problem of least squares parameter estimation from single-trajectory data for discrete-time, unstable, closed-loop nonlinear stochastic systems. Specifically, we consider nonlinear systems with linearly parametrised uncertainty and additive i.i.d. process noise, in feedback with a control policy that is intentionally perturbed by an exploratory input. Assuming the open-loop dynamics satisfy a particular sub-exponential input-to-state growth property, and a region of the state space produces informative data, we establish non-asymptotic guarantees on the estimation error at times when the state trajectory evolves in this region. If the whole state space is informative, high-probability guarantees on the error hold for all times. Examples are provided where our results are useful for analysis beyond existing works.

We consider the problem of least squares parameter estimation from single-trajectory data for discrete-time, unstable, closed-loop nonlinear stochastic systems, with linearly parameterised uncertainty. Assuming a region of the state space produces informative data, and the system is sub-exponentially unstable, we establish non-asymptotic guarantees on the estimation error at times where the state trajectory evolves in this region. If the whole state space is informative, high probability guarantees on the error hold for all times. Examples are provided where our results are useful for analysis, but existing results are not.

We consider deterministic nonlinear discrete-time systems whose inputs are generated by policy iteration (PI) for undiscounted cost functions. We first assume that PI is recursively feasible, in the sense that the optimization problems solved at each iteration admit a solution. In this case, we provide novel conditions to establish recursive robust stability properties for a general attractor, meaning that the policies generated at each iteration ensure a robust <inline-formula><tex-math notation="LaTeX">$\mathcal {KL}$</tex-math></inline-formula>-stability property with respect to a general state measure. We then derive novel explicit bounds on the mismatch between the (suboptimal) value function returned by PI at each iteration and the optimal one. However, we show by a counterexample that PI may fail to be recursively feasible, disallowing the mentioned stability and near-optimality guarantees. We therefore also present a modification of PI so that recursive feasibility is guaranteed a priori under mild conditions. This modified algorithm, called PI<inline-formula><tex-math notation="LaTeX">$^{+}$</tex-math></inline-formula>, is shown to preserve the recursive robust stability when the attractor is compact. In addition, PI<inline-formula><tex-math notation="LaTeX">$^{+}$</tex-math></inline-formula> enjoys the same near-optimality properties as its PI counterpart under the same assumptions.

This paper presents a novel event-triggered control (ETC) design framework based on measured $\mathcal{L}_{p}$ norms. We consider a class of systems with finite $\mathcal{L}_{p}$ gain from the network-induced error to a chosen output. The $\mathcal{L}_{p}$ norms of the network-induced error and the chosen output since the last sampling time are used to formulate a class of triggering rules. Based on a small-gain condition, we derive an explicit expression for the $\mathcal{L}_{p}$ gain of the resulting closed-loop systems and present a time-regularization, which can be used to guarantee a lower bound on the inter-sampling times. The proposed framework is based on a different stability- and triggering concept compared to ETC approaches from the literature, and thus may yield new types of dynamical properties for the closed-loop system. However, for specific output choices it can lead to similar triggering rules as “standard” static and dynamic ETC approaches based on input-to-state stability and yields therefore a novel interpretation for some of the existing triggering rules. We illustrate the proposed framework with a numerical example from the literature.

Given a discounted cost, we study deterministic discrete-time systems whose inputs are generated by policy iteration (PI). We provide novel near-optimality and stability properties, while allowing for non-stabilizing initial policies. That is, we first give novel bounds on the mismatch between the value function generated by PI and the optimal value function, which are less conservative in general than those encountered in the dynamic programming literature for the considered class of systems. Then, we show that the systems in closed-loop with policies generated by PI are stabilizing under mild conditions, after a finite (and known) number of iterations.

The state of charge (SOC) of lithium-ion batteries needs to be accurately estimated for safety and reliability purposes. For battery packs made of a large number of cells, it is not always feasible to design one SOC estimator per cell due to limited computational resources. Instead, only the minimum and the maximum SOC need to be estimated. The challenge is that the cells having the minimum and maximum SOC typically change over time. In this context, we present a low-dimensional hybrid estimator of the minimum (maximum) SOC, whose convergence is analytically guaranteed. We consider for this purpose a battery consisting of cells interconnected in series, which we model by electrical equivalent circuit models. We then present the hybrid estimator, which runs an observer designed for a single cell at any time instant, selected by a switching-like logic mechanism. We establish a practical exponential stability property for the estimation error on the minimum (maximum) SOC thereby guaranteeing the ability of the hybrid scheme to generate accurate estimates of the minimum (maximum) SOC. The analysis relies on non-smooth hybrid Lyapunov techniques. A numerical illustration is provided to showcase the relevance of the proposed approach.

This paper studies the stabilization problem of networked control systems (NCSs) with random packet dropouts caused by stochastic channels. To describe the effects of stochastic channels on the information transmission, the transmission times are assumed to be deterministic, whereas the packet transmission is assumed to be random. We first propose a stochastic scheduling protocol to model random packet dropouts, and address the properties of the proposed stochastic scheduling protocol. The proposed scheduling protocol provides a unified modelling framework for a general class of random packet dropouts due to different stochastic channels. Next, the proposed scheduling protocol is embedded into the closed-loop system, which leads to a stochastic hybrid model for NCSs with random packet dropouts. Based on this stochastic hybrid model, we follow the emulation approach to establish sufficient conditions to guarantee uniform global asymptotical stability in probability. In particular, an upper bound on the maximally allowable transmission interval is derived explicitly for all stochastic protocols satisfying Lyapunov conditions that guarantee uniform global asymptotic stability in probability. Finally, two numerical examples are presented to demonstrate the derived results.

We present rules to stabilize the origin of a networked system, where data exchanges between the plant and the controller only occur when an output-dependent inequality has been satisfied for a given amount of time. This strategy, called Event-Holding Control (EHC), differs from time-regularized event-triggered control (ETC) techniques, which generate transmissions as soon as a triggering condition is verified and the time elapsed since the last transmission is larger than a given bound. Indeed, the clock involved in EHC is not running continuously after each transmission instant, but only when a criterion is verified. We propose an output-based design of these triggering mechanisms that are robust to additive measurement noise and ensure an input-to-state stability (ISS) property. This EHC scheme naturally has a positive lower bound on the transmission interval. Additionally, we show via an example that, in presence of measurement noise, Zeno-like behavior, where events are generated near the minimum inter-event time consistently, may occur when the system is close to the attractor. We introduce space-regularization to mitigate this issue, resulting in an input-to-state practical stability (ISpS) property rather than ISS.

This work presents a novel masking protocol to secure the communication between a nonlinear plant and a non-linear observer. Communication is secured in two senses. First, the privacy of the plant is preserved during the communication. Second, the protocol can detect a false-data injection attack in the communication link. The masking protocol is based on the use of washout-filters in nonlinear observers and the internal model principle.