We study deterministic, discrete linear time-invariant systems with infinite-horizon discounted quadratic cost. It is well-known that standard stabilizability and detectability properties are not enough in general to conclude stability properties for the system in closed-loop with the optimal controller when the discount factor is small. In this context, we first review some of the stability conditions based on the optimal value function found in the learning and control literature and highlight their conservatism. We then propose novel (necessary and) sufficient conditions, still based on the optimal value function, under which stability of the origin for the optimal closed-loop system is guaranteed. Afterwards, we focus on the scenario where the optimal feedback law is not stabilizing because of the discount factor and the goal is to design an alternative stabilizing near-optimal static state-feedback law. We present both linear matrix inequality-based conditions and a variant of policy iteration to construct such stabilizing near-optimal controllers. The methods are illustrated via numerical examples.

We introduce TROOP, a tree-based Riccati optimistic online planner, that is designed to generate near-optimal control laws for discrete-time switched linear systems with switched quadratic costs. The key challenge that we address is balancing computational resources against control performance, which is important as constructing near-optimal inputs often requires substantial amount of computations. TROOP addresses this trade-off by adopting an online best-first search strategy inspired by A*, allowing for efficient estimates of the optimal value function. The control laws obtained guarantee both near-optimality and stability properties for the closed-loop system. These properties depend on the planning depth, which determines how far into the future the algorithm explores and is closely related to the amount of computations. TROOP thus strikes a balance between computational efficiency and control performance, which is illustrated by numerical simulations on an example.

We analyze the stability of general nonlinear discrete-time stochastic systems controlled by optimal inputs that minimize an infinite-horizon discounted cost. Under a novel stochastic formulation of cost-controllability and detectability assumptions inspired by the related literature on deterministic systems, we prove that uniform semi-global practical recurrence holds for the closed-loop system, where the adjustable parameter is the discount factor. Under additional continuity assumptions, we further prove that this property is robust.

In this article, we analyze the stability properties of stochastic linear systems in closed loop with an optimal policy that minimizes a discounted quadratic cost in expectation. In particular, the linear system is perturbed by both additive and multiplicative stochastic disturbances. We provide conditions under which mean-square boundedness, mean-square stability, and recurrence properties hold for the closed-loop system. We distinguish two cases, when these properties are verified for any value of the discount factor sufficiently close to 1, or when they hold for a fixed value of the discount factor in which case tighter conditions are derived, as illustrated in an example. The analysis exploits properties of the optimal value function, as well as a detectability property of the system with respect to the stage cost, to construct a Lyapunov function for the stochastic linear quadratic regulator problem.

We consider the problem of least squares parameter estimation from single-trajectory data for discrete-time, unstable, closed-loop nonlinear stochastic systems, with linearly parameterised uncertainty. Assuming a region of the state space produces informative data, and the system is sub-exponentially unstable, we establish non-asymptotic guarantees on the estimation error at times where the state trajectory evolves in this region. If the whole state space is informative, high probability guarantees on the error hold for all times. Examples are provided where our results are useful for analysis, but existing results are not.

We consider deterministic nonlinear discrete-time systems whose inputs are generated by policy iteration (PI) for undiscounted cost functions. We first assume that PI is recursively feasible, in the sense that the optimization problems solved at each iteration admit a solution. In this case, we provide novel conditions to establish recursive robust stability properties for a general attractor, meaning that the policies generated at each iteration ensure a robust <inline-formula><tex-math notation="LaTeX">$\mathcal {KL}$</tex-math></inline-formula>-stability property with respect to a general state measure. We then derive novel explicit bounds on the mismatch between the (suboptimal) value function returned by PI at each iteration and the optimal one. However, we show by a counterexample that PI may fail to be recursively feasible, disallowing the mentioned stability and near-optimality guarantees. We therefore also present a modification of PI so that recursive feasibility is guaranteed a priori under mild conditions. This modified algorithm, called PI<inline-formula><tex-math notation="LaTeX">$^{+}$</tex-math></inline-formula>, is shown to preserve the recursive robust stability when the attractor is compact. In addition, PI<inline-formula><tex-math notation="LaTeX">$^{+}$</tex-math></inline-formula> enjoys the same near-optimality properties as its PI counterpart under the same assumptions.

This paper presents a novel event-triggered control (ETC) design framework based on measured $\mathcal{L}_{p}$ norms. We consider a class of systems with finite $\mathcal{L}_{p}$ gain from the network-induced error to a chosen output. The $\mathcal{L}_{p}$ norms of the network-induced error and the chosen output since the last sampling time are used to formulate a class of triggering rules. Based on a small-gain condition, we derive an explicit expression for the $\mathcal{L}_{p}$ gain of the resulting closed-loop systems and present a time-regularization, which can be used to guarantee a lower bound on the inter-sampling times. The proposed framework is based on a different stability- and triggering concept compared to ETC approaches from the literature, and thus may yield new types of dynamical properties for the closed-loop system. However, for specific output choices it can lead to similar triggering rules as “standard” static and dynamic ETC approaches based on input-to-state stability and yields therefore a novel interpretation for some of the existing triggering rules. We illustrate the proposed framework with a numerical example from the literature.

Given a discounted cost, we study deterministic discrete-time systems whose inputs are generated by policy iteration (PI). We provide novel near-optimality and stability properties, while allowing for non-stabilizing initial policies. That is, we first give novel bounds on the mismatch between the value function generated by PI and the optimal value function, which are less conservative in general than those encountered in the dynamic programming literature for the considered class of systems. Then, we show that the systems in closed-loop with policies generated by PI are stabilizing under mild conditions, after a finite (and known) number of iterations.

The state of charge (SOC) of lithium-ion batteries needs to be accurately estimated for safety and reliability purposes. For battery packs made of a large number of cells, it is not always feasible to design one SOC estimator per cell due to limited computational resources. Instead, only the minimum and the maximum SOC need to be estimated. The challenge is that the cells having the minimum and maximum SOC typically change over time. In this context, we present a low-dimensional hybrid estimator of the minimum (maximum) SOC, whose convergence is analytically guaranteed. We consider for this purpose a battery consisting of cells interconnected in series, which we model by electrical equivalent circuit models. We then present the hybrid estimator, which runs an observer designed for a single cell at any time instant, selected by a switching-like logic mechanism. We establish a practical exponential stability property for the estimation error on the minimum (maximum) SOC thereby guaranteeing the ability of the hybrid scheme to generate accurate estimates of the minimum (maximum) SOC. The analysis relies on non-smooth hybrid Lyapunov techniques. A numerical illustration is provided to showcase the relevance of the proposed approach.

This paper studies the stabilization problem of networked control systems (NCSs) with random packet dropouts caused by stochastic channels. To describe the effects of stochastic channels on the information transmission, the transmission times are assumed to be deterministic, whereas the packet transmission is assumed to be random. We first propose a stochastic scheduling protocol to model random packet dropouts, and address the properties of the proposed stochastic scheduling protocol. The proposed scheduling protocol provides a unified modelling framework for a general class of random packet dropouts due to different stochastic channels. Next, the proposed scheduling protocol is embedded into the closed-loop system, which leads to a stochastic hybrid model for NCSs with random packet dropouts. Based on this stochastic hybrid model, we follow the emulation approach to establish sufficient conditions to guarantee uniform global asymptotical stability in probability. In particular, an upper bound on the maximally allowable transmission interval is derived explicitly for all stochastic protocols satisfying Lyapunov conditions that guarantee uniform global asymptotic stability in probability. Finally, two numerical examples are presented to demonstrate the derived results.

We present rules to stabilize the origin of a networked system, where data exchanges between the plant and the controller only occur when an output-dependent inequality has been satisfied for a given amount of time. This strategy, called Event-Holding Control (EHC), differs from time-regularized event-triggered control (ETC) techniques, which generate transmissions as soon as a triggering condition is verified and the time elapsed since the last transmission is larger than a given bound. Indeed, the clock involved in EHC is not running continuously after each transmission instant, but only when a criterion is verified. We propose an output-based design of these triggering mechanisms that are robust to additive measurement noise and ensure an input-to-state stability (ISS) property. This EHC scheme naturally has a positive lower bound on the transmission interval. Additionally, we show via an example that, in presence of measurement noise, Zeno-like behavior, where events are generated near the minimum inter-event time consistently, may occur when the system is close to the attractor. We introduce space-regularization to mitigate this issue, resulting in an input-to-state practical stability (ISpS) property rather than ISS.

This work presents a novel masking protocol to secure the communication between a nonlinear plant and a non-linear observer. Communication is secured in two senses. First, the privacy of the plant is preserved during the communication. Second, the protocol can detect a false-data injection attack in the communication link. The masking protocol is based on the use of washout-filters in nonlinear observers and the internal model principle.