We study deterministic, discrete linear time-invariant systems with infinite-horizon discounted quadratic cost. It is well-known that standard stabilizability and detectability properties are not enough in general to conclude stability properties for the system in closed-loop with the optimal controller when the discount factor is small. In this context, we first review some of the stability conditions based on the optimal value function found in the learning and control literature and highlight their conservatism. We then propose novel (necessary and) sufficient conditions, still based on the optimal value function, under which stability of the origin for the optimal closed-loop system is guaranteed. Afterwards, we focus on the scenario where the optimal feedback law is not stabilizing because of the discount factor and the goal is to design an alternative stabilizing near-optimal static state-feedback law. We present both linear matrix inequality-based conditions and a variant of policy iteration to construct such stabilizing near-optimal controllers. The methods are illustrated via numerical examples.

We introduce TROOP, a tree-based Riccati optimistic online planner, that is designed to generate near-optimal control laws for discrete-time switched linear systems with switched quadratic costs. The key challenge that we address is balancing computational resources against control performance, which is important as constructing near-optimal inputs often requires substantial amount of computations. TROOP addresses this trade-off by adopting an online best-first search strategy inspired by A*, allowing for efficient estimates of the optimal value function. The control laws obtained guarantee both near-optimality and stability properties for the closed-loop system. These properties depend on the planning depth, which determines how far into the future the algorithm explores and is closely related to the amount of computations. TROOP thus strikes a balance between computational efficiency and control performance, which is illustrated by numerical simulations on an example.

We analyze the stability of general nonlinear discrete-time stochastic systems controlled by optimal inputs that minimize an infinite-horizon discounted cost. Under a novel stochastic formulation of cost-controllability and detectability assumptions inspired by the related literature on deterministic systems, we prove that uniform semi-global practical recurrence holds for the closed-loop system, where the adjustable parameter is the discount factor. Under additional continuity assumptions, we further prove that this property is robust.

In this article, we analyze the stability properties of stochastic linear systems in closed loop with an optimal policy that minimizes a discounted quadratic cost in expectation. In particular, the linear system is perturbed by both additive and multiplicative stochastic disturbances. We provide conditions under which mean-square boundedness, mean-square stability, and recurrence properties hold for the closed-loop system. We distinguish two cases, when these properties are verified for any value of the discount factor sufficiently close to 1, or when they hold for a fixed value of the discount factor in which case tighter conditions are derived, as illustrated in an example. The analysis exploits properties of the optimal value function, as well as a detectability property of the system with respect to the stage cost, to construct a Lyapunov function for the stochastic linear quadratic regulator problem.

We consider the problem of least squares parameter estimation from single-trajectory data for discrete-time, unstable, closed-loop nonlinear stochastic systems, with linearly parameterised uncertainty. Assuming a region of the state space produces informative data, and the system is sub-exponentially unstable, we establish non-asymptotic guarantees on the estimation error at times where the state trajectory evolves in this region. If the whole state space is informative, high probability guarantees on the error hold for all times. Examples are provided where our results are useful for analysis, but existing results are not.

We consider deterministic nonlinear discrete-time systems whose inputs are generated by policy iteration (PI) for undiscounted cost functions. We first assume that PI is recursively feasible, in the sense that the optimization problems solved at each iteration admit a solution. In this case, we provide novel conditions to establish recursive robust stability properties for a general attractor, meaning that the policies generated at each iteration ensure a robust <inline-formula><tex-math notation="LaTeX">$\mathcal {KL}$</tex-math></inline-formula>-stability property with respect to a general state measure. We then derive novel explicit bounds on the mismatch between the (suboptimal) value function returned by PI at each iteration and the optimal one. However, we show by a counterexample that PI may fail to be recursively feasible, disallowing the mentioned stability and near-optimality guarantees. We therefore also present a modification of PI so that recursive feasibility is guaranteed a priori under mild conditions. This modified algorithm, called PI<inline-formula><tex-math notation="LaTeX">$^{+}$</tex-math></inline-formula>, is shown to preserve the recursive robust stability when the attractor is compact. In addition, PI<inline-formula><tex-math notation="LaTeX">$^{+}$</tex-math></inline-formula> enjoys the same near-optimality properties as its PI counterpart under the same assumptions.

This paper presents a novel event-triggered control (ETC) design framework based on measured $\mathcal{L}_{p}$ norms. We consider a class of systems with finite $\mathcal{L}_{p}$ gain from the network-induced error to a chosen output. The $\mathcal{L}_{p}$ norms of the network-induced error and the chosen output since the last sampling time are used to formulate a class of triggering rules. Based on a small-gain condition, we derive an explicit expression for the $\mathcal{L}_{p}$ gain of the resulting closed-loop systems and present a time-regularization, which can be used to guarantee a lower bound on the inter-sampling times. The proposed framework is based on a different stability- and triggering concept compared to ETC approaches from the literature, and thus may yield new types of dynamical properties for the closed-loop system. However, for specific output choices it can lead to similar triggering rules as “standard” static and dynamic ETC approaches based on input-to-state stability and yields therefore a novel interpretation for some of the existing triggering rules. We illustrate the proposed framework with a numerical example from the literature.