Downloads & Free Reading Options - Results

Contents: Deterministic Dynamic Programming and Viscosity Solutions; Stochastic Control; Robust Control; Optimal Feedback Control of Quantum Systems; Optimal Risk-Sensitive Feedback Control of Quantum Systems. Lecture Notes Collection FreeScience.info ID2207 Obtained from http://users.cecs.anu.edu.au/~Matthew.James/pubs/qc-summer-school-2005-james.pdf http://www.freescience.info/go.php?pagename=books&id=2207

We consider the problem of optimally controlling a system of either ODEs or SDEs with respect to a vector-valued cost functional. Optimisation of the cost is considered with respect to a partial ordering generated by a given proper cone $K$. Since in the vector case minima may not exist, we define vectorial value functions as (Pareto) minimals of the ordering. The main results of this paper are that we demostrate existence of value maps of the system as minimals of the ordering contained in the trade-off manifold of all minimals and also prove that for any direction in the dual cone $K^*$, the projection of the minimal along this direction is a scalar viscosity solution of a certain HJB equation. This allows to construct a family of feedback controls in both the deterministic and stochastic case by using PDE theory.

Optimal control problems of forward-backward stochastic Volterra integral equations (FBSVIEs in short) are formulated and studied. A general duality principle is established for linear backward stochastic integral equation and linear stochastic Fredholm-Volterra integral equation with mean-field. With the help of such a duality principle, together with some other new delicate and subtle skills, Pontryagin type maximum principles are proved for two optimal control problems of FBSVIEs.

We consider a kind of stochastic exit time optimal control problems, in which the cost function is defined through a nonlinear backward stochastic differential equation. We study the regularity of the value function for such a control problem. Then extending Peng's backward semigroup method, we show the dynamic programming principle. Moreover, we prove that the value function is a viscosity solution to the following generalized Hamilton-Jacobi-Bellman equation with Dirichlet boundary: \[ \left\{ \begin{array} [c]{l} \inf\limits_{v\in V}\left\{\mathcal{L}(x,v)u(x)+f(x,u(x),\nabla u(x) \sigma(x,v),v)\right\}=0, \quad x\in D,\medskip\\ u(x)=g(x),\quad x\in \partial D, \end{array} \right. \] where $D$ is a bounded set in $\mathbb{R}^{d}$, $V$ is a compact metric space in $\mathbb{R}^{k}$, and for $u\in C^{2}(D)$ and $(x,v)\in D\times V$, \[\mathcal{L}(x,v)u(x):=\frac{1}{2}\sum_{i,j=1}^{d}(\sigma\sigma^{\ast})_{i,j}(x,v)\frac{\partial^{2}u}{\partial x_{i}\partial x_{j}}(x) +\sum_{i=1}^{d}b_{i}(x,v)\frac{\partial u}{\partial x_{i}}(x). \]

We study methods for solving stochastic control problems of systems of forward-backward mean-field equations with delay, in finite or infinite horizon. Necessary and sufficient maximum principles under partial information are given. The results are applied to solve a recursive utility optimal problem

In this paper, we study the existence of an optimal strategy for the stochastic control of diffusion in general case and a saddle-point for zero-sum stochastic differential games. The problem is formulated as an extended BSDE with logarithmic growth in the $z$-variable and terminal value in some $L^p$ space. We also show the existence and uniqueness of solution of this BSDE.

The control of a surface ship along a prescribed straight-line path is formulated as a stationary, linear, state-variable control problem. The open-loop characteristics of this problem are studied using data for a Mariner type ship at two speeds and varying water depth-to-draft ratio and for the tanker Tokyo Maru at one speed and varying water depth-to-draft ratio using data obtained by Fujino. Optimal stochastic control systems using a Kalman-Bucy filter and a state-feedback controller are designed for both vessels at various conditions. These designs are developed to control the ship when subject to random, zero-mean disturbances. The design disturbances are the yaw moment and sway force due to a passing ship which are modeled by first-order shaping filters in the design derivation. System performance is studied by the evaluation of the Root Mean Square (RMS) response of the controlled ship to the modeled design disturbances and by digital computer simulation of the response of the controlled ship to initial condition errors and the specific disturbances due to a passing ship. The effects of vessel speed and water depth on the design and performance of these controllers are studied in detail. These results yield guidance for the selection of design conditions for the design of constant-gain controllers and provide an assessment of the need for adaptive controllers which can adjust the gains to remain optimal as the ship characteristics change with vessel speed and water depth.

The current problems of the proposal deal with littoral monitoring of large regions of the ocean using single or multiple gliders. This portion of the joint proposal with UCSB deals with the stochastic modeling and control of gliders in the ocean environment. Basically, the goals are to model stochastic flows on gliders, considered as particles forced by the ocean, and design optimal control to keep the gliders in a specific region. That is, the control will be designed to increase loitering time in a specified region. Without controls, the stochastic forcing of the environment will cause the gliders to leave the region of interest in finite time. Controls were designed to optimize the lifetime by minimizing the actuation intervals.

Optimal control of nonlinear saturating systems with stochastic inputs

We prove a version of the maximum principle, in the sense of Pontryagin, for the optimal control of a stochastic partial differential equation driven by a finite dimensional Wiener process. The equation is formulated in a semi-abstract form that allows direct applications to a large class of controlled stochastic parabolic equations. We allow for a diffusion coefficient dependent on the control parameter, and the space of control actions is general, so that in particular we need to introduce two adjoint processes. The second adjoint process takes values in a suitable space of operators on $L^4$.

We formulate a very general framework for optimal dynamic stochastic control problems which allows for a control-dependent informational structure. The issue of informational consistency is investigated. Bellman's principle is formulated and proved. In a series of related results, we expound on the informational structure in the context of (completed) natural filtrations of stochastic processes.

In this paper a new approach to the control of systems represented by stochastic differential equations (SDEs) is developed in which stochastic control is viewed as deterministic control with a particular form of constraint structure. Specifically, the characteristic non-anticipativity property of the control processes is formulated as an equality constraint on the set of possibly anticipative processes. The optimal non-anticipative control is then recovered by minimizing, over the class of possibly anticipating processes, a cost function modified by the inclusion of a Lagrange multiplier term to enforce the nonanticipativity constraint. This unconstrained minimization is carried out pathwise-i.e., separately for each value of a certain random parameter and hence reduces to a parameterized family of deterministic optimal control problems. Solution of the controlled SDEs with anticipative controls are defined by a decomposition method. It is shown that the value function of the control problem is the unique global solution of a robust equation (random partial differential equation) associated to a linear backward Hamilton-Jacobi-Bellman stochastic partial differential equation (HJB SPDE).

Optimal control problems of forward-backward stochastic Volterra integral equations (FBSVIEs, in short) with closed control regions are formulated and studied. Instead of using spike variation method as one may imagine, here we turn to treat the non-convexity of the control regions by borrowing some tools in set-valued analysis and adapting them into our stochastic control systems. A duality principle between linear backward stochastic Volterra integral equations and linear stochastic Fredholm-Volterra integral equations with conditional expectation are derived, which extends and improves the corresponding results in [25], [30]. Some first order necessary optimality conditions for optimal controls of FBSVIEs are established. In contrast with existed common routines to treat the non-convexity of stochastic control problems, here only one adjoint system and one-order differentiability requirements of the coefficients are needed.

Linear-quadratic optimal control problems are considered for mean-field stochastic differential equations with deterministic coefficients. Time-inconsistency feature of the problems is carefully investigated. Both open-loop and closed-loop equilibrium solutions are presented for such kind of problems. Open-loop solutions are presented by means of variational method with decoupling of forward-backward stochastic differential equations, which lead to a Riccati equation system lack of symmetry. Closed-loop solutions are presented by means of multi-person differential games, the limit of which leads to a Riccati equation system with a symmetric structure.

Linear-quadratic optimal control problems are considered for mean-field stochastic differential equations with deterministic coefficients. Time-inconsistency feature of the problems is carefully investigated. Both open-loop and closed-loop equilibrium solutions are presented for such kind of problems. Open-loop solutions are presented by means of variational method with decoupling of forward-backward stochastic differential equations, which lead to a Riccati equation system lack of symmetry. Closed-loop solutions are presented by means of multi-person differential games, the limit of which leads to a Riccati equation system with a symmetric structure.

Linear-quadratic optimal control problems are considered for mean-field stochastic differential equations with deterministic coefficients. Time-inconsistency feature of the problems is carefully investigated. Both open-loop and closed-loop equilibrium solutions are presented for such kind of problems. Open-loop solutions are presented by means of variational method with decoupling of forward-backward stochastic differential equations, which lead to a Riccati equation system lack of symmetry. Closed-loop solutions are presented by means of multi-person differential games, the limit of which leads to a Riccati equation system with a symmetric structure.

Stochastic maximum principle of nonlinear controlled forward-backward systems, where the set of strict (classical) controls need not be convex and the diffusion coefficient depends explicitly on the variable control, is an open problem impossible to solve by the classical method of spike variation. In this paper, we introduce a new approach to solve this open problem and we establish necessary as well as sufficient conditions of optimality, in the form of global stochastic maximum principle, for two models. The first concerns the relaxed controls, who are a measure-valued processes. The second is a restriction of the first to strict control problems.

In this paper, the design of the optimal decentralized state-feedback controllers is considered for a wireless sensor and actuator network (WSAN) with stochastic network-induced delays and packet losses. In particular, taking advantage of multiple controllers, we model the WSAN as a wireless networked control system (NCS) with decentralized controllers, and then formulate the stochastic optimal state-feedback control problem as a non-cooperative linear quadratic (LQ) game. The optimal control law of each controller is obtained that is a function of the current plant state and all past control signals. The performance of the proposed stochastic optimal control algorithm is investigated using both a genetic control system and a load frequency control (LFC) system in power grid.

Output-Feedback Stochastic Model Predictive Control based on Stochastic Optimal Control for nonlinear systems is computationally intractable because of the need to solve a Finite Horizon Stochastic Optimal Control Problem. However, solving this problem leads to an optimal probing nature of the resulting control law, called dual control, which trades off benefits of exploration and exploitation. In practice, intractability of Stochastic Model Predictive Control is typically overcome by replacement of the underlying Stochastic Optimal Control problem by more amenable approximate surrogate problems, which however come at a loss of the optimal probing nature of the control signals. While probing can be superimposed in some approaches, this is done sub-optimally. In this paper, we examine approximation of the system dynamics by a Partially Observable Markov Decision Process with its own Finite Horizon Stochastic Optimal Control Problem, which can be solved for an optimal control policy, implemented in receding horizon fashion. This procedure enables maintaining probing in the control actions. We further discuss a numerical example in healthcare decision making, highlighting the duality in stochastic optimal receding horizon control.

This article is concerned with an optimal control problem derived by mean-field forward-backward stochastic differential equation with noisy observation, where the drift coefficients of the state equation and the observation equation are linear with respect to the state and its expectation. The control problem is different from the existing literature on optimal control for mean-field stochastic systems, and has more applications in mathematical finance, e.g., asset-liability management problem with recursive utility, systematic risk model. Using a backward separation method with a decomposition technique, two optimality conditions along with two coupled forward-backward optimal filters are derived. Several linear-quadratic optimal control problems for mean-field forward-backward stochastic differential equations are studied. Closed-form optimal solutions are explicitly obtained in detailed situations.

This work is devoted to the study of optimal control of stochastic functional differential equations (SFDEs) and its application to mathematical finance. By using the Dynkin formula and solution of the Dirichlet-Poisson problem, the Hamilton-Jacobi-Bellman (HJB) equation and the converse HJB equation are derived. Furthermore, applications are given to an optimal portfolio selection problem.

This paper is devoted to present a method of proving verification theorems for stochastic optimal control of finite dimensional diffusion processes without control in the diffusion term. The value function is assumed to be continuous in time and once differentiable in the space variable ($C^{0,1}$) instead of once differentiable in time and twice in space ($C^{1,2}$), like in the classical results. The results are obtained using a time dependent Fukushima - Dirichlet decomposition proved in a companion paper by the same authors using stochastic calculus via regularization. Applications, examples and comparison with other similar results are also given.

This paper is devoted to present a method of proving verification theorems for stochastic optimal control of finite dimensional diffusion processes without control in the diffusion term. The value function is assumed to be continuous in time and once differentiable in the space variable ($C^{0,1}$) instead of once differentiable in time and twice in space ($C^{1,2}$), like in the classical results. The results are obtained using a time dependent Fukushima - Dirichlet decomposition proved in a companion paper by the same authors using stochastic calculus via regularization. Applications, examples and comparison with other similar results are also given.

Motion control of underway replenishment operations is achieved through the use of sliding mode control with a Linear Quadratic Gaussian compensator design. External disturbances include first-order wave force and moment as well as slowly varying interaction forces and moments between the two ships. Feedback control is used to provide adequate stability of motions while feedforward control with disturbance estimation and compensation achieves the desired steady state accuracy. The results demonstrate that satisfactory path keeping during operations can be maintained for various ship proximity distances and environmental conditions.

A linear quadratic optimal stochastic control problem with random coefficients and indefinite state/control weight costs is usually linked to an indefinite stochastic Riccati equation (SRE) which is a matrix-valued quadratic backward stochastic differential equation along with an algebraic constraint involving the unknown. Either the optimal control problem or the SRE is solvable only if the given data satisfy a certain structure condition that has yet to be precisely defined. In this paper, by introducing a notion of subsolution for the SRE, we derive several novel sufficient conditions for the existence and uniqueness of the solution to the SRE and for the solvability of the associated optimal stochastic control problem.

We consider the performance of periodically driven stochastic heat engines in the linear response regime. Reaching the theoretical bounds for efficiency and efficiency at maximum power typically requires full control over the design and the driving of the system. We develop a framework which allows to quantify the role that limited control over the system has on the performance. Specifically, we show that optimizing the driving entering the work extraction for a given temperature protocol leads to a universal, one-parameter dependence for both maximum efficiency and maximum power as a function of efficiency. In particular, we show that reaching Carnot efficiency (and, hence, Curzon-Ahlborn efficiency at maximum power) requires to have control over the amplitude of the full Hamiltonian of the system. Since the kinetic energy cannot be controlled by an external parameter, heat engines based on underdamped dynamics can typically not reach Carnot efficiency. We illustrate our general theory with a paradigmatic case study of a heat engine consisting of an underdamped charged particle in a modulated two-dimensional harmonic trap in the presence of a magnetic field.

This paper presents a method to approximately solve stochastic optimal control problems in which the cost function and the system dynamics are polynomial. For stochastic systems with polynomial dynamics, the moments of the state can be expressed as a, possibly infinite, system of deterministic linear ordinary differential equations. By casting the problem as a deterministic control problem in moment space, semidefinite programming is used to find a lower bound on the optimal solution. The constraints in the semidefinite program are imposed by the ordinary differential equations for moment dynamics and semidefiniteness of the outer product of moments. From the solution to the semidefinite program, an approximate optimal control strategy can be constructed using a least squares method. In the linear quadratic case, the method gives an exact solution to the optimal control problem. In more complex problems, an infinite number of moment differential equations would be required to compute the optimal control law. In this case, we give a procedure to increase the size of the semidefinite program, leading to increasingly accurate approximations to the true optimal control strategy.

Path integral control solves a class of stochastic optimal control problems with a Monte Carlo (MC) method for an associated Hamilton-Jacobi-Bellman (HJB) equation. The MC approach avoids the need for a global grid of the domain of the HJB equation and, therefore, path integral control is in principle applicable to control problems of moderate to large dimension. The class of problems path integral control can solve, however, is defined by requirements on the cost function, the noise covariance matrix and the control input matrix. We relax the requirements on the cost function by introducing a new state that represents an augmented running cost. In our new formulation the cost function can contain stochastic integral terms and linear control costs, which are important in applications in engineering, economics and finance. We find an efficient numerical implementation of our grid-free MC approach and demonstrate its performance and usefulness in examples from hierarchical electric load management. The dimension of one of our examples is large enough to make classical grid-based HJB solvers impractical.

In this research we continue our investigations of approximation techniques for a wide class of discrete and continuous time stochastic control problems. Emphasis is placed on the development and theoretical justification of techniques which yield computationally tractable algorithms that answer the following: (1) approximations to the optimal cost and the cost of using particular control; (2) approximations to the optimal control; (3) evaluation of the relative performance of two controls; and (4) estimates for the deterioration in system performance due to the failure to observe system components. (Author)

In this research we continue our investigations of approximation techniques for a wide class of discrete and continuous time stochastic control problems. Emphasis is placed on the development and theoretical justification of techniques which yield computationally tractable algorithms that answer the following: (1) approximations to the optimal cost and the cost of using particular control; (2) approximations to the optimal control; (3) evaluation of the relative performance of two controls; and (4) estimates for the deterioration in system performance due to the failure to observe system components. (Author)

A general method for parallel and vector numerical solutions of stochastic dynamic programming problems is described for optimal control of general nonlinear, continuous time, multibody dynamical systems, perturbed by Poisson as well as Gaussian random white noise. Possible applications include lumped flight dynamics models for uncertain environments, such as large scale and background random atmospheric fluctuations. The numerical formulation is highly suitable for a vector multiprocessor or vectorizing supercomputer, and results exhibit high processor efficiency and numerical stability. Advanced computing techniques, data structures, and hardware help alleviate Bellman's curse of dimensionality in dynamic programming computations.

Solutions of stochastic Volterra (integral) equations are not Markov processes, and therefore classical methods, like dynamic programming, cannot be used to study optimal control problems for such equations. However, we show that by using {\em Malliavin calculus} it is possible to formulate a modified functional type of {\em maximum principle} suitable for such systems. This principle also applies to situations where the controller has only partial information available to base her decisions upon. We present both a sufficient and a necessary maximum principle of this type, and then we use the results to study some specific examples. In particular, we solve an optimal portfolio problem in a financial market model with memory.

In this paper we investigate a kind of optimal control problem of coupled forward-backward stochastic system with jumps whose cost functional is defined through a coupled forward-backward stochastic differential equation with Brownian motion and Poisson random measure. For this end, we first study the regularity of solutions for this kind of forward-backward stochastic differential equations. We obtain that the value function is a deterministic function and satisfies the dynamic programming principle for this kind of optimal control problem. Moreover, we prove that the value functions is a viscosity solutions of the associated Hamilton-Jacobi-Bellman equations with integral-differential operators.

We consider a stochastic control problem, where the control domain is convex and the system is governed by a nonlinear backward stochastic differential equation. With a L1 terminal data, we derive necessary optimality conditions in the form of stochastic maximum principle.

Recent results in the study of the Hamilton Jacobi Bellman (HJB) equation have led to the discovery of a formulation of the value function as a linear Partial Differential Equation (PDE) for stochastic nonlinear systems with a mild constraint on their disturbances. This has yielded promising directions for research in the planning and control of nonlinear systems. This work proposes a new method obtaining approximate solutions to these linear stochastic optimal control (SOC) problems. A candidate polynomial with variable coefficients is proposed as the solution to the SOC problem. A Sum of Squares (SOS) relaxation is then taken to the partial differential constraints, leading to a hierarchy of semidefinite relaxations with improving sub-optimality gap. The resulting approximate solutions are shown to be guaranteed over- and under-approximations for the optimal value function.

This paper is concerned with a linear quadratic (LQ, for short) optimal control problem for mean-field backward stochastic differential equations (MF-BSDE, for short) driven by a Poisson random martingale measure and a Brownian motion. Firstly, by the classic convex variation principle, the existence and uniqueness of the optimal control is established. Secondly, the optimal control is characterized by the stochastic Hamilton system which turns out to be a linear fully coupled mean-field forward-backward stochastic differential equation with jumps by the duality method. Thirdly, in terms of a decoupling technique, the stochastic Hamilton system is decoupled by introducing two Riccati equations and a MF-BSDE with jumps. Then an explicit representation for the optimal control is obtained.

This work presents a multiscale framework to solve a class of stochastic optimal control problems in the context of robot motion planning and control in a complex environment. In order to handle complications resulting from a large decision space and complex environmental geometry, two key concepts are adopted: (a) a diffusion wavelet representation of the Markov chain for hierarchical abstraction of the state space; and (b) a desirability function-based representation of the Markov decision process (MDP) to efficiently calculate the optimal policy. In the proposed framework, a global plan that compressively takes into account the long time/length-scale state transition is first obtained by approximately solving an MDP whose desirability function is represented by coarse scale bases in the hierarchical abstraction. Then, a detailed local plan is computed by solving an MDP that considers wavelet bases associated with a focused region of the state space, guided by the global plan. The resulting multiscale plan is utilized to finally compute a continuous-time optimal control policy within a receding horizon implementation. Two numerical examples are presented to demonstrate the applicability and validity of the proposed approach.

Motion planning and control problems are embedded and essential in almost all robotics applications. These problems are often formulated as stochastic optimal control problems and solved using dynamic programming algorithms. Unfortunately, most existing algorithms that guarantee convergence to optimal solutions suffer from the curse of dimensionality: the run time of the algorithm grows exponentially with the dimension of the state space of the system. We propose novel dynamic programming algorithms that alleviate the curse of dimensionality in problems that exhibit certain low-rank structure. The proposed algorithms are based on continuous tensor decompositions recently developed by the authors. Essentially, the algorithms represent high-dimensional functions (e.g., the value function) in a compressed format, and directly perform dynamic programming computations (e.g., value iteration, policy iteration) in this format. Under certain technical assumptions, the new algorithms guarantee convergence towards optimal solutions with arbitrary precision. Furthermore, the run times of the new algorithms scale polynomially with the state dimension and polynomially with the ranks of the value function. This approach realizes substantial computational savings in "compressible" problem instances, where value functions admit low-rank approximations. We demonstrate the new algorithms in a wide range of problems, including a simulated six-dimensional agile quadcopter maneuvering example and a seven-dimensional aircraft perching example. In some of these examples, we estimate computational savings of up to ten orders of magnitude over standard value iteration algorithms. We further demonstrate the algorithms running in real time on board a quadcopter during a flight experiment under motion capture.

Motion planning and control problems are embedded and essential in almost all robotics applications. These problems are often formulated as stochastic optimal control problems and solved using dynamic programming algorithms. Unfortunately, most existing algorithms that guarantee convergence to optimal solutions suffer from the curse of dimensionality: the run time of the algorithm grows exponentially with the dimension of the state space of the system. We propose novel dynamic programming algorithms that alleviate the curse of dimensionality in problems that exhibit certain low-rank structure. The proposed algorithms are based on continuous tensor decompositions recently developed by the authors. Essentially, the algorithms represent high-dimensional functions (e.g., the value function) in a compressed format, and directly perform dynamic programming computations (e.g., value iteration, policy iteration) in this format. Under certain technical assumptions, the new algorithms guarantee convergence towards optimal solutions with arbitrary precision. Furthermore, the run times of the new algorithms scale polynomially with the state dimension and polynomially with the ranks of the value function. This approach realizes substantial computational savings in "compressible" problem instances, where value functions admit low-rank approximations. We demonstrate the new algorithms in a wide range of problems, including a simulated six-dimensional agile quadcopter maneuvering example and a seven-dimensional aircraft perching example. In some of these examples, we estimate computational savings of up to ten orders of magnitude over standard value iteration algorithms. We further demonstrate the algorithms running in real time on board a quadcopter during a flight experiment under motion capture.

In this research we continue our investigations of approximation techniques for a wide class of discrete and continuous time stochastic control problems. Emphasis is placed on the development and theoretical justification of techniques which yield computationally tractable algorithms that answer the following: (1) approximations to the optimal cost and the cost of using particular control; (2) approximations to the optimal control; (3) evaluation of the relative performance of two controls; and (4) estimates for the deterioration in system performance due to the failure to observe system components. (Author)

A number of results have been obtained for problems of stochastic control and stochastic differential games. Stochastic linear-quadratic, continuous time, stochastic control problems are solved for systems with noise processes that are arbitrary fractional Brownian motions or more generally square integrable continuous processes. Linear-quadratic control problems for stochastic partial differential equations with fractional Brownian motions having the Hurst parameter in the interval (1/2,1), which includes the long range dependent processes, are solved. A direct method for solving stochastic control problems is given that does not require the solution of Hamilton-Jacobi-Bellman equations or the solution of backward stochastic differential equations. Both of these latter methods present serious difficulties for solutions. A stochastic control problem is explicitly solved for a system that evolves in the two-sphere which is useful for applications. Linear exponential quadratic Gaussian control problems for both continuous and discrete time systems are solved in a simple direct way. Linear exponential-quadratic control problems for stochastic partial differential equations are explicitly solved. Discrete time linear quadratic control problems for systems with correlated noise are explicitly solved. Linear-quadratic stochastic differential games are explicitly solved for systems with arbitrary noise. Stochastic differential games for stochastic partial differential equations with fractional Brownian motions are explicitly solved. Ergodic control problems for linear exponential quadratic control for stochastic partial differential equations with fractional brownian motions are explicitly solved. Stochastic differential games that evolve in spheres are explicitly solved. Various aspects of stochastic adaptive control are described. Some linear quadratic control problems with state dependent noise are explicitly solved.

ADA001248

We consider a variant of the classical linear quadratic Gaussian regulator (LQG) in which penalties on the endpoint state are replaced by the specification of the terminal state distribution. The resulting theory considerably differs from LQG as well as from formulations that bound the probability of violating state constraints. We develop results for optimal state-feedback control in the two cases where i) steering of the state distribution is to take place over a finite window of time with minimum energy, and ii) the goal is to maintain the state at a stationary distribution over an infinite horizon with minimum power. For both problems the distribution of noise and state are Gaussian. In the first case, we show that provided the system is controllable, the state can be steered to any terminal Gaussian distribution over any specified finite time-interval. In the second case, we characterize explicitly the covariance of admissible stationary state distributions that can be maintained with constant state-feedback control. The conditions for optimality are expressed in terms of a system of dynamically coupled Riccati equations in the finite horizon case and in terms of algebraic conditions for the stationary case. In the case where the noise and control share identical input channels, the Riccati equations for finite-horizon steering become homogeneous and can be solved in closed form. The present paper is largely based on our recent work in arxiv.org/abs/1408.2222, arxiv.org/abs/1410.3447 and presents an overview of certain key results.

This paper is concerned with Markov diffusion processes which obey stochastic differential equations depending on a small parameter E. The parameter enters as a coefficient in the noise term of the stochastic differential equation. The Ventcel-Freidlin estimates give asymptotic formulas (as E approaches 0) for such quantities as the probability of exit from a region D through a given portion N of the boundary increment of D, the mean exit time, and the probability of exit by a given time T. A new method to obtain such estimates is given, using ideas from stochastic control theory.

A major focus of the research of this grant has been the control of stochastic systems with a large family of noise processes that are important for modeling and that have not previously been used or solved for control problems. Among these noise processes are the family of fractional Brownian motions. While Brownian motion is included in this family, all of the other members are neither semimartingales nor Markov processes. Thus the usual stochastic calculus or the methods from Markov processes cannot be used. However, empirical data in a wide variety of physical phenomena, such as turbulence and telecommunications, provides evidence of the necessity for using these fractional Brownian motions in mathematical models. Thus it is important to consider control systems that are driven by an arbitrary fractional Brownian motion.

This paper is mainly concerned with the solutions to both forward and backward mean-field stochastic partial differential equation and the corresponding optimal control problem for mean-field stochastic partial differential equation. We first prove the continuous dependence theorems of forward and backward mean-field stochastic partial differential equations and show the existence and uniqueness of solutions to them. Then we establish necessary and sufficient optimality conditions of the control problem in the form of Pontryagin's maximum principles. To illustrate the theoretical results, we apply stochastic maximum principles to study an example, an infinite-dimensional linear-quadratic control problem of mean-field type. Further an application to a Cauchy problem for a controlled stochastic linear PDE of mean-field type are studied.

This report documents codes for the numerical solution of control and optimal control problems for diffusion or reflected diffusion models in dimensions two to four and for continuous time Markov chain control problems where the state space of the chain is a grid in such a Euclidean space. The control appears linearly in the dynamics and cost function but otherwise the process and cost function are general. The underlying numerical methods use efficient forms of the approximation in policy space and multigrid type methods, based on the Markov chain approximation method of 7.

The primary goal of this research has been the optimal control of linear and nonlinear systems driven by fractional Brownian motions and other stochastic processes. For the control of both continuous time and discrete time finite dimensional linear systems with quadratic cost functionals and arbitrary noise processes with finite second moments, explicit optimal controls are determined. Linear-quadratic control problems for stochastic partial differential equations driven by fractional Brownian motions are explicitly solved. For the control of a continuous time linear systems with Brownian motion or a discrete time linear system with a white Gaussian noise and costs that are the exponential of quadratic functionals are solved in a simple, direct way.

In the Control Systems Division of the Systems Dynamics Laboratory of the NASA/MSFC, a Ground Facility (GF), in which the dynamics and control system concepts being considered for Large Space Structures (LSS) applications can be verified, was designed and built. One of the important aspects of the GF is to design an analytical model which will be as close to experimental data as possible so that a feasible control law can be generated. Using Hyland's Maximum Entropy/Optimal Projection Approach, a procedure was developed in which the maximum entropy principle is used for stochastic modeling and the optimal projection technique is used for a reduced-order dynamic compensator design for a high-order plant.

This paper presents a method for finding optimal controls of nonlinear systems subject to random excitations. The method is capable to generate global control solutions when state and control constraints are present. The solution is global in the sense that controls for all initial conditions in a region of the state space are obtained. The approach is based on Bellman's Principle of optimality, the Gaussian closure and the Short-time Gaussian approximation. Examples include a system with a state-dependent diffusion term, a system in which the infinite hierarchy of moment equations cannot be analytically closed, and an impact system with a elastic boundary. The uncontrolled and controlled dynamics are studied by creating a Markov chain with a control dependent transition probability matrix via the Generalized Cell Mapping method. In this fashion, both the transient and stationary controlled responses are evaluated. The results show excellent control performances.