Downloads & Free Reading Options - Results

This paper is concerned with the optimal control of continuous-time Markov processes. The admissible control laws are based on white-noise corrupted observations of a function on the state processes. A 'separated' control problem is introduced, whose states are probability measures on the original state space. The original and separated control problems are related via the nonlinear filter equation. The existence of a minimum for the separated problem is established. Under more restrictive assumptions it is shown that the minimum expected cost for the separated problem equals the infimum of expected costs for the original problem with partially observed states.

Assuring safety in discrete time stochastic hybrid systems is particularly difficult when only noisy or incomplete observations of the state are available. We first review a formulation of the probabilistic safety problem under noisy hybrid observations as a dynamic program over an equivalent information state. Two methods for approximately solving the dynamic program are presented. The first method approximates the hybrid system as an equivalent finite state Markov decision process, so that the information state is a probability mass function. The second approach approximates an indicator function over the safe region using radial basis functions, to represent the information state as a Gaussian mixture. In both cases, we discretize the hybrid observation process and generate a sampled set of information states, then use point-based value iteration to under-approximate the safety probability and synthesize a suboptimal control policy. We obtain error bounds and convergence results in both cases, assuming switched affine dynamics and additive Gaussian noise on the continuous states and observations. We compare the performance of the finite state and Gaussian mixture approaches on a simple numerical example.

A class of multidimensional stochastic control problems with noisy data and bounded controls encountered in aerospace design is examined. The emphasis is on suboptimal design, the optimality being taken in quadratic mean sense. To that effect the problem is viewed as a stochastic version of the Lurie problem known from nonlinear control theory. The main result is a separation theorem (involving a nonlinear Kalman-like filter) suitable for Lurie-type approximations. The theorem allows for discontinuous characteristics. As a byproduct the existence of strong solutions to a class of non-Lipschitzian stochastic differential equations in dimensions is proven.