Downloads & Free Reading Options - Results

We are concerned with the linear-quadratic optimal stochastic control problem with random coefficients. Under suitable conditions, we prove that the value field $V(t,x,\omega), (t,x,\omega)\in [0,T]\times R^n\times \Omega$, is quadratic in $x$, and has the following form: $V(t,x)=\langle K_tx, x\rangle$ where $K$ is an essentially bounded nonnegative symmetric matrix-valued adapted processes. Using the dynamic programming principle (DPP), we prove that $K$ is a continuous semi-martingale of the form $$K_t=K_0+\int_0^t \, dk_s+\sum_{i=1}^d\int_0^tL_s^i\, dW_s^i, \quad t\in [0,T]$$ with $k$ being a continuous process of bounded variation and $$E\left[\left(\int_0^T|L_s|^2\, ds\right)^p\right]

In this paper, we show how a simulated Markov decision process (MDP) built by the so-called \emph{baseline} policies, can be used to compute a different policy, namely the \emph{simulated optimal} policy, for which the performance of this policy is guaranteed to be better than the baseline policy in the real environment. This technique has immense applications in fields such as news recommendation systems, health care diagnosis and digital online marketing. Our proposed algorithm iteratively solves for a "good" policy in the simulated MDP in an offline setting. Furthermore, we provide a performance bound on sub-optimality for the control policy generated by the proposed algorithm.

In this paper, the optimal control problem of neutral stochastic functional differential equation (NSFDE) is discussed. A class of so-called neutral backward stochastic functional equations of Volterra type (VNBSFEs) are introduced as the adjoint equation. The existence and uniqueness of VNBSFE is established. The Pontryagin maximum principle is constructed for controlled NSFDE with Lagrange type cost functional.

We consider a problem of optimal control of an infinite horizon system governed by forward-backward stochastic differential equations with delay. Sufficient and necessary maximum principles for optimal control under partial information in infinite horizon are derived. We illustrate our results by an application to a problem of optimal consumption with respect to recursive utility from a cash flow with delay.

This paper is concerned with a stochastic linear quadratic (LQ, for short) optimal control problem. The notions of open-loop and closed-loop solvabilities are introduced. A simple example shows that these two solvabilities are different. Closed-loop solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. Conditions ensuring the convexity of the cost functional are discussed, including the issue that how negative the control weighting matrix-valued function R(s) can be. Finiteness of the LQ problem is characterized by the convergence of the solutions to a family of Riccati equations. Then, a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. Finally, an illustrative example is presented.

In this paper, we present an Uzawa-based heuristic that is adapted to some type of stochastic optimal control problems. More precisely, we consider dynamical systems that can be divided into small-scale independent subsystems, though linked through a static almost sure coupling constraint at each time step. This type of problem is common in production/portfolio management where subsystems are, for instance, power units, and one has to supply a stochastic power demand at each time step. We outline the framework of our approach and present promising numerical results on a simplified power management problem.

This paper presents a method for finding optimal controls of nonlinear systems subject to random excitations. The method is capable to generate global control solutions when state and control constraints are present. The solution is global in the sense that controls for all initial conditions in a region of the state space are obtained. The approach is based on Bellman's Principle of optimality, the Gaussian closure and the Short-time Gaussian approximation. Examples include a system with a state-dependent diffusion term, a system in which the infinite hierarchy of moment equations cannot be analytically closed, and an impact system with a elastic boundary. The uncontrolled and controlled dynamics are studied by creating a Markov chain with a control dependent transition probability matrix via the Generalized Cell Mapping method. In this fashion, both the transient and stationary controlled responses are evaluated. The results show excellent control performances.

In the Control Systems Division of the Systems Dynamics Laboratory of the NASA/MSFC, a Ground Facility (GF), in which the dynamics and control system concepts being considered for Large Space Structures (LSS) applications can be verified, was designed and built. One of the important aspects of the GF is to design an analytical model which will be as close to experimental data as possible so that a feasible control law can be generated. Using Hyland's Maximum Entropy/Optimal Projection Approach, a procedure was developed in which the maximum entropy principle is used for stochastic modeling and the optimal projection technique is used for a reduced-order dynamic compensator design for a high-order plant.

This paper is the first part of our series work to establish pointwise second-order necessary conditions for stochastic optimal controls. In this part, both drift and diffusion terms may contain the control variable but the control region is assumed to be convex. Under some assumptions in terms of Malliavin calculus, we establish the desired necessary condition for stochastic singular optimal controls in the classical sense.

We prove a version of the maximum principle, in the sense of Pontryagin, for the optimal control of a stochastic partial differential equation driven by a finite dimensional Wiener process. The equation is formulated in a semi-abstract form that allows direct applications to a large class of controlled stochastic parabolic equations. We allow for a diffusion coefficient dependent on the control parameter, and the space of control actions is general, so that in particular we need to introduce two adjoint processes. The second adjoint process takes values in a suitable space of operators on $L^4$.

We formulate a very general framework for optimal dynamic stochastic control problems which allows for a control-dependent informational structure. The issue of informational consistency is investigated. Bellman's principle is formulated and proved. In a series of related results, we expound on the informational structure in the context of (completed) natural filtrations of stochastic processes.

For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. This notion can be informally introduced as follows. At the very first time step $t_0$, the decision maker formulates an optimization problem that yields optimal decision rules for all the forthcoming time step $t_0, t_1, ..., T$; at the next time step $t_1$, he is able to formulate a new optimization problem starting at time $t_1$ that yields a new sequence of optimal decision rules. This process can be continued until final time $T$ is reached. A family of optimization problems formulated in this way is said to be time consistent if the optimal strategies obtained when solving the original problem remain optimal for all subsequent problems. The notion of time consistency, well-known in the field of Economics, has been recently introduced in the context of risk measures, notably by Artzner et al. (2007) and studied in the Stochastic Programming framework by Shapiro (2009) and for Markov Decision Processes (MDP) by Ruszczynski (2009). We here link this notion with the concept of "state variable" in MDP, and show that a significant class of dynamic optimization problems are dynamically consistent, provided that an adequate state variable is chosen.

222p. ; 25cm

The present paper continues the study of infinite dimensional calculus via regularization, started by C. Di Girolami and the second named author, introducing the notion of "weak Dirichlet process" in this context. Such a process $\X$, taking values in a Hilbert space $H$, is the sum of a local martingale and a suitable "orthogonal" process. The new concept is shown to be useful in several contexts and directions. On one side, the mentioned decomposition appears to be a substitute of an It\^o type formula applied to $f(t, \X(t))$ where $f:[0,T] \times H \rightarrow \R$ is a $C^{0,1}$ function and, on the other side, the idea of weak Dirichlet process fits the widely used notion of "mild solution" for stochastic PDE. As a specific application, we provide a verification theorem for stochastic optimal control problems whose state equation is an infinite dimensional stochastic evolution equation.

The finite state semi-Markov process is a generalization over the Markov chain in which the sojourn time distribution is any general distribution. In this article we provide a sufficient stochastic maximum principle for the optimal control of a semi-Markov modulated jump-diffusion process in which the drift, diffusion and the jump kernel of the jump-diffusion process is modulated by a semi-Markov process. We also connect the sufficient stochastic maximum principle with the dynamic programming equation. We apply our results to finite horizon risk-sensitive control portfolio optimization problem and to a quadratic loss minimization problem.

We study the problem of optimal inside control of an SPDE (a stochastic evolution equation) driven by a Brownian motion and a Poisson random measure. Our optimal control problem is new in two ways: (i) The controller has access to inside information, i.e. access to information about a future state of the system, (ii) The integro-differential operator of the SPDE might depend on the control. In the first part of the paper, we formulate a sufficient and a necessary maximum principle for this type of control problem, in two cases: (1) When the control is allowed to depend both on time t and on the space variable x. (2) When the control is not allowed to depend on x. In the second part of the paper, we apply the results above to the problem of optimal control of an SDE system when the inside controller has only noisy observations of the state of the system. Using results from nonlinear filtering, we transform this noisy observation SDE inside control problem into a full observation SPDE insider control problem. The results are illustrated by explicit examples.

We consider a stochastic optimal control problem governed by a stochastic differential equation with delay in the control. Using a result of existence and uniqueness of a sufficiently regular mild solution of the associated Hamilton-Jacobi-Bellman (HJB) equation, see the companion paper "Stochastic Optimal Control with Delay in the Control I: solving the HJB equation through partial smoothing ", we solve the control problem by proving a Verification Theorem and the existence of optimal feedback controls.

We study quadratic optimal stochastic control problems with control dependent noise state equation perturbed by an affine term and with stochastic coefficients. Both infinite horizon case and ergodic case are treated. To this purpose we introduce a Backward Stochastic Riccati Equation and a dual backward stochastic equation, both considered in the whole time line. Besides some stabilizability conditions we prove existence of a solution for the two previous equations defined as limit of suitable finite horizon approximating problems. This allows to perform the synthesis of the optimal control.

In this paper we investigate the optimal control problem for a class of stochastic Cauchy evolution problem with non standard boundary dynamic and control. The model is composed by an infinite dimensional dynamical system coupled with a finite dimensional dynamics, which describes the boundary conditions of the internal system. In other terms, we are concerned with non standard boundary conditions, as the value at the boundary is governed by a different stochastic differential equation.

This paper investigates the near-optimal control for a kind of linear stochastic control systems governed by the forward-backward stochastic differential equations, where both the drift and diffusion terms are allowed to depend on controls and the control domain is non-convex. In the previous works of the second and third authors (see [Automatica 46 (2010) 397-404]), some open problem of near optimal control with the control-dependent diffusion is addressed and our current paper can be viewed as some direct response to it. The necessary and sufficient conditions of the near-optimality are established within the framework of optimality variational principle developed by Yong [SIAM J. Control Optim. 48 (2010) 4119--4156] and obtained by the convergence technique to treat the optimal control of FBSDEs in unbounded control domains by Wu [Automatica 49 (2013) 1473--1480]. Some new estimates are given here to handle the near optimality. In addition, two illustrating examples are discussed as well.

A Linear-quadratic optimal control problem is considered for mean-field stochastic differential equations with deterministic coefficients. By a variational method, the optimality system is derived, which turns out to be a linear mean-field forward-backward stochastic differential equation. Using a decoupling technique, two Riccati differential equations are obtained, which are uniquely solvable under certain conditions. Then a feedback representation is obtained for the optimal control.

A Linear-quadratic optimal control problem is considered for mean-field stochastic differential equations with deterministic coefficients. By a variational method, the optimality system is derived, which turns out to be a linear mean-field forward-backward stochastic differential equation. Using a decoupling technique, two Riccati differential equations are obtained, which are uniquely solvable under certain conditions. Then a feedback representation is obtained for the optimal control.

We take a new look at the relation between the optimal transport problem and the Schr\"{o}dinger bridge problem from the stochastic control perspective. We show that the connections are richer and deeper than described in existing literature. In particular: a) We give an elementary derivation of the Benamou-Brenier fluid dynamics version of the optimal transport problem; b) We provide a new fluid dynamics version of the Schr\"{o}dinger bridge problem; c) We observe that the latter provides an important connection with optimal transport without zero noise limits; d) We propose and solve a fluid dynamic version of optimal transport with prior; e) We can then view optimal transport with prior as the zero noise limit of Schr\"{o}dinger bridges when the prior is any Markovian evolution. In particular, we work out the Gaussian case. A numerical example of the latter convergence involving Brownian particles is also provided.

An asymptotic framework for optimal control of multiclass stochastic processing networks, using formal diffusion approximations under suitable temporal and spatial scaling, by Brownian control problems (BCP) and their equivalent workload formulations (EWF), has been developed by Harrison (1988). This framework has been implemented in many works for constructing asymptotically optimal control policies for a broad range of stochastic network models. To date all asymptotic optimality results for such networks correspond to settings where the solution of the EWF is a reflected Brownian motion in the positive orthant with normal reflections. In this work we consider a well studied stochastic network which is perhaps the simplest example of a model with more than one dimensional workload process. In the regime considered here, the singular control problem corresponding to the EWF does not have a simple form explicit solution, however by considering an associated free boundary problem one can give a representation for an optimal controlled process as a two dimensional reflected Brownian motion in a Lipschitz domain whose boundary is determined by the solution of the free boundary problem. Using the form of the optimal solution we propose a sequence of control policies, given in terms of suitable thresholds, for the scaled stochastic network control problems and prove that this sequence of policies is asymptotically optimal. As suggested by the solution of the EWF, the policy we propose requires a server to idle under certain conditions which are specified in terms of the thresholds determined from the free boundary.

The optimal fire distribution policy obtained using a stochastic combat attrition model is compared with that for a deterministic one. The same optimal control problem for a homogeneous force in combat against a heterogeneous force of two homogeneous types is considered using two different models for the attrition mechanism in a fight-to-the-finish the Lanchester-type differential equation formulation and its analagous stochastic version of a continuous parameter Markov chain with stationary transition probabilities. Considering dynamic programming methodology, a computer program was developed to numerically determine the optimal fire distribution policy (closed-loop or feedback) for the stochastic attrition process. Numerical values are generated for several parameter sets and compared with the optimal fire distribution policy for the corresponding deterministic attrition process. Results indicate that the optimal fire distribution policy for the stochastic model is more complex than the deterministic one.

The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions $\big\{{\bf P}_{B_i|B^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$ and $\big\{ {\bf P}_{B_i|B_{i-M}^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$, where $M$ is the memory of the channel, $B^n {\stackrel{\triangle}{=}} \{B_j: j=\ldots, 0,1, \ldots, n\}$ are the channel outputs and $A^n{\stackrel{\triangle}{=}} \{A_j: j=\ldots, 0,1, \ldots, n\}$ are the channel inputs. The characterizations of FTFI capacity, are obtained by first identifying the information structures of the optimal channel input conditional distributions ${\cal P}_{[0,n]} {\stackrel{\triangle}{=}} \big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}: i=0, \ldots, n\big\}$, which maximize directed information. The main theorem states, for any channel with memory $M$, the optimal channel input conditional distributions occur in the subset satisfying conditional independence $\stackrel{\circ}{\cal P}_{[0,n]}{\stackrel{\triangle}{=}} \big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}= {\bf P}_{A_i|B_{i-M}^{i-1}}: i=1, \ldots, n\big\}$, and the characterization of FTFI capacity is given by $C_{A^n \rightarrow B^n}^{FB, M} {\stackrel{\triangle}{=}} \sup_{ \stackrel{\circ}{\cal P}_{[0,n]} } \sum_{i=0}^n I(A_i; B_i|B_{i-M}^{i-1}) $. The methodology utilizes stochastic optimal control theory and a variational equality of directed information, to derive upper bounds on $I(A^n \rightarrow B^n)$, which are achievable over specific subsets of channel input conditional distributions ${\cal P}_{[0,n]}$, which are characterized by conditional independence. For any of the above classes of channel distributions and transmission cost functions, a direct analogy, in terms of conditional independence, of the characterizations of FTFI capacity and Shannon's capacity formulae of Memoryless Channels is identified.

In this paper, we study a time-inconsistent stochastic optimal control problem with a recursive cost functional by a multi-person hierarchical differential game approach. An equilibrium strategy of this problem is constructed and a corresponding equilibrium Hamilton-Jacobi-Bellman (HJB, for short) equation is established to characterize the associated equilibrium value function. Moreover, a well-posedness result of the equilibrium HJB equation is established under certain conditions.

In this research we continue our investigations of approximation techniques for a wide class of discrete and continuous time stochastic control problems. Emphasis is placed on the development and theoretical justification of techniques which yield computationally tractable algorithms that answer the following: (1) approximations to the optimal cost and the cost of using particular control; (2) approximations to the optimal control; (3) evaluation of the relative performance of two controls; and (4) estimates for the deterioration in system performance due to the failure to observe system components. (Author)

Time-consistency is an essential requirement in risk sensitive optimal control problems to make rational decisions. An optimization problem is time consistent if its solution policy does not depend on the time sequence of solving the optimization problem. On the other hand, a dynamic risk measure is time consistent if a certain outcome is considered less risky in the future implies this outcome is also less risky at current stage. In this paper, we study time-consistency of risk constrained problem where the risk metric is time consistent. From the Bellman optimality condition in [1], we establish an analytical "risk-to-go" that results in a time consistent optimal policy. Finally we demonstrate the effectiveness of the analytical solution by solving Haviv's counter-example [2] in time inconsistent planning.

In the G-framework, we establish existence of an optimal stochastic relaxed control for stochastic differential equations driven by a G-Brownian motion.

We study a constrained stochastic control problem with jumps; the jump times of the controlled process are given by a Poisson process. The cost functional comprises quadratic components for an absolutely continuous control and the controlled process and an absolute value component for the control of the jump size of the process. We characterize the value function by a "polynomial" of degree two whose coefficients depend on the state of the system; these coefficients are given by a coupled system of ODEs. The problem hence reduces from solving the Hamilton Jacobi Bellman (HJB) equation (i.e., a PDE) to solving an ODE whose solution is available in closed form. The state space is separated by a time dependent boundary into a continuation region where the optimal jump size of the controlled process is positive and a stopping region where it is zero. We apply the optimization problem to a problem faced by investors in the financial market who have to liquidate a position in a risky asset and have access to a dark pool with adverse selection.

In this paper we study, by probabilistic techniques, the convergence of the value function for a two-scale, infinite-dimensional, stochastic controlled system as the ratio between the two evolution speeds diverges. The value function is represented as the solution of a backward stochastic differential equation (BSDE) that it is shown to converge towards a reduced BSDE. The noise is assumed to be additive both in the slow and the fast equations for the state. Some non degeneracy condition on the slow equation are required. The limit BSDE involves the solution of an ergodic BSDE and is itself interpreted as the value function of an auxiliary stochastic control problem on a reduced state space.

We address a class of backward stochastic differential equations on a bounded interval, where the driving noise is a marked, or multivariate, point process. Assuming that the jump times are totally inaccessible and a technical condition holds (see Assumption (A) below), we prove existence and uniqueness results under Lipschitz conditions on the coefficients. Some counter-examples show that our assumptions are indeed needed. We use a novel approach that allows reduction to a (finite or infinite) system of deterministic differential equations, thus avoiding the use of martingale representation theorems and allowing potential use of standard numerical methods. Finally, we apply the main results to solve an optimal control problem for a marked point process, formulated in a classical way.

We consider a control problem where the system is driven by a decoupled as well as a coupled forward-backward stochastic differential equation. We prove the existence of an optimal control in the class of relaxed controls, which are measure-valued processes, generalizing the usual strict controls. The proof is based on some tightness properties and weak convergence on the space D of c\`adl\`ag functions, endowed with the Jakubowsky S-topology. Moreover, under some convexity assumptions, we show that the relaxed optimal control is realized by a strict control.

Title from cover

This paper is concerned with one kind of stochastic recursive optimal control problem with time delay, where the controlled system is described by a stochastic differential delayed equation (SDDE). The cost functional is of the recursive utility' type, which is formulated as the solution to a backward stochastic differential delayed equation (BSDDE). Two approaches to study such problem are Bellman's dynamic programming and Pontryagin's maximum principle. When only the pointwise and distributed time delays in the state variable, and only distributed time delay in the generator of BSDDE, a generalized Hamilton-Jacobi-Bellman equation in finite dimensional space is obtained applying the dynamic programming approach and a generalized It\^{o}'s formula. Sufficient and necessary maximum principles are both derived, where the adjoint equation is a forward-backward stochastic differential delayed equation (FBSDDE). Moreover, the relationship between these two approaches are investigated. Under some differentiability assumptions, connections among the value function, the adjoint processes and the generalized Hamiltonian function are investigated. A consumption and portfolio optimization problem with recursive utility and bounded memory in the financial market, is discussed to show the applications of our result. Explicit solutions in a finite dimensional space derived by the two different approaches, coincide.

In this paper, we study one kind of stochastic recursive optimal control problem with the obstacle constraints for the cost function where the cost function is described by the solution of one reflected backward stochastic differential equations. We will give the dynamic programming principle for this kind of optimal control problem and show that the value function is the unique viscosity solution of the obstacle problem for the corresponding Hamilton-Jacobi-Bellman equations.

We apply the stochastic Perron method of Bayraktar and S\^irbu to a general infinite horizon optimal control problem, where the state $X$ is a controlled diffusion process, and the state constraint is described by a closed set. We prove that the value function $v$ is bounded from below (resp., from above) by a viscosity supersolution (resp., subsolution) of the related state constrained problem for the Hamilton-Jacobi-Bellman equation. In the case of a smooth domain, under some additional assumptions, these estimates allow to identify $v$ with a unique continuous constrained viscosity solution of this equation.

This paper deals with a nonsmooth version of the connection between the maximum principle and dynamic programming principle, for the stochastic recursive control problem when the control domain is convex. By employing the notions of sub- and super-jets, the set inclusions are derived among the value function and the adjoint processes. The general case for non-convex control domain is open.

This paper deals with a nonsmooth version of the connection between the maximum principle and dynamic programming principle, for the stochastic recursive control problem when the control domain is convex. By employing the notions of sub- and super-jets, the set inclusions are derived among the value function and the adjoint processes. The general case for non-convex control domain is open.

This paper is devoted to the analysis of a finite horizon discrete-time stochastic optimal control problem, in presence of constraints. We study the regularity of the value function which comes from the dynamic programming algorithm. We derive accurate estimates of the Lipschitz constant of the value function, by means of a regularity result of the multifunction that defines the admissible control set. In the last section we discuss an application to an optimal asset-allocation problem.

Here is investigated the bilinear optimal control problem of quantum mechanical systems with final observation governed by a stochastic nonlinear Schr\"odinger equation perturbed by a linear multiplicative Wiener process. The existence of an open loop optimal control and first order Lagrange optimality conditions are derived, via Skorohod's representation theorem, Ekeland's variational principle and the existence for the linearized dual backward stochastic equation. Moreover, our approach in particular applies to the deterministic case.

Equivalences are known between problems of singular stochastic control (SSC) with convex performance criteria and related questions of optimal stopping, see for example Karatzas and Shreve [SIAM J. Control Optim. 22 (1984)]. The aim of this paper is to investigate how far connections of this type generalise to a non convex problem of purchasing electricity. Where the classical equivalence breaks down we provide alternative connections to optimal stopping problems. We consider a non convex infinite time horizon SSC problem whose state consists of an uncontrolled diffusion representing a real-valued commodity price, and a controlled increasing bounded process representing an inventory. We analyse the geometry of the action and inaction regions by characterising their (optimal) boundaries. Unlike the case of convex SSC problems we find that the optimal boundaries may be both reflecting and repelling and it is natural to interpret the problem as one of SSC with discretionary stopping.

This paper deals with a stochastic recursive optimal control problem, where the diffusion coefficient depends on the control variable and the control domain is not necessarily convex. We focus on the connection between the general maximum principle and the dynamic programming principle for such control problem without the assumption that the value is smooth enough, the set inclusions among the sub- and super-jets of the value function and the first-order and second-order adjoint processes as well as the generalized Hamiltonian function are established. Moreover, by comparing these results with the classical ones in Yong and Zhou [{\em Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999}], it is natural to obtain the first- and second-order adjoint equations of Hu [{\em Direct method on stochastic maximum principle for optimization with recursive utilities, arXiv:1507.03567v1 [math.OC], 13 Jul. 2015}].

Stochastic optimal control problems governed by delay equations with delay in the control are usually more difficult to study than the the ones when the delay appears only in the state. This is particularly true when we look at the associated Hamilton-Jacobi-Bellman (HJB) equation. Indeed, even in the simplified setting (introduced first by Vinter and Kwong for the deterministic case) the HJB equation is an infinite dimensional second order semilinear Partial Differential Equation (PDE) that does not satisfy the so-called "structure condition" which substantially means that "the noise enters the system with the control." The absence of such condition, together with the lack of smoothing properties which is a common feature of problems with delay, prevents the use of the known techniques (based on Backward Stochastic Differential Equations (BSDEs) or on the smoothing properties of the linear part) to prove the existence of regular solutions of this HJB equation and so no results on this direction have been proved till now. In this paper we provide a result on existence of regular solutions of such kind of HJB equations and we use it to solve completely the corresponding control problem finding optimal feedback controls also in the more difficult case of pointwise delay. The main tool used is a partial smoothing property that we prove for the transition semigroup associated to the uncontrolled problem. Such results holds for a specific class of equations and data which arises naturally in many applied problems.

We consider a class of optimal control problems of stochastic delay differential equations (SDDE) that arise in connection with optimal advertising under uncertainty for the introduction of a new product to the market, generalizing classical work of Nerlove and Arrow (1962). In particular, we deal with controlled SDDE where the delay enters both the state and the control. Following ideas of Vinter and Kwong (1981) (which however hold only in the deterministic case), we reformulate the problem as an infinite dimensional stochastic control problem to which we associate, through the dynamic programming principle, a second order Hamilton-Jacobi-Bellman equation. We show a verification theorem and we exhibit some simple cases where such equation admits an explicit smooth solution, allowing us to construct optimal feedback controls.

A new control design methodology is developed: Stochastic Optimal Feedforward and Feedback Technology (SOFFT). Traditional design techniques optimize a single cost function (which expresses the design objectives) to obtain both the feedforward and feedback control laws. This approach places conflicting demands on the control law such as fast tracking versus noise atttenuation/disturbance rejection. In the SOFFT approach, two cost functions are defined. The feedforward control law is designed to optimize one cost function, the feedback optimizes the other. By separating the design objectives and decoupling the feedforward and feedback design processes, both objectives can be achieved fully. A new measure of command tracking performance, Z-plots, is also developed. By analyzing these plots at off-nominal conditions, the sensitivity or robustness of the system in tracking commands can be predicted. Z-plots provide an important tool for designing robust control systems. The Variable-Gain SOFFT methodology was used to design a flight control system for the F/A-18 aircraft. It is shown that SOFFT can be used to expand the operating regime and provide greater performance (flying/handling qualities) throughout the extended flight regime. This work was performed under the NASA SBIR program. ICS plans to market the software developed as a new module in its commercial CACSD software package: ACET.

In this research we continue our investigations of approximation techniques for a wide class of discrete and continuous time stochastic control problems. Emphasis is placed on the development and theoretical justification of techniques which yield computationally tractable algorithms that answer the following: (1) approximations to the optimal cost and the cost of using particular control; (2) approximations to the optimal control; (3) evaluation of the relative performance of two controls; and (4) estimates for the deterioration in system performance due to the failure to observe system components. (Author)

This thesis presents new systems analysis methods that are appropriate for complex, nonlinear systems that are driven by uncertain inputs. These methods extend the ability of discrete dynamic programming (DDP) to system models that include six or more state variables and a similar number of stochastic variables. This is accomplished by interpolation and quadrature methods that have high-order accuracy and that provide significant computational savings over traditional DDP interpolation and quadrature methods. These new methods significantly improve our ability to apply DDP to large-scale systems. Using these methods, DDP can solve a variety of systems analysis problems without resorting to the simplifying assumptions required by other stochastic optimization methods. This is demonstrated in the application of DDP to problems with as many as seven state variables. Of particular interest, this thesis applied DDP to the practical problem of conjunctively managing groundwater and surface water. Moreover, the applications also demonstrate that DDP can be a powerfill planning tool, such as when evaluating a range of capacity expansion alternatives.

This document proposes an approach for the control of a general class of discrete-time nonlinear stochastic systems. The system model incorporates a deterministic linear portion together with a nonlinear function of the state and/or control vectors in combination with a white noise vector, where no Gaussian assumption is made. Under certain conditions imposed on the statistics of the additive nonlinear stochastic term, and assuming perfect state information, the optimal control, which minimizes a quadratic performance index subject to the nonlinear control system constraint, is shown to be a linear function of the state vector. This work also shows that for certain infinite horizon problems, an uncertainty threshold can be found such that the designer can, a priori, put an upper bound on the allowable noise covariance to obtain a bounded optimal constant feedback control.