Downloads & Free Reading Options - Results

We are concerned with the linear-quadratic optimal stochastic control problem with random coefficients. Under suitable conditions, we prove that the value field $V(t,x,\omega), (t,x,\omega)\in [0,T]\times R^n\times \Omega$, is quadratic in $x$, and has the following form: $V(t,x)=\langle K_tx, x\rangle$ where $K$ is an essentially bounded nonnegative symmetric matrix-valued adapted processes. Using the dynamic programming principle (DPP), we prove that $K$ is a continuous semi-martingale of the form $$K_t=K_0+\int_0^t \, dk_s+\sum_{i=1}^d\int_0^tL_s^i\, dW_s^i, \quad t\in [0,T]$$ with $k$ being a continuous process of bounded variation and $$E\left[\left(\int_0^T|L_s|^2\, ds\right)^p\right]

In this paper, we show how a simulated Markov decision process (MDP) built by the so-called \emph{baseline} policies, can be used to compute a different policy, namely the \emph{simulated optimal} policy, for which the performance of this policy is guaranteed to be better than the baseline policy in the real environment. This technique has immense applications in fields such as news recommendation systems, health care diagnosis and digital online marketing. Our proposed algorithm iteratively solves for a "good" policy in the simulated MDP in an offline setting. Furthermore, we provide a performance bound on sub-optimality for the control policy generated by the proposed algorithm.

In this paper, the optimal control problem of neutral stochastic functional differential equation (NSFDE) is discussed. A class of so-called neutral backward stochastic functional equations of Volterra type (VNBSFEs) are introduced as the adjoint equation. The existence and uniqueness of VNBSFE is established. The Pontryagin maximum principle is constructed for controlled NSFDE with Lagrange type cost functional.

We consider a problem of optimal control of an infinite horizon system governed by forward-backward stochastic differential equations with delay. Sufficient and necessary maximum principles for optimal control under partial information in infinite horizon are derived. We illustrate our results by an application to a problem of optimal consumption with respect to recursive utility from a cash flow with delay.

This paper is concerned with a stochastic linear quadratic (LQ, for short) optimal control problem. The notions of open-loop and closed-loop solvabilities are introduced. A simple example shows that these two solvabilities are different. Closed-loop solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. Conditions ensuring the convexity of the cost functional are discussed, including the issue that how negative the control weighting matrix-valued function R(s) can be. Finiteness of the LQ problem is characterized by the convergence of the solutions to a family of Riccati equations. Then, a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. Finally, an illustrative example is presented.

In this paper, we present an Uzawa-based heuristic that is adapted to some type of stochastic optimal control problems. More precisely, we consider dynamical systems that can be divided into small-scale independent subsystems, though linked through a static almost sure coupling constraint at each time step. This type of problem is common in production/portfolio management where subsystems are, for instance, power units, and one has to supply a stochastic power demand at each time step. We outline the framework of our approach and present promising numerical results on a simplified power management problem.

In this paper, we study one kind of stochastic recursive optimal control problem with the obstacle constraints for the cost function where the cost function is described by the solution of one reflected backward stochastic differential equations. We will give the dynamic programming principle for this kind of optimal control problem and show that the value function is the unique viscosity solution of the obstacle problem for the corresponding Hamilton-Jacobi-Bellman equations.

Optimal control of nonlinear saturating systems with stochastic inputs

We apply the stochastic Perron method of Bayraktar and S\^irbu to a general infinite horizon optimal control problem, where the state $X$ is a controlled diffusion process, and the state constraint is described by a closed set. We prove that the value function $v$ is bounded from below (resp., from above) by a viscosity supersolution (resp., subsolution) of the related state constrained problem for the Hamilton-Jacobi-Bellman equation. In the case of a smooth domain, under some additional assumptions, these estimates allow to identify $v$ with a unique continuous constrained viscosity solution of this equation.

This paper is devoted to the analysis of a finite horizon discrete-time stochastic optimal control problem, in presence of constraints. We study the regularity of the value function which comes from the dynamic programming algorithm. We derive accurate estimates of the Lipschitz constant of the value function, by means of a regularity result of the multifunction that defines the admissible control set. In the last section we discuss an application to an optimal asset-allocation problem.

This paper is devoted to the analysis of a finite horizon discrete-time stochastic optimal control problem, in presence of constraints. We study the regularity of the value function which comes from the dynamic programming algorithm. We derive accurate estimates of the Lipschitz constant of the value function, by means of a regularity result of the multifunction that defines the admissible control set. In the last section we discuss an application to an optimal asset-allocation problem.

This paper deals with a nonsmooth version of the connection between the maximum principle and dynamic programming principle, for the stochastic recursive control problem when the control domain is convex. By employing the notions of sub- and super-jets, the set inclusions are derived among the value function and the adjoint processes. The general case for non-convex control domain is open.

Here is investigated the bilinear optimal control problem of quantum mechanical systems with final observation governed by a stochastic nonlinear Schr\"odinger equation perturbed by a linear multiplicative Wiener process. The existence of an open loop optimal control and first order Lagrange optimality conditions are derived, via Skorohod's representation theorem, Ekeland's variational principle and the existence for the linearized dual backward stochastic equation. Moreover, our approach in particular applies to the deterministic case.

This paper first presents necessary and sufficient conditions for the solvability of discrete time, mean-field, stochastic linear-quadratic optimal control problems. Then, by introducing several sequences of bounded linear operators, the problem becomes an operator stochastic LQ problem, in which the optimal control is a linear state feedback. Furthermore, from the form of the optimal control, the problem changes to a matrix dynamic optimization problem. Solving this optimization problem, we obtain the optimal feedback gain and thus the optimal control. Finally, by completing the square, the optimality of the above control is validated.

Optimal control problems of forward-backward stochastic Volterra integral equations (FBSVIEs, in short) with closed control regions are formulated and studied. Instead of using spike variation method as one may imagine, here we turn to treat the non-convexity of the control regions by borrowing some tools in set-valued analysis and adapting them into our stochastic control systems. A duality principle between linear backward stochastic Volterra integral equations and linear stochastic Fredholm-Volterra integral equations with conditional expectation are derived, which extends and improves the corresponding results in [25], [30]. Some first order necessary optimality conditions for optimal controls of FBSVIEs are established. In contrast with existed common routines to treat the non-convexity of stochastic control problems, here only one adjoint system and one-order differentiability requirements of the coefficients are needed.

Linear-quadratic optimal control problems are considered for mean-field stochastic differential equations with deterministic coefficients. Time-inconsistency feature of the problems is carefully investigated. Both open-loop and closed-loop equilibrium solutions are presented for such kind of problems. Open-loop solutions are presented by means of variational method with decoupling of forward-backward stochastic differential equations, which lead to a Riccati equation system lack of symmetry. Closed-loop solutions are presented by means of multi-person differential games, the limit of which leads to a Riccati equation system with a symmetric structure.

We study quadratic optimal stochastic control problems with control dependent noise state equation perturbed by an affine term and with stochastic coefficients. Both infinite horizon case and ergodic case are treated. To this purpose we introduce a Backward Stochastic Riccati Equation and a dual backward stochastic equation, both considered in the whole time line. Besides some stabilizability conditions we prove existence of a solution for the two previous equations defined as limit of suitable finite horizon approximating problems. This allows to perform the synthesis of the optimal control.

In this paper we investigate the optimal control problem for a class of stochastic Cauchy evolution problem with non standard boundary dynamic and control. The model is composed by an infinite dimensional dynamical system coupled with a finite dimensional dynamics, which describes the boundary conditions of the internal system. In other terms, we are concerned with non standard boundary conditions, as the value at the boundary is governed by a different stochastic differential equation.

This paper investigates the near-optimal control for a kind of linear stochastic control systems governed by the forward-backward stochastic differential equations, where both the drift and diffusion terms are allowed to depend on controls and the control domain is non-convex. In the previous works of the second and third authors (see [Automatica 46 (2010) 397-404]), some open problem of near optimal control with the control-dependent diffusion is addressed and our current paper can be viewed as some direct response to it. The necessary and sufficient conditions of the near-optimality are established within the framework of optimality variational principle developed by Yong [SIAM J. Control Optim. 48 (2010) 4119--4156] and obtained by the convergence technique to treat the optimal control of FBSDEs in unbounded control domains by Wu [Automatica 49 (2013) 1473--1480]. Some new estimates are given here to handle the near optimality. In addition, two illustrating examples are discussed as well.

A Linear-quadratic optimal control problem is considered for mean-field stochastic differential equations with deterministic coefficients. By a variational method, the optimality system is derived, which turns out to be a linear mean-field forward-backward stochastic differential equation. Using a decoupling technique, two Riccati differential equations are obtained, which are uniquely solvable under certain conditions. Then a feedback representation is obtained for the optimal control.

A Linear-quadratic optimal control problem is considered for mean-field stochastic differential equations with deterministic coefficients. By a variational method, the optimality system is derived, which turns out to be a linear mean-field forward-backward stochastic differential equation. Using a decoupling technique, two Riccati differential equations are obtained, which are uniquely solvable under certain conditions. Then a feedback representation is obtained for the optimal control.

We take a new look at the relation between the optimal transport problem and the Schr\"{o}dinger bridge problem from the stochastic control perspective. We show that the connections are richer and deeper than described in existing literature. In particular: a) We give an elementary derivation of the Benamou-Brenier fluid dynamics version of the optimal transport problem; b) We provide a new fluid dynamics version of the Schr\"{o}dinger bridge problem; c) We observe that the latter provides an important connection with optimal transport without zero noise limits; d) We propose and solve a fluid dynamic version of optimal transport with prior; e) We can then view optimal transport with prior as the zero noise limit of Schr\"{o}dinger bridges when the prior is any Markovian evolution. In particular, we work out the Gaussian case. A numerical example of the latter convergence involving Brownian particles is also provided.

An asymptotic framework for optimal control of multiclass stochastic processing networks, using formal diffusion approximations under suitable temporal and spatial scaling, by Brownian control problems (BCP) and their equivalent workload formulations (EWF), has been developed by Harrison (1988). This framework has been implemented in many works for constructing asymptotically optimal control policies for a broad range of stochastic network models. To date all asymptotic optimality results for such networks correspond to settings where the solution of the EWF is a reflected Brownian motion in the positive orthant with normal reflections. In this work we consider a well studied stochastic network which is perhaps the simplest example of a model with more than one dimensional workload process. In the regime considered here, the singular control problem corresponding to the EWF does not have a simple form explicit solution, however by considering an associated free boundary problem one can give a representation for an optimal controlled process as a two dimensional reflected Brownian motion in a Lipschitz domain whose boundary is determined by the solution of the free boundary problem. Using the form of the optimal solution we propose a sequence of control policies, given in terms of suitable thresholds, for the scaled stochastic network control problems and prove that this sequence of policies is asymptotically optimal. As suggested by the solution of the EWF, the policy we propose requires a server to idle under certain conditions which are specified in terms of the thresholds determined from the free boundary.

In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning, we propose a novel algorithm called the incremental Markov Decision Process (iMDP) to compute incrementally control policies that approximate arbitrarily well an optimal policy in terms of the expected cost. The main idea behind the algorithm is to generate a sequence of finite discretizations of the original problem through random sampling of the state space. At each iteration, the discretized problem is a Markov Decision Process that serves as an incrementally refined model of the original problem. We show that with probability one, (i) the sequence of the optimal value functions for each of the discretized problems converges uniformly to the optimal value function of the original stochastic optimal control problem, and (ii) the original optimal value function can be computed efficiently in an incremental manner using asynchronous value iterations. Thus, the proposed algorithm provides an anytime approach to the computation of optimal control policies of the continuous problem. The effectiveness of the proposed approach is demonstrated on motion planning and control problems in cluttered environments in the presence of process noise.

We consider a controlled state equation of parabolic type on the halfline $(0,+\infty)$ with boundary conditions of Dirichlet type in which the unknown is equal to the sum of the control and of a white noise in time. We study finite horizon and infinite horizon optimal control problem related by menas of backward stochastic differential equations.

We consider a stochastic control model driven by a fractional Brownian motion. This model is a formal approximation to a queueing network with an on-off input process. We study stochastic control problems associated with the long-run average cost, the infinite horizon discounted cost, and the finite horizon cost. In addition, we find a solution to a constrained minimization problem as an application of our solution to the long-run average cost problem. We also establish Abelian limit relationships among the value functions of the above control problems.

This paper presents a method to approximately solve stochastic optimal control problems in which the cost function and the system dynamics are polynomial. For stochastic systems with polynomial dynamics, the moments of the state can be expressed as a, possibly infinite, system of deterministic linear ordinary differential equations. By casting the problem as a deterministic control problem in moment space, semidefinite programming is used to find a lower bound on the optimal solution. The constraints in the semidefinite program are imposed by the ordinary differential equations for moment dynamics and semidefiniteness of the outer product of moments. From the solution to the semidefinite program, an approximate optimal control strategy can be constructed using a least squares method. In the linear quadratic case, the method gives an exact solution to the optimal control problem. In more complex problems, an infinite number of moment differential equations would be required to compute the optimal control law. In this case, we give a procedure to increase the size of the semidefinite program, leading to increasingly accurate approximations to the true optimal control strategy.

We consider the performance of periodically driven stochastic heat engines in the linear response regime. Reaching the theoretical bounds for efficiency and efficiency at maximum power typically requires full control over the design and the driving of the system. We develop a framework which allows to quantify the role that limited control over the system has on the performance. Specifically, we show that optimizing the driving entering the work extraction for a given temperature protocol leads to a universal, one-parameter dependence for both maximum efficiency and maximum power as a function of efficiency. In particular, we show that reaching Carnot efficiency (and, hence, Curzon-Ahlborn efficiency at maximum power) requires to have control over the amplitude of the full Hamiltonian of the system. Since the kinetic energy cannot be controlled by an external parameter, heat engines based on underdamped dynamics can typically not reach Carnot efficiency. We illustrate our general theory with a paradigmatic case study of a heat engine consisting of an underdamped charged particle in a modulated two-dimensional harmonic trap in the presence of a magnetic field.

In this paper we present a novel sampling-based numerical scheme designed to solve a certain class of stochastic optimal control problems, utilizing forward and backward stochastic differential equations (FBSDEs). By means of a nonlinear version of the Feynman-Kac lemma, we obtain a probabilistic representation of the solution to the nonlinear Hamilton-Jacobi-Bellman equation, expressed in the form of a decoupled system of FBSDEs. This system of FBSDEs can then be simulated by employing linear regression techniques. To enhance the efficiency of the proposed scheme when treating more complex nonlinear systems, we then derive an iterative modification based on Girsanov's theorem on the change of measure, which features importance sampling. The modified scheme is capable of learning the optimal control without requiring an initial guess. We present simulations that validate the algorithm and demonstrate its efficiency in treating nonlinear dynamics.

The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions $\big\{{\bf P}_{B_i|B^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$ and $\big\{ {\bf P}_{B_i|B_{i-M}^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$, where $M$ is the memory of the channel, $B^n {\stackrel{\triangle}{=}} \{B_j: j=\ldots, 0,1, \ldots, n\}$ are the channel outputs and $A^n{\stackrel{\triangle}{=}} \{A_j: j=\ldots, 0,1, \ldots, n\}$ are the channel inputs. The characterizations of FTFI capacity, are obtained by first identifying the information structures of the optimal channel input conditional distributions ${\cal P}_{[0,n]} {\stackrel{\triangle}{=}} \big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}: i=0, \ldots, n\big\}$, which maximize directed information. The main theorem states, for any channel with memory $M$, the optimal channel input conditional distributions occur in the subset satisfying conditional independence $\stackrel{\circ}{\cal P}_{[0,n]}{\stackrel{\triangle}{=}} \big\{ {\bf P}_{A_i|A^{i-1}, B^{i-1}}= {\bf P}_{A_i|B_{i-M}^{i-1}}: i=1, \ldots, n\big\}$, and the characterization of FTFI capacity is given by $C_{A^n \rightarrow B^n}^{FB, M} {\stackrel{\triangle}{=}} \sup_{ \stackrel{\circ}{\cal P}_{[0,n]} } \sum_{i=0}^n I(A_i; B_i|B_{i-M}^{i-1}) $. The methodology utilizes stochastic optimal control theory and a variational equality of directed information, to derive upper bounds on $I(A^n \rightarrow B^n)$, which are achievable over specific subsets of channel input conditional distributions ${\cal P}_{[0,n]}$, which are characterized by conditional independence. For any of the above classes of channel distributions and transmission cost functions, a direct analogy, in terms of conditional independence, of the characterizations of FTFI capacity and Shannon's capacity formulae of Memoryless Channels is identified.

The optimal fire distribution policy obtained using a stochastic combat attrition model is compared with that for a deterministic one. The same optimal control problem for a homogeneous force in combat against a heterogeneous force of two homogeneous types is considered using two different models for the attrition mechanism in a fight-to-the-finish the Lanchester-type differential equation formulation and its analagous stochastic version of a continuous parameter Markov chain with stationary transition probabilities. Considering dynamic programming methodology, a computer program was developed to numerically determine the optimal fire distribution policy (closed-loop or feedback) for the stochastic attrition process. Numerical values are generated for several parameter sets and compared with the optimal fire distribution policy for the corresponding deterministic attrition process. Results indicate that the optimal fire distribution policy for the stochastic model is more complex than the deterministic one.

We propose a novel reformulation of the stochastic optimal control problem as an approximate inference problem, demonstrating, that such a interpretation leads to new practical methods for the original problem. In particular we characterise a novel class of iterative solutions to the stochastic optimal control problem based on a natural relaxation of the exact dual formulation. These theoretical insights are applied to the Reinforcement Learning problem where they lead to new model free, off policy methods for discrete and continuous problems.

We propose a novel reformulation of the stochastic optimal control problem as an approximate inference problem, demonstrating, that such a interpretation leads to new practical methods for the original problem. In particular we characterise a novel class of iterative solutions to the stochastic optimal control problem based on a natural relaxation of the exact dual formulation. These theoretical insights are applied to the Reinforcement Learning problem where they lead to new model free, off policy methods for discrete and continuous problems.

We study an optimal relaxed control problem for a class of semilinear stochastic PDEs on Banach spaces perturbed by multiplicative noise and driven by a cylindrical Wiener process. The state equation is controlled through the nonlinear part of the drift coefficient which satisfies a dissipative-type condition with respect to the state variable. The main tools of our study are the factorization method for stochastic convolutions in UMD type-2 Banach spaces and certain compactness properties of the factorization operator and of the class of Young measures on Suslin metrisable control sets.

We consider a unifying framework for stochastic control problem including the following features: partial observation, path-dependence (both with respect to the state and the control), and without any non-degeneracy condition on the stochastic differential equation (SDE) for the controlled state process, driven by a Wiener process. In this context, we develop a general methodology, refereed to as the randomization method, studied in [23] for classical Markovian control under full observation, and consisting basically in replacing the control by an exogenous process independent of the driving noise of the SDE. Our first main result is to prove the equivalence between the primal control problem and the randomized control problem where optimization is performed over change of equivalent probability measures affecting the characteristics of the exogenous process. The randomized problem turns out to be associated by duality and separation argument to a backward SDE, which leads to the so-called randomized dynamic programming principle and randomized equation in terms of the path-dependent filter, and then characterizes the value function of the primal problem. In particular, classical optimal control problems with partial observation affected by non-degenerate Gaussian noise fall within the scope of our framework, and are treated by means of an associated backward SDE.

222p. ; 25cm

In this research we continue our investigations of approximation techniques for a wide class of discrete and continuous time stochastic control problems. Emphasis is placed on the development and theoretical justification of techniques which yield computationally tractable algorithms that answer the following: (1) approximations to the optimal cost and the cost of using particular control; (2) approximations to the optimal control; (3) evaluation of the relative performance of two controls; and (4) estimates for the deterioration in system performance due to the failure to observe system components. (Author)

An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closed-loop solvability of the problem) is characterized by the existence of a regular solution to the coupled two (generalized) Riccati equations, together with some constraints on the adapted solution to a linear backward stochastic differential equation and a linear terminal value problem of an ordinary differential equation.

We study the optimal control of general stochastic McKean-Vlasov equation. Such problem is motivated originally from the asymptotic formulation of cooperative equilibrium for a large population of particles (players) in mean-field interaction under common noise. Our first main result is to state a dynamic programming principle for the value function in the Wasserstein space of probability measures, which is proved from a flow property of the conditional law of the controlled state process. Next, by relying on the notion of differentiability with respect to probability measures due to P.L. Lions [32], and It{\^o}'s formula along a flow of conditional measures, we derive the dynamic programming Hamilton-Jacobi-Bellman equation, and prove the viscosity property together with a uniqueness result for the value function. Finally, we solve explicitly the linear-quadratic stochastic McKean-Vlasov control problem and give an application to an interbank systemic risk model with common noise.

In this paper, we study a stochastic recursive optimal control problem in which the cost functional is described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Under standard assumptions, we establish the dynamic programming principle and the related fully nonlinear HJB equation in the framework of G-expectation. Finally, we show that the value function is the viscosity solution of the obtained HJB equation.

We study a combined optimal control/stopping problem under a nonlinear expectation ${\cal E}^f$ induced by a BSDE with jumps, in a Markovian framework. The terminal reward function is only supposed to be Borelian. The value function $u$ associated with this problem is generally irregular. We first establish a {\em sub- (resp. super-) optimality principle of dynamic programming} involving its {\em upper- (resp. lower-) semicontinuous envelope} $u^*$ (resp. $u_*$). This result, called {\em weak} dynamic programming principle (DPP), extends that obtained in \cite{BT} in the case of a classical expectation to the case of an ${\cal E}^f$-expectation and Borelian terminal reward function. Using this {\em weak} DPP, we then prove that $u^*$ (resp. $u_*$) is a {\em viscosity sub- (resp. super-) solution} of a nonlinear Hamilton-Jacobi-Bellman variational inequality.

Time-consistency is an essential requirement in risk sensitive optimal control problems to make rational decisions. An optimization problem is time consistent if its solution policy does not depend on the time sequence of solving the optimization problem. On the other hand, a dynamic risk measure is time consistent if a certain outcome is considered less risky in the future implies this outcome is also less risky at current stage. In this paper, we study time-consistency of risk constrained problem where the risk metric is time consistent. From the Bellman optimality condition in [1], we establish an analytical "risk-to-go" that results in a time consistent optimal policy. Finally we demonstrate the effectiveness of the analytical solution by solving Haviv's counter-example [2] in time inconsistent planning.

In the G-framework, we establish existence of an optimal stochastic relaxed control for stochastic differential equations driven by a G-Brownian motion.

For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. This notion can be informally introduced as follows. At the very first time step $t_0$, the decision maker formulates an optimization problem that yields optimal decision rules for all the forthcoming time step $t_0, t_1, ..., T$; at the next time step $t_1$, he is able to formulate a new optimization problem starting at time $t_1$ that yields a new sequence of optimal decision rules. This process can be continued until final time $T$ is reached. A family of optimization problems formulated in this way is said to be time consistent if the optimal strategies obtained when solving the original problem remain optimal for all subsequent problems. The notion of time consistency, well-known in the field of Economics, has been recently introduced in the context of risk measures, notably by Artzner et al. (2007) and studied in the Stochastic Programming framework by Shapiro (2009) and for Markov Decision Processes (MDP) by Ruszczynski (2009). We here link this notion with the concept of "state variable" in MDP, and show that a significant class of dynamic optimization problems are dynamically consistent, provided that an adequate state variable is chosen.

We study the problem of optimal inside control of an SPDE (a stochastic evolution equation) driven by a Brownian motion and a Poisson random measure. Our optimal control problem is new in two ways: (i) The controller has access to inside information, i.e. access to information about a future state of the system, (ii) The integro-differential operator of the SPDE might depend on the control. In the first part of the paper, we formulate a sufficient and a necessary maximum principle for this type of control problem, in two cases: (1) When the control is allowed to depend both on time t and on the space variable x. (2) When the control is not allowed to depend on x. In the second part of the paper, we apply the results above to the problem of optimal control of an SDE system when the inside controller has only noisy observations of the state of the system. Using results from nonlinear filtering, we transform this noisy observation SDE inside control problem into a full observation SPDE insider control problem. The results are illustrated by explicit examples.

We consider a stochastic optimal control problem governed by a stochastic differential equation with delay in the control. Using a result of existence and uniqueness of a sufficiently regular mild solution of the associated Hamilton-Jacobi-Bellman (HJB) equation, see the companion paper "Stochastic Optimal Control with Delay in the Control I: solving the HJB equation through partial smoothing ", we solve the control problem by proving a Verification Theorem and the existence of optimal feedback controls.

The finite state semi-Markov process is a generalization over the Markov chain in which the sojourn time distribution is any general distribution. In this article we provide a sufficient stochastic maximum principle for the optimal control of a semi-Markov modulated jump-diffusion process in which the drift, diffusion and the jump kernel of the jump-diffusion process is modulated by a semi-Markov process. We also connect the sufficient stochastic maximum principle with the dynamic programming equation. We apply our results to finite horizon risk-sensitive control portfolio optimization problem and to a quadratic loss minimization problem.

The present paper continues the study of infinite dimensional calculus via regularization, started by C. Di Girolami and the second named author, introducing the notion of "weak Dirichlet process" in this context. Such a process $\X$, taking values in a Hilbert space $H$, is the sum of a local martingale and a suitable "orthogonal" process. The new concept is shown to be useful in several contexts and directions. On one side, the mentioned decomposition appears to be a substitute of an It\^o type formula applied to $f(t, \X(t))$ where $f:[0,T] \times H \rightarrow \R$ is a $C^{0,1}$ function and, on the other side, the idea of weak Dirichlet process fits the widely used notion of "mild solution" for stochastic PDE. As a specific application, we provide a verification theorem for stochastic optimal control problems whose state equation is an infinite dimensional stochastic evolution equation.

We study a constrained stochastic control problem with jumps; the jump times of the controlled process are given by a Poisson process. The cost functional comprises quadratic components for an absolutely continuous control and the controlled process and an absolute value component for the control of the jump size of the process. We characterize the value function by a "polynomial" of degree two whose coefficients depend on the state of the system; these coefficients are given by a coupled system of ODEs. The problem hence reduces from solving the Hamilton Jacobi Bellman (HJB) equation (i.e., a PDE) to solving an ODE whose solution is available in closed form. The state space is separated by a time dependent boundary into a continuation region where the optimal jump size of the controlled process is positive and a stopping region where it is zero. We apply the optimization problem to a problem faced by investors in the financial market who have to liquidate a position in a risky asset and have access to a dark pool with adverse selection.

We address a class of backward stochastic differential equations on a bounded interval, where the driving noise is a marked, or multivariate, point process. Assuming that the jump times are totally inaccessible and a technical condition holds (see Assumption (A) below), we prove existence and uniqueness results under Lipschitz conditions on the coefficients. Some counter-examples show that our assumptions are indeed needed. We use a novel approach that allows reduction to a (finite or infinite) system of deterministic differential equations, thus avoiding the use of martingale representation theorems and allowing potential use of standard numerical methods. Finally, we apply the main results to solve an optimal control problem for a marked point process, formulated in a classical way.