Downloads & Free Reading Options - Results

This paper obtains the maximum principle for both stochastic (global) open-loop and stochastic (global) closed-loop Stackelberg differential games. For the closed-loop case, we use the theory of controlled forward-backward stochastic differential equations to derive the maximum principle for the leader's optimal strategy. In the special case of the open-loop linear quadratic Stackelberg game, we consider the follower's Hamiltonian system as the leader's state equation, derive the related stochastic Riccati equation, and show the existence and uniqueness of the solution to the Riccati equation under appropriate assumptions. However, for the closed-loop linear quadratic Stackelberg game, we can write the related Riccati equation consisting of forward-backward stochastic differential equations, while leaving the existence of its solution as an open problem.

For a zero-sum stochastic game which does not satisfy the Isaacs condition, we provide a value function representation for an Isaacs-type equation whose Hamiltonian lies in between the lower and upper Hamiltonians, as a convex combination of the two. For the general case (i.e. the convex combination is time and state dependent) our representation amounts to a random change of the rules of the game, to allow each player at any moment to see the other player's action or not, according to a coin toss with probabilities of heads and tails given by the convex combination appearing in the PDE. If the combination is state independent, then the rules can be set all in advance, in a deterministic way. This means that tossing the coin along the game, or tossing it repeatedly right at the beginning leads to the same value. The representations are asymptotic, over time discretizations. Space discretization is possible as well, leading to similar results.

This paper proposes a systems approach to social sciences based on mathematical framework derived from a generalization of the mathematical kinetic theory and on theoretical tools of game theory. Social systems are modeled as a living evolutionary ensemble composed by many individuals, who express specific strategies, cooperate, compete and might aggregate into groups which pursue a common interest. A critical analysis on the complexity features of social system is developed and a differential structure is derived to provide a general framework toward modeling.

Mean field type models describing the limiting behavior of stochastic differential games as the number of players tends to +$\infty$, have been recently introduced by J-M. Lasry and P-L. Lions. Under suitable assumptions, they lead to a system of two coupled partial differential equations, a forward Bellman equation and a backward Fokker-Planck equations. Finite difference schemes for the approximation of such systems have been proposed in previous works. Here, we prove the convergence of these schemes towards a weak solution of the system of partial differential equations.

In this paper we propose a numerical method to obtain an approximation of Nash equilibria for multi-player non-cooperative games with a special structure. We consider the infinite horizon problem in a case which leads to a system of Hamilton-Jacobi equations. The numerical method is based on the Dynamic Programming Principle for every equation and on a global fixed point iteration. We present the numerical solutions of some two-player games in one and two dimensions. The paper has an experimental nature, but some features and properties of the approximation scheme are discussed.

A multiple integral functional is equivalent to a curvilinear integral functional, if the domain is a hyper-parallelepiped, but equivalence is only theoretical. The introduction of this kind of functionals in multitime optimal control problems, particularly in multitime differential games, is due to recent works of Udriste research group. The purpose of this paper is to introduce those ingredients that are necessary to formulate and to prove theorems about multitime differential games based on a multiple integral functional and an $m$-flow as constraint. The most important idea is to use a generating vector field for basic functions. The original results include: fundamental properties of multitime upper and lower values, viscosity solutions of multitime (dHJIU) PDEs, representation formula of viscosity solutions for a multitime (dHJ) PDE, and max-min representations.

In this paper we introduce the discontinuous universal feedback for the problem of Nash equilibrium in two person non-zero sum differential game. We assume that there exist functions satisfying some conditions analogous to the infinitesimal conditions on value function in zero sum differential games. Under this assumption we prove the existence of universal feedback Nash equilibrium.

In this paper we introduce the discontinuous universal feedback for the problem of Nash equilibrium in two person non-zero sum differential game. We assume that there exist functions satisfying some conditions analogous to the infinitesimal conditions on value function in zero sum differential games. Under this assumption we prove the existence of universal feedback Nash equilibrium.

In this paper we introduce the discontinuous universal feedback for the problem of Nash equilibrium in two person non-zero sum differential game. We assume that there exist functions satisfying some conditions analogous to the infinitesimal conditions on value function in zero sum differential games. Under this assumption we prove the existence of universal feedback Nash equilibrium.

In this paper we study zero-sum two-player stochastic differential games with the help of theory of Backward Stochastic Differential Equations (BSDEs). At the one hand we generalize the results of the pioneer work of Fleming and Souganidis by considering cost functionals defined by controlled BSDEs and by allowing the admissible control processes to depend on events occurring before the beginning of the game (which implies that the cost functionals become random variables), on the other hand the application of BSDE methods, in particular that of the notion of stochastic "backward semigroups" introduced by Peng allows to prove a dynamic programming principle for the upper and the lower value functions of the game in a straight-forward way, without passing by additional approximations. The upper and the lower value functions are proved to be the unique viscosity solutions of the upper and the lower Hamilton-Jacobi-Bellman-Isaacs equations, respectively. For this Peng's BSDE method is translated from the framework of stochastic control theory into that of stochastic differential games.

The problem is to control the drift of a Markov diffusion process in such a way that the probability that the process exits from a given region D during a given finite time interval is minimum. An asymptotic formula for the minimum exit probability when the process is nearly deterministic is given. This formula involves the lower value of an associated differential game. It is related to a result of Ventsel and Freidlin for nearly deterministic, uncontrolled diffusions. (Author)

In this paper, we present an optimal control problem for stochastic differential games under Markov regime-switching forward-backward stochastic differential equations with jumps and partial information. First, we prove a sufficient maximum principle for non zero-sum stochastic differential game problems and obtain equilibrium point for such games. Second, we prove an equivalent maximum principle for non zero-sum stochastic differential games. The zero-sum stochastic differential games equivalent maximum principle is then obtained as a corollary. We apply the obtained results to study a problem of robust utility maximization under penalty entropy. We also apply the result to find optimal investment of an insurance firm under model uncertainty.

We analyze a zero-sum stochastic differential game between two competing players who can choose unbounded controls. The payoffs of the game are defined through backward stochastic differential equations. We prove that each player's priority value satisfies a weak dynamic programming principle and thus solves the associated fully non-linear partial differential equation in the viscosity sense.

We investigate a two-player zero-sum stochastic differential game in which the players have an asymmetric information on the random payoff. We prove that the game has a value and characterize this value in terms of dual solutions of some second order Hamilton-Jacobi equation.

In this paper, we consider a linear quadratic stochastic two-person zero-sum differential game. The controls for both players are allowed to appear in both drift and diffusion of the state equation. The weighting matrices in the performance functional are not assumed to be definite/non-singular. A necessary and sufficient condition for the existence of a closed-loop saddle point is established in terms of the solvability of a Riccati differential equation with certain regularity. It is possible that the closed-loop saddle point fails to exist, and at the same time, the corresponding Riccati equation admits a solution (which does not have needed regularity). Also, we will indicate that the solution of the Riccati equation may be non-unique.

We show a method of uniform approximation of the value functions of uniformly nondegenerate stochastic differential games in smooth domains up to a constant over $K$ with the ones having second-order derivatives bounded by a constant times $K$ for any $K\geq1$

Book Source: Digital Library of India Item 2015.199543 dc.contributor.author: Krishnanand, Rammohan, Ragade dc.date.accessioned: 2015-07-08T13:15:58Z dc.date.available: 2015-07-08T13:15:58Z dc.date.digitalpublicationdate: 2005-08-27 dc.identifier.barcode: 5990010101970 dc.identifier.origpath: /rawdataupload/upload/0101/972 dc.identifier.copyno: 1 dc.identifier.uri: http://www.new.dli.ernet.in/handle/2015/199543 dc.description.scannerno: 14 dc.description.scanningcentre: IIIT, Allahabad dc.description.main: 1 dc.description.tagged: 0 dc.description.totalpages: 91 dc.format.mimetype: application/pdf dc.language.iso: English dc.publisher: Indian Institute Of Technology Kanpur dc.rights: Out_of_copyright dc.source.library: Indian Institute Of Technology Kanpur dc.subject.classification: Technology dc.subject.classification: Electrical Engineering dc.title: Studies In Differential Games With Applications To Optimal Control Under Uncertainty

In this second part of our two-part paper, we invoke the stochastic maximum principle, conditional Hamiltonian and the coupled backward-forward stochastic differential equations of the first part [1] to derive team optimal decentralized strategies for distributed stochastic differential systems with noiseless information structures. We present examples of such team games of nonlinear as well as linear quadratic forms. In some cases we obtain closed form expressions of the optimal decentralized strategies. Through the examples, we illustrate the effect of information signaling among the decision makers in reducing the computational complexity of optimal decentralized decision strategies.

This paper is concerned with stochastic differential games (SDGs) defined through fully coupled forward-backward stochastic differential equations (FBSDEs) which are governed by Brownian motion and Poisson random measure. For SDGs, the upper and the lower value functions are defined by the controlled fully coupled FBSDEs with jumps. Using a new transformation introduced in [6], we prove that the upper and the lower value functions are deterministic. Then, after establishing the dynamic programming principle for the upper and the lower value functions of this SDGs, we prove that the upper and the lower value functions are the viscosity solutions to the associated upper and the lower Hamilton-Jacobi-Bellman-Isaacs (HJBI) equations, respectively. Furthermore, for a special case (when $\sigma,\ h$ do not depend on $y,\ z,\ k$), under the Isaacs' condition, we get the existence of the value of the game.

This paper is concerned with a non-zero sum differential game problem of an anticipated forward-backward stochastic differential delayed equation under partial information. We establish a necessary maximum principle and sufficient verification theorem of the game system by virtue of the duality and convex variational method. We apply the theoretical results and stochastic filtering theory to study a linear-quadratic game system and derive the explicit form of the Nash equilibrium point and discuss the existence and uniqueness in particular cases. As an application, we consider a time-delayed pension fund manage problem with nonlinear expectation and obtain the Nash equilibrium point.

We study stochastic differential games of jump diffusions, where the players have access to inside information. Our approach is based on anticipative stochastic calculus, white noise, Hida-Malliavin calculus, forward integrals and the Donsker delta functional. We obtain a characterization of Nash equilibria of such games in terms of the corresponding Hamiltonians. This is used to study applications to insider games in finance, specifically optimal insider consumption and optimal insider portfolio under model uncertainty.

We study four systems and their interactions. First, we formulate a unified system of coupled forward-backward stochastic partial differential equations (FB-SPDEs) with Levy jumps, whose drift, diffusion, and jump coefficients may involve partial differential operators. A solution to the FB-SPDEs is defined by a 4-tuple general dimensional random vector-field process evolving in time together with position parameters over a domain (e.g., a hyperbox or a manifold). Under an infinite sequence of generalized local linear growth and Lipschitz conditions, the well-posedness of an adapted 4-tuple strong solution is proved over a suitably constructed topological space. Second, we consider a unified system of FB-SDEs, a special form of the FB-SPDEs, however, with skew boundary reflections. Under randomized linear growth and Lipschitz conditions together with a general completely-S condition on reflections, we prove the well-posedness of an adapted 6-tuple weak solution with boundary regulators to the FB-SDEs by the Skorohod problem and an oscillation inequality. Particularly, if the spectral radii in some sense for reflection matrices are strictly less than the unity, an adapted 6-tuple strong solution is concerned. Third, we formulate a stochastic differential game (SDG) with general number of players based on the FB-SDEs. By a solution to the FB-SPDEs, we get a solution to the FB-SDEs under a given control rule and then obtain a Pareto optimal Nash equilibrium policy process to the SDG. Fourth, we study the applications of the FB-SPDEs/FB-SDEs in queueing systems and quantum statistics while we use them to motivate the SDG.

The problem of prior and delayed commitment in zero sum stochastic differential games is discussed. A new formulation and solution based on the delayed-commitment model is derived and its significant implications to stochastic game and control are considered.

March 30 1970 - June 19, 1970.

In the present work, we consider 2-person zero-sum stochastic differential games with a nonlinear pay-off functional which is defined through a backward stochastic differential equation. Our main objective is to study for such a game the problem of the existence of a value without Isaacs condition. Not surprising, this requires a suitable concept of mixed strategies which, to the authors' best knowledge, was not known in the context of stochastic differential games. For this, we consider nonanticipative strategies with a delay defined through a partition $\pi$ of the time interval $[0,T]$. The underlying stochastic controls for the both players are randomized along $\pi$ by a hazard which is independent of the governing Brownian motion, and knowing the information available at the left time point $t_{j-1}$ of the subintervals generated by $\pi$, the controls of Players 1 and 2 are conditionally independent over $[t_{j-1},t_j)$. It is shown that the associated lower and upper value functions $W^{\pi}$ and $U^{\pi}$ converge uniformly on compacts to a function $V$, the so-called value in mixed strategies, as the mesh of $\pi$ tends to zero. This function $V$ is characterized as the unique viscosity solution of the associated Hamilton-Jacobi-Bellman-Isaacs equation.

This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work is the unique solvability of a class of linear backward stochastic differential equations in an infinite horizon.

In this paper, we investigate Nash equilibrium payoffs for nonzero-sum stochastic differential games with reflection. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for nonzero-sum stochastic differential games with nonlinear cost functionals defined by doubly controlled reflected backward stochastic differential equations.

Includes bibliographical references (leaf 12)

The value of a zero-sum differential games is known to exist, under Isaacs' condition, as the unique viscosity solution of a Hamilton-Jacobi-Bellman equation. In this note we provide a self-contained proof based on the construction of $\ep$-optimal strategies, which is inspired by the "extremal aiming" method from Krasovskii and Subbotin.

We consider a team game reward, and we derive a stochastic Pontryagin's maximum principle for distributed stochastic differential systems with decentralized noisy information structures. Our methodology utilizes the semi martingale representation theorem, variational methods, and backward stochastic differential equations. Furthermore, we derive necessary and sufficient optimality conditions that characterize team and person-by-person optimality of decentralized strategies. Finally, we apply the stochastic maximum principle to several examples from the application areas of communications, filtering and control.

A qualitative criterion for a pursuer to intercept a target in a class of differential games is obtained in terms of \emph{future cones}: Topological cones that contain all attainable trajectories of target or interceptor originating from an initial position. An interception solution exists after some initial time iff the future cone of the target lies within the future cone of the interceptor. The solution may be regarded as a kind of Nash equillibrium. This result is applied to two examples: 1. The game of Two Cars: The future cone condition is shown to be equivalent to conditions for interception obtained by Cockayne. \cite{ref2} 2. Satellite warfare: The future cone for a spacecraft or direct-ascent antisatellite weapon (ASAT) maneuvering in a central gravitational field is obtained and is shown to equal that for a spacecraft which maneuvers solely by means of a single velocity change at the cone vertex. The latter result is illustrated with an analysis of the January 2007 interception of the FengYun-1C spacecraft.

Chapter 2 (Sector B11 - B20

We study zero-sum stochastic differential games with player dynamics governed by a nondegenerate controlled diffusion process. Under the assumption of uniform stability, we establish the existence of a solution to the Isaac's equation for the ergodic game and characterize the optimal stationary strategies. The data is not assumed to be bounded, nor do we assume geometric ergodicity. Thus our results extend previous work in the literature. We also study a relative value iteration scheme that takes the form of a parabolic Isaac's equation. Under the hypothesis of geometric ergodicity we show that the relative value iteration converges to the elliptic Isaac's equation as time goes to infinity. We use these results to establish convergence of the relative value iteration for risk-sensitive control problems under an asymptotic flatness assumption.

The direct method developed by I. S. Pontryagin for solving linear differential games is generalized to the non-stationary case.

This paper provides a decomposition technique for the purpose of simplifying the solution of certain zero-sum differential games. The games considered terminate when the state reaches a target, which can be expressed as the union of a collection of target subsets; the decomposition consists of replacing the original target by each of the target subsets. The value of the original game is then obtained as the lower envelope of the values of the collection of games resulting from the decomposition, which can be much easier to solve than the original game. Criteria are given for the validity of the decomposition. The paper includes examples, illustrating the application of the technique to pursuit/evasion games, where the decomposition arises from considering the interaction of individual pursuer/evader pairs.

In this paper, we develop a method for solving a large class of non-convex Hamilton-Jacobi partial differential equations (HJ PDE). The method yields decoupled subproblems, which can be solved in an embarrassingly parallel fashion. The complexity of the resulting algorithm is polynomial in the problem dimension; hence, it overcomes the curse of dimensionality [1, 2]. We extend previous work in[6] and apply the Hopf formula to solve HJ PDE involving non-convex Hamiltonians. We propose an ADMM approach for finding the minimizer associated with the Hopf formula. Some explicit formulae of proximal maps, as well as newly-defined stretch operators, are used in the numerical solutions of ADMM subproblems. Our approach is expected to have wide applications in continuous dynamic games, control theory problems, and elsewhere.

In this paper the set of value functions of all-possible zero-sum differential games with terminal payoff is characterized. The necessary and sufficient condition for a given function to be a value of some differential game with terminal payoff is obtained.

A new proof for the existence of a value and of good strategies for a zero-sum two-person game is given. This proof seems to have some interest because of two distinguishing traits: (a) although the theorem to be proved is of an algebraical nature, a very simple proof obtains by analytical means; and (b) the proof is constructive in a sense that lends itself to utilization when actually computing the solutions of specific games. The procedure could be mechanized with relative ease, both for digital and for analogy methods. In the latter case it is probably much less sensitive to the precision of the equipment, than the somewhat related problem of linear equation solving or matrix inversion.

The theory of nonzero sum differential games (NZSDG) is extended for a class of problems in which the nonlinear system equations have bounded control variables appearing linearly. For terminal cost functions this class of problems is shown to exhibit 'bang-bang' type control laws with the possibility of singular controls. A condition is derived to test for continuity of the influence functions when controls switch from a nonsingular to a singular control on singular surfaces. Two generalized forms of the transversality conditions are derived for NZSDG theory extending results of Dreyfus and Isaacs. NZSDG theory is shown to be useful in modeling combat problems in which the goals of the players are not diametrically opposed. A two player and three player penetrator-interceptor problem are presented as a NZSDG. Numerical solutions for a totally singular problem are carried out to illustrate application and a typical solution.

The zero-sum, perfect information pursuit-evasion differential game is reviewed. The purpose of the thesis is to formulate a method for generating near-optimal closed-loop solutions to these problems. The method is then applied to a number of example problems in order to check its validity. This method deals with solutions in the small and is based on updating the two-point boundary-value problem by use of the neighboring extremal path concept. The two differential game problems examined are a simple motion problem and a rocket problem. Two separate cases were studied for each problem. One was the fixed final time problem and the other was the free final time with a terminal constraint. Analysis of the results obtained, supports the feasibility of this method to provide near-optimal closed-loop solutions to differential game problems.

Methods of the calculus of variations were employed to obtain necessary conditions that differential games of a certain type have a saddle-point. By strengthening the necessary conditions, sufficient conditions for the existence of a saddle-point and a method of constructing the saddle-point were derived.

Variational methods to solve pursuit-evasion differential game

Variational methods to solve pursuit-evasion differential game

A heuristic that has emerged in the area of importance sampling is that the changes of measure used to prove large deviation lower bounds give good performance when used for importance sampling. Recent work, however, has suggested that the heuristic is incorrect in many situations. The perspective put forth in the present paper is that large deviation theory suggests many changes of measure, and that not all are suitable for importance sampling. In the setting of Cramer's Theorem, the traditional interpretation of the heuristic suggests a fixed change of distribution on the underlying independent and identically distributed summands. In contrast, we consider importance sampling schemes where the exponential change of measure is adaptive, in the sense that it depends on the historical empirical mean. The existence of asymptotically optimal schemes within this class is demonstrated. The result indicates that an adaptive change of measure, rather than a static change of measure, is what the large deviations analysis truly suggests. The proofs utilize a control-theoretic approach to large deviations, which naturally leads to the construction of asymptotically optimal adaptive schemes in terms of a limit Bellman equation. Numerical examples contrasting the adaptive and standard schemes are presented, as well as an interpretation of their different performances in terms of differential games.

A heuristic that has emerged in the area of importance sampling is that the changes of measure used to prove large deviation lower bounds give good performance when used for importance sampling. Recent work, however, has suggested that the heuristic is incorrect in many situations. The perspective put forth in the present paper is that large deviation theory suggests many changes of measure, and that not all are suitable for importance sampling. In the setting of Cramer's Theorem, the traditional interpretation of the heuristic suggests a fixed change of distribution on the underlying independent and identically distributed summands. In contrast, we consider importance sampling schemes where the exponential change of measure is adaptive, in the sense that it depends on the historical empirical mean. The existence of asymptotically optimal schemes within this class is demonstrated. The result indicates that an adaptive change of measure, rather than a static change of measure, is what the large deviations analysis truly suggests. The proofs utilize a control-theoretic approach to large deviations, which naturally leads to the construction of asymptotically optimal adaptive schemes in terms of a limit Bellman equation. Numerical examples contrasting the adaptive and standard schemes are presented, as well as an interpretation of their different performances in terms of differential games.

We consider a multidimensional stochastic differential game that emerges from a multiclass M/M/1 queueing problem under heavy-traffic with model uncertainty. Namely, it is assumed that the decision maker is uncertain about the rates of arrivals to the system and the rates of service and acts to optimize an overall cost that accounts for this uncertainty. We show that the multidimensional game can be reduced to a one-dimensional stochastic differential game. Then, the value function of the games is characterized as the unique solution to a free-boundary problem from which we extract equilibria for both the reduced and the multidimensional games. Finally, we analyze the dependence of the value function and the equilibria on the ambiguity parameters.

For a non-cooperative differential game, the value functions of the various players satisfy a system of Hamilton-Jacobi equations. In the present paper, we consider a class of infinite-horizon games with nonlinear costs exponentially discounted in time. By the analysis of the value functions, we establish the existence of Nash equilibrium solutions in feedback form and provide results and counterexamples on their uniqueness and stability.

For a non-cooperative differential game, the value functions of the various players satisfy a system of Hamilton-Jacobi equations. In the present paper, we consider a class of infinite-horizon games with nonlinear costs exponentially discounted in time. By the analysis of the value functions, we establish the existence of Nash equilibrium solutions in feedback form and provide results and counterexamples on their uniqueness and stability.

partial report for period September 1973 - November 1974.