Downloads & Free Reading Options - Results

Specific aims of our projects have been to study the following topics: (1) Backward Stochastic Differential Equations with reflection and connection with Dynkin games; (2) A deterministic approach to discrete-time Dynkin games; (3) Singular control problems with application to irreversible investment; (4) Synchronization and optimality for d-armed bandit problems; (5) Adaptive control of a diffusion to a goal, and connection to a parabolic Monge-Ampere-type equation; (6) Backward Stochastic Differential Equations with Constraints on the gains-process; (7) Control and stopping of a diffusion process on an interval. Findings include establishing existence and uniqueness results for solutions to these problems and their analytical characterizations. These results are significant because they provide explicit ways of constructing solutions or even explicit expressions for optimal processes in the above problems.

We extend the definition of a Stochastic Hybrid Automaton (SHA) to overcome limitations that make it difficult to use for on-line control. Since guard sets do not specify the exact event causing a transition, we introduce a clock structure (borrowed from timed automata), timer states, and guard functions that disambiguate how transitions occur. In the modified SHA, we formally show that every transition is associated with an explicit element of an underlying event set. This also makes it possible to uniformly treat all events observed on a sample path of a stochastic hybrid system and generalize the performance sensitivity estimators derived through Infinitesimal Perturbation Analysis (IPA). We eliminate the need for a case-by-case treatment of different event types and provide a unified set of matrix IPA equations. We illustrate our approach by revisiting an optimization problem for single node finite-capacity stochastic flow systems to obtain performance sensitivity estimates in this new setting.

We extend the definition of a Stochastic Hybrid Automaton (SHA) to overcome limitations that make it difficult to use for on-line control. Since guard sets do not specify the exact event causing a transition, we introduce a clock structure (borrowed from timed automata), timer states, and guard functions that disambiguate how transitions occur. In the modified SHA, we formally show that every transition is associated with an explicit element of an underlying event set. This also makes it possible to uniformly treat all events observed on a sample path of a stochastic hybrid system and generalize the performance sensitivity estimators derived through Infinitesimal Perturbation Analysis (IPA). We eliminate the need for a case-by-case treatment of different event types and provide a unified set of matrix IPA equations. We illustrate our approach by revisiting an optimization problem for single node finite-capacity stochastic flow systems to obtain performance sensitivity estimates in this new setting.

A new class of algorithms is introduced and analyzed for bound and linearly constrained optimization problems with stochastic objective functions and a mixture of design variable types. The generalized pattern search (GPS) class of algorithms is extended to a new problem setting in which objective function evaluations require sampling from a model of a stochastic system. The approach combines GPS with ranking and selection (R&S) statistical procedures to select new iterates. The derivative-free algorithms require only black-box simulation responses and are applicable over domains with mixed variables (continuous, discrete numeric, and discrete categorical) to include bound and linear constraints on the continuous variables. A convergence analysis for the general class of algorithms establishes almost sure convergence of an iteration subsequence to stationary points appropriately defined in the mixed-variable domain. Additionally, specific algorithm instances are implemented that provide computational enhancements to the basic algorithm. Implementation alternatives include the use modern R&S procedures designed to provide efficient sampling strategies and the use of surrogate functions that augment the search by approximating the unknown objective function with nonparametric response surfaces. In a computational evaluation, six variants of the algorithm are tested along with four competing methods on 26 standardized test problems. The numerical results validate the use of advanced implementations as a means to improve algorithm performance.

For high dimensional or nonlinear problems there are serious limitations on the power of available computational methods for the optimization or parametric optimization of stochastic systems of diffusion type. The paper developes an effective Monte Carlo method for obtaining good estimators of systems sensitivities with respect to system parameters, when the system is interest over a long period of time. The value of the method is borne out by numerical experiments, and the computational requirements are favorable with respect to competing methods when the dimension is high or the nonlinearities 'severe'. The method is a type of derivative of likelihood ratio method method. For a wide class of problems, the cost function or dynamics need not be smooth in the state variables; for example, where the cost is the probability of an event or sign functions appear in the dynamics. Under appropriate conditions, it is shown that the invariant measures are differentiable with respect to the parameters. Since the basic diffusion (or other) model cannot be simulated exactly, simulatable approximations are discussed in detail, and estimators are obtained and analyzed. It is shown that these estimators and their expectations converge to those for the original problem. Keywords: Parametric optimization of stochastic systems, Ergodic control.

We describe methods for proving upper and lower bounds on infinite-time averages in deterministic dynamical systems and on stationary expectations in stochastic systems. The dynamics and the quantities to be bounded are assumed to be polynomial functions of the state variables. The methods are computer-assisted, using sum-of-squares polynomials to formulate sufficient conditions that can be checked by semidefinite programming. In the deterministic case, we seek tight bounds that apply to particular local attractors. An obstacle to proving such bounds is that they do not hold globally; they are generally violated by trajectories starting outside the local basin of attraction. We describe two closely related ways past this obstacle: one that requires knowing a subset of the basin of attraction, and another that considers the zero-noise limit of the corresponding stochastic system. The bounding methods are illustrated using the van der Pol oscillator. We bound deterministic averages on the attracting limit cycle above and below to within 1%, which requires a lower bound that does not hold for the unstable fixed point at the origin. We obtain similarly tight upper and lower bounds on stochastic expectations for a range of noise amplitudes. Limitations of our methods for certain types of deterministic systems are discussed, along with prospects for improvement.

Polynomial Chaos Expansions represent a powerful tool to simulate stochastic models of dynamical systems. Yet, deriving the expansion's coefficients for complex systems might require a significant and non-trivial manipulation of the model, or the computation of large numbers of simulation runs, rendering the approach too time consuming and impracticable for applications with more than a handful of random variables. We introduce a novel computationally tractable technique for computing the coefficients of polynomial chaos expansions. The approach exploits a regularization technique with a particular choice of weighting matrices, which allow to take into account the specific features of Polynomial Chaos expansions. The method, completely based on convex optimization, can be applied to problems with a large number of random variables and uses a modest number of Monte Carlo simulations, while avoiding model manipulations. Additional information on the stochastic process, when available, can be also incorporated in the approach by means of convex constraints. We show the effectiveness of the proposed technique in three applications in diverse fields, including the analysis of a nonlinear electric circuit, a chaotic model of organizational behavior, finally a chemical oscillator.

Solid state drives (SSDs) have seen wide deployment in mobiles, desktops, and data centers due to their high I/O performance and low energy consumption. As SSDs write data out-of-place, garbage collection (GC) is required to erase and reclaim space with invalid data. However, GC poses additional writes that hinder the I/O performance, while SSD blocks can only endure a finite number of erasures. Thus, there is a performance-durability tradeoff on the design space of GC. To characterize the optimal tradeoff, this paper formulates an analytical model that explores the full optimal design space of any GC algorithm. We first present a stochastic Markov chain model that captures the I/O dynamics of large-scale SSDs, and adapt the mean-field approach to derive the asymptotic steady-state performance. We further prove the model convergence and generalize the model for all types of workload. Inspired by this model, we propose a randomized greedy algorithm (RGA) that can operate along the optimal tradeoff curve with a tunable parameter. Using trace-driven simulation on DiskSim with SSD add-ons, we demonstrate how RGA can be parameterized to realize the performance-durability tradeoff.

Solid state drives (SSDs) have seen wide deployment in mobiles, desktops, and data centers due to their high I/O performance and low energy consumption. As SSDs write data out-of-place, garbage collection (GC) is required to erase and reclaim space with invalid data. However, GC poses additional writes that hinder the I/O performance, while SSD blocks can only endure a finite number of erasures. Thus, there is a performance-durability tradeoff on the design space of GC. To characterize the optimal tradeoff, this paper formulates an analytical model that explores the full optimal design space of any GC algorithm. We first present a stochastic Markov chain model that captures the I/O dynamics of large-scale SSDs, and adapt the mean-field approach to derive the asymptotic steady-state performance. We further prove the model convergence and generalize the model for all types of workload. Inspired by this model, we propose a randomized greedy algorithm (RGA) that can operate along the optimal tradeoff curve with a tunable parameter. Using trace-driven simulation on DiskSim with SSD add-ons, we demonstrate how RGA can be parameterized to realize the performance-durability tradeoff.

Solid state drives (SSDs) have seen wide deployment in mobiles, desktops, and data centers due to their high I/O performance and low energy consumption. As SSDs write data out-of-place, garbage collection (GC) is required to erase and reclaim space with invalid data. However, GC poses additional writes that hinder the I/O performance, while SSD blocks can only endure a finite number of erasures. Thus, there is a performance-durability tradeoff on the design space of GC. To characterize the optimal tradeoff, this paper formulates an analytical model that explores the full optimal design space of any GC algorithm. We first present a stochastic Markov chain model that captures the I/O dynamics of large-scale SSDs, and adapt the mean-field approach to derive the asymptotic steady-state performance. We further prove the model convergence and generalize the model for all types of workload. Inspired by this model, we propose a randomized greedy algorithm (RGA) that can operate along the optimal tradeoff curve with a tunable parameter. Using trace-driven simulation on DiskSim with SSD add-ons, we demonstrate how RGA can be parameterized to realize the performance-durability tradeoff.