Downloads & Free Reading Options - Results

This paper, discusses some research issues related to the general topic of optimizing a stochastic system via simulation. In particular, we devote extensive attention to finite difference estimators of objective function gradients and present a number of new limit theorems. We also discuss a new family of orthogonal function approximations to the global behavior of the objective function. We show that if the objective function is sufficiently smooth, the convergence rate can be made arbitrarily close to n to the minus half power in the number of observations required. The paper concludes with a brief discussion of how these ideas can be integrated into an optimization algorithm.

We describe methods for proving upper and lower bounds on infinite-time averages in deterministic dynamical systems and on stationary expectations in stochastic systems. The dynamics and the quantities to be bounded are assumed to be polynomial functions of the state variables. The methods are computer-assisted, using sum-of-squares polynomials to formulate sufficient conditions that can be checked by semidefinite programming. In the deterministic case, we seek tight bounds that apply to particular local attractors. An obstacle to proving such bounds is that they do not hold globally; they are generally violated by trajectories starting outside the local basin of attraction. We describe two closely related ways past this obstacle: one that requires knowing a subset of the basin of attraction, and another that considers the zero-noise limit of the corresponding stochastic system. The bounding methods are illustrated using the van der Pol oscillator. We bound deterministic averages on the attracting limit cycle above and below to within 1%, which requires a lower bound that does not hold for the unstable fixed point at the origin. We obtain similarly tight upper and lower bounds on stochastic expectations for a range of noise amplitudes. Limitations of our methods for certain types of deterministic systems are discussed, along with prospects for improvement.

For linear stochastic systems with time delay, the optimal control is derived that minimizes the ensemble average of a quadratic (in states and control) performance measure. The optimal control obtained is functionally dependent upon the expected values of the state variables conditioned on the measurements. It is shown that the optimal control and estimation can be performed independently; i.e., the separation theorem holds for the class of problems considered. The optimal control is linearly dependent upon the best estimates which minimize the expected value of the estimation error squared.

A new class of algorithms is introduced and analyzed for bound and linearly constrained optimization problems with stochastic objective functions and a mixture of design variable types. The generalized pattern search (GPS) class of algorithms is extended to a new problem setting in which objective function evaluations require sampling from a model of a stochastic system. The approach combines GPS with ranking and selection (R&S) statistical procedures to select new iterates. The derivative-free algorithms require only black-box simulation responses and are applicable over domains with mixed variables (continuous, discrete numeric, and discrete categorical) to include bound and linear constraints on the continuous variables. A convergence analysis for the general class of algorithms establishes almost sure convergence of an iteration subsequence to stationary points appropriately defined in the mixed-variable domain. Additionally, specific algorithm instances are implemented that provide computational enhancements to the basic algorithm. Implementation alternatives include the use modern R&S procedures designed to provide efficient sampling strategies and the use of surrogate functions that augment the search by approximating the unknown objective function with nonparametric response surfaces. In a computational evaluation, six variants of the algorithm are tested along with four competing methods on 26 standardized test problems. The numerical results validate the use of advanced implementations as a means to improve algorithm performance.

A research approach is presented for solving stochastic, multi-objective optimization problems. First, the class of mesh adaptive direct search (MADS) algorithms for nonlinearly constrained optimization is extended to mixed variable problems. The resulting algorithm, MV-MADS, is then extended to stochastic problems (MVMADS-RS), via a ranking and selection procedure. Finally, a two-stage method is developed that combines the generalized pattern search/ranking and selection (MGPS-RS) algorithms for single-objective, mixed variable, stochastic problems with a multi-objective approach that makes use of interactive techniques for the specification of aspiration and reservation levels, scalarization functions, and multi-objective ranking and selection. A convergence analysis for the general class of algorithms establishes almost sure convergence of an iteration subsequence to stationary points appropriately defined in the mixed-variable domain. Seven specific instances of the new algorithm are implemented and tested on 11 multi-objective test problems from the literature and an engineering design problem.

Two new algorithms are presented for multi-objective optimization of mixed-variable, stochastic systems. Both are based off of prior algorithms, but combine those pre-exsting algorithms with several other methods, to include single-objective formulations, surrogates, n-dimensional visualizations, aspiration an reservation levels, and direct search methods. Results are shown for a test set of 13 problems, ranging from 2 to 8 objectives, and including non-convex, mixed-variable, and discontinuous problems.

The design of modern manufacturing systems presents a number of challenges. In particular, the stochastic nature of machine failures in combination with the large number of decision variables makes optimization of such systems difficult. This paper presents two new approaches to optimization of the complex stochastic systems that arise in a manufacturing context; both are Monte Carlo simulation-oriented, and are therefore broadly applicable. The first technique involves using a likelihood ratio gradient estimate to drive a Robbins-Monro algorithm, and is relevant to problems in which the decision variables are continuous. The second idea employs homotopy methods to follow an optimal path in decision variable space, and can be used for both discrete and continuous optimization.

Specific aims of our projects have been to study the following topics: (1) Backward Stochastic Differential Equations with reflection and connection with Dynkin games; (2) A deterministic approach to discrete-time Dynkin games; (3) Singular control problems with application to irreversible investment; (4) Synchronization and optimality for d-armed bandit problems; (5) Adaptive control of a diffusion to a goal, and connection to a parabolic Monge-Ampere-type equation; (6) Backward Stochastic Differential Equations with Constraints on the gains-process; (7) Control and stopping of a diffusion process on an interval. Findings include establishing existence and uniqueness results for solutions to these problems and their analytical characterizations. These results are significant because they provide explicit ways of constructing solutions or even explicit expressions for optimal processes in the above problems.

Polynomial Chaos Expansions represent a powerful tool to simulate stochastic models of dynamical systems. Yet, deriving the expansion's coefficients for complex systems might require a significant and non-trivial manipulation of the model, or the computation of large numbers of simulation runs, rendering the approach too time consuming and impracticable for applications with more than a handful of random variables. We introduce a novel computationally tractable technique for computing the coefficients of polynomial chaos expansions. The approach exploits a regularization technique with a particular choice of weighting matrices, which allow to take into account the specific features of Polynomial Chaos expansions. The method, completely based on convex optimization, can be applied to problems with a large number of random variables and uses a modest number of Monte Carlo simulations, while avoiding model manipulations. Additional information on the stochastic process, when available, can be also incorporated in the approach by means of convex constraints. We show the effectiveness of the proposed technique in three applications in diverse fields, including the analysis of a nonlinear electric circuit, a chaotic model of organizational behavior, finally a chemical oscillator.

Solid state drives (SSDs) have seen wide deployment in mobiles, desktops, and data centers due to their high I/O performance and low energy consumption. As SSDs write data out-of-place, garbage collection (GC) is required to erase and reclaim space with invalid data. However, GC poses additional writes that hinder the I/O performance, while SSD blocks can only endure a finite number of erasures. Thus, there is a performance-durability tradeoff on the design space of GC. To characterize the optimal tradeoff, this paper formulates an analytical model that explores the full optimal design space of any GC algorithm. We first present a stochastic Markov chain model that captures the I/O dynamics of large-scale SSDs, and adapt the mean-field approach to derive the asymptotic steady-state performance. We further prove the model convergence and generalize the model for all types of workload. Inspired by this model, we propose a randomized greedy algorithm (RGA) that can operate along the optimal tradeoff curve with a tunable parameter. Using trace-driven simulation on DiskSim with SSD add-ons, we demonstrate how RGA can be parameterized to realize the performance-durability tradeoff.

For high dimensional or nonlinear problems there are serious limitations on the power of available computational methods for the optimization or parametric optimization of stochastic systems of diffusion type. The paper developes an effective Monte Carlo method for obtaining good estimators of systems sensitivities with respect to system parameters, when the system is interest over a long period of time. The value of the method is borne out by numerical experiments, and the computational requirements are favorable with respect to competing methods when the dimension is high or the nonlinearities 'severe'. The method is a type of derivative of likelihood ratio method method. For a wide class of problems, the cost function or dynamics need not be smooth in the state variables; for example, where the cost is the probability of an event or sign functions appear in the dynamics. Under appropriate conditions, it is shown that the invariant measures are differentiable with respect to the parameters. Since the basic diffusion (or other) model cannot be simulated exactly, simulatable approximations are discussed in detail, and estimators are obtained and analyzed. It is shown that these estimators and their expectations converge to those for the original problem. Keywords: Parametric optimization of stochastic systems, Ergodic control.

We extend the definition of a Stochastic Hybrid Automaton (SHA) to overcome limitations that make it difficult to use for on-line control. Since guard sets do not specify the exact event causing a transition, we introduce a clock structure (borrowed from timed automata), timer states, and guard functions that disambiguate how transitions occur. In the modified SHA, we formally show that every transition is associated with an explicit element of an underlying event set. This also makes it possible to uniformly treat all events observed on a sample path of a stochastic hybrid system and generalize the performance sensitivity estimators derived through Infinitesimal Perturbation Analysis (IPA). We eliminate the need for a case-by-case treatment of different event types and provide a unified set of matrix IPA equations. We illustrate our approach by revisiting an optimization problem for single node finite-capacity stochastic flow systems to obtain performance sensitivity estimates in this new setting.

We extend the definition of a Stochastic Hybrid Automaton (SHA) to overcome limitations that make it difficult to use for on-line control. Since guard sets do not specify the exact event causing a transition, we introduce a clock structure (borrowed from timed automata), timer states, and guard functions that disambiguate how transitions occur. In the modified SHA, we formally show that every transition is associated with an explicit element of an underlying event set. This also makes it possible to uniformly treat all events observed on a sample path of a stochastic hybrid system and generalize the performance sensitivity estimators derived through Infinitesimal Perturbation Analysis (IPA). We eliminate the need for a case-by-case treatment of different event types and provide a unified set of matrix IPA equations. We illustrate our approach by revisiting an optimization problem for single node finite-capacity stochastic flow systems to obtain performance sensitivity estimates in this new setting.

We extend the definition of a Stochastic Hybrid Automaton (SHA) to overcome limitations that make it difficult to use for on-line control. Since guard sets do not specify the exact event causing a transition, we introduce a clock structure (borrowed from timed automata), timer states, and guard functions that disambiguate how transitions occur. In the modified SHA, we formally show that every transition is associated with an explicit element of an underlying event set. This also makes it possible to uniformly treat all events observed on a sample path of a stochastic hybrid system and generalize the performance sensitivity estimators derived through Infinitesimal Perturbation Analysis (IPA). We eliminate the need for a case-by-case treatment of different event types and provide a unified set of matrix IPA equations. We illustrate our approach by revisiting an optimization problem for single node finite-capacity stochastic flow systems to obtain performance sensitivity estimates in this new setting.