Downloads & Free Reading Options - Results

Title from cover

We provide a framework to bound the probability that accumulated errors were never above a given threshold on numerical algorithms. Such algorithms are used for example in aircraft and nuclear power plants. This report contains simple formulas based on Levy's and Markov's inequalities and it presents a formal theory of random variables with a special focus on producing concrete results. We selected four very common applications that fit in our framework and cover the common practices of systems that evolve for a long time. We compute the number of bits that remain continuously significant in the first two applications with a probability of failure around one out of a billion, where worst case analysis considers that no significant bit remains. We are using PVS as such formal tools force explicit statement of all hypotheses and prevent incorrect uses of theorems.

A computer program implements stochastic evolutionary algorithms for planning and optimizing collision-free paths for robots and their jointed limbs. Stochastic evolutionary algorithms can be made to produce acceptably close approximations to exact, optimal solutions for path-planning problems while often demanding much less computation than do exhaustive-search and deterministic inverse-kinematics algorithms that have been used previously for this purpose. Hence, the present software is better suited for application aboard robots having limited computing capabilities (see figure). The stochastic aspect lies in the use of simulated annealing to (1) prevent trapping of an optimization algorithm in local minima of an energy-like error measure by which the fitness of a trial solution is evaluated while (2) ensuring that the entire multidimensional configuration and parameter space of the path-planning problem is sampled efficiently with respect to both robot joint angles and computation time. Simulated annealing is an established technique for avoiding local minima in multidimensional optimization problems, but has not, until now, been applied to planning collision-free robot paths by use of low-power computers.

A computer program implements stochastic evolutionary algorithms for planning and optimizing collision-free paths for robots and their jointed limbs. Stochastic evolutionary algorithms can be made to produce acceptably close approximations to exact, optimal solutions for path-planning problems while often demanding much less computation than do exhaustive-search and deterministic inverse-kinematics algorithms that have been used previously for this purpose. Hence, the present software is better suited for application aboard robots having limited computing capabilities (see figure). The stochastic aspect lies in the use of simulated annealing to (1) prevent trapping of an optimization algorithm in local minima of an energy-like error measure by which the fitness of a trial solution is evaluated while (2) ensuring that the entire multidimensional configuration and parameter space of the path-planning problem is sampled efficiently with respect to both robot joint angles and computation time. Simulated annealing is an established technique for avoiding local minima in multidimensional optimization problems, but has not, until now, been applied to planning collision-free robot paths by use of low-power computers.

We consider several stochastic service systems, and study the asymptotic behavior of the moments of various quantities that have application to models for random interval graphs and algorithms for searching for an idle server or empty waiting station. In two cases the moments turn out to involve Lambert series for the generating functions for the sums of powers of divisors of positive integers. For these cases we are able to obtain complete asymptotic expansions for the moments of the quantities in question.

We prove a central limit theorem applicable to one dimensional stochastic approximation algorithms that converge to a point where the error terms of the algorithm do not vanish. We show how this applies to a certain class of these algorithms that in particular covers a generalized P\'olya urn model, which is also discussed. In addition, we show how to scale these algorithms in some cases where we cannot determine the limiting distribution but expect it to be non-normal.

This paper studies parallelization schemes for stochastic Vector Quantization algorithms in order to obtain time speed-ups using distributed resources. We show that the most intuitive parallelization scheme does not lead to better performances than the sequential algorithm. Another distributed scheme is therefore introduced which obtains the expected speed-ups. Then, it is improved to fit implementation on distributed architectures where communications are slow and inter-machines synchronization too costly. The schemes are tested with simulated distributed architectures and, for the last one, with Microsoft Windows Azure platform obtaining speed-ups up to 32 Virtual Machines.

Several deterministic and stochastic multi-variable global optimization algorithms (Conjugate Gradient, Nelder-Mead, Quasi-Newton, and Global) are investigated in conjunction with energy minimization principle to resolve the pressure and volumetric flow rate fields in single ducts and networks of interconnected ducts. The algorithms are tested with seven types of fluid: Newtonian, power law, Bingham, Herschel-Bulkley, Ellis, Ree-Eyring and Casson. The results obtained from all those algorithms for all these types of fluid agree very well with the analytically derived solutions as obtained from the traditional methods which are based on the conservation principles and fluid constitutive relations. The results confirm and generalize the findings of our previous investigations that the energy minimization principle is at the heart of the flow dynamics systems. The investigation also enriches the methods of Computational Fluid Dynamics for solving the flow fields in tubes and networks for various types of Newtonian and non-Newtonian fluids.

This Annual Technical Report summarizes a continuation of the investigation into the use of digital nonlinear filters in conjunction with deterministic control algorithms. The problem of stabilization and control of nonlinear stochastic systems observed by noisy measurement data arises in many Air Force systems. Inherent in this problem is the problem of processing noise contaminated measurement data to obtain accurate estimates of the state of the system. If it is possible to estimate the state of the system accurately, then well-known classical deterministic control techniques may often be used to give adequate system performance. This approach will greatly reduce the complexity of the control algorithm over that required by a truly 'optimal' stochastic control policy. On the other hand, the use of recently developed filtering techniques in place of the simpler linearized or extended Kalman filter can greatly increase the accuracy of the state estimates and, thereby, improve system performance and alleviate divergence problems. (Author)

We propose here some new sampling algorithms for Path Sampling in the case when stochastic dynamics are used. In particular, we present a new proposal function for equilibrium sampling of paths with a Monte-Carlo dynamics (the so-called ``brownian tube'' proposal). This proposal is based on the continuity of the dynamics with respect to the random forcing, and generalizes all previous approaches. The efficiency of this proposal is demonstrated using some measure of decorrelation in path space. We also discuss a switching strategy that allows to transform ensemble of paths at a finite rate while remaining at equilibrium, in contrast with the usual Jarzynski like switching. This switching is very interesting to sample constrained paths starting from unconstrained paths, or to perform simulated annealing in a rigorous way.

We propose here some new sampling algorithms for Path Sampling in the case when stochastic dynamics are used. In particular, we present a new proposal function for equilibrium sampling of paths with a Monte-Carlo dynamics (the so-called ``brownian tube'' proposal). This proposal is based on the continuity of the dynamics with respect to the random forcing, and generalizes all previous approaches. The efficiency of this proposal is demonstrated using some measure of decorrelation in path space. We also discuss a switching strategy that allows to transform ensemble of paths at a finite rate while remaining at equilibrium, in contrast with the usual Jarzynski like switching. This switching is very interesting to sample constrained paths starting from unconstrained paths, or to perform simulated annealing in a rigorous way.

This report contains two main topics, each of which is connected to the stochastic realization problem. First, it considers some structural and algorithmic problems in wide sense stochastic realization theory which also have applicability to many problems outside the realm of stochastic realization theory but are here formulated in that framework. It considers some geometric questions concerning the solution set of the positive real lemma and provide a Hamiltonian framework for the non-Riccati algorithms of Kailath and Lindquist; these are then applied to the stochastic realization problem. Secondly, it applies the basic techniques and concepts of the strict sense (proper) stochastic realization theory of Lindquist and Picci and Ruckebusch to the discrete-time smoothing problem. This provides a natural interpretation of the Mayne-Fraser two-point formula as well as many other smoothing results, the interpretations of which have hitherto been quite unclear from a probabilistic point of view. Hence we have laid the ground work for a theory of smoothing which has so far been lacking.

Stochastic local search algorithms are frequently used to numerically solve hard combinatorial optimization or decision problems. We give numerical and approximate analytical descriptions of the dynamics of such algorithms applied to random satisfiability problems. We find two different dynamical regimes, depending on the number of constraints per variable: For low constraintness, the problems are solved efficiently, i.e. in linear time. For higher constraintness, the solution times become exponential. We observe that the dynamical behavior is characterized by a fast equilibration and fluctuations around this equilibrium. If the algorithm runs long enough, an exponentially rare fluctuation towards a solution appears.

We will survey recent work in the design of approximation algorithms for several discrete stochastic optimization problems, with a particular focus on 2-stage problems with recourse. In each of the problems we discuss, we are given a probability distribution over inputs, and the aim is to find a feasible solution that minimizes the expected cost of the solution found (with respect to the input distribution); an approximation algorithm finds a solution that is guaranteed to be nearly optimal. Among the specific problems that we shall discuss are stochastic generalizations of the traditional deterministic facility location problem, a simple single-machine scheduling problem, and the traveling salesman problem.These results build on techniques initially developed in the context of deterministic approximation, including rounding approaches, primal-dual algorithms, as well as a simple random sampling technique. Furthermore, although the focus of this stream of work was for discrete optimization problems, new insights for solving 2-stage stochastic linear programming problems were gained along the way. ©2008 Microsoft Corporation. All rights reserved.

We will survey recent work in the design of approximation algorithms for several discrete stochastic optimization problems, with a particular focus on 2-stage problems with recourse. In each of the problems we discuss, we are given a probability distribution over inputs, and the aim is to find a feasible solution that minimizes the expected cost of the solution found (with respect to the input distribution); an approximation algorithm finds a solution that is guaranteed to be nearly optimal. Among the specific problems that we shall discuss are stochastic generalizations of the traditional deterministic facility location problem, a simple single-machine scheduling problem, and the traveling salesman problem.These results build on techniques initially developed in the context of deterministic approximation, including rounding approaches, primal-dual algorithms, as well as a simple random sampling technique. Furthermore, although the focus of this stream of work was for discrete optimization problems, new insights for solving 2-stage stochastic linear programming problems were gained along the way. ©2008 Microsoft Corporation. All rights reserved.

The first aim of this paper is to establish the weak convergence rate of nonlinear two-time-scale stochastic approximation algorithms. Its second aim is to introduce the averaging principle in the context of two-time-scale stochastic approximation algorithms. We first define the notion of asymptotic efficiency in this framework, then introduce the averaged two-time-scale stochastic approximation algorithm, and finally establish its weak convergence rate. We show, in particular, that both components of the averaged two-time-scale stochastic approximation algorithm simultaneously converge at the optimal rate $\sqrt{n}$.

Motivation: The Stochastic Simulation Algorithm (SSA) has largely diffused in the field of systems biology. This approach needs many realizations for establishing statistical results on the system under study. It is very computationnally demanding, and with the advent of large models this burden is increasing. Hence parallel implementation of SSA are needed to address these needs. At the very heart of the SSA is the selection of the next reaction to occur at each time step, and to the best of our knowledge all implementations are based on an inverse transformation method. However, this method involves a random number of steps to select this next reaction and is poorly amenable to a parallel implementation. Results: Here, we introduce a parallel acceptance-rejection algorithm to select the K next reactions to occur. This algorithm uses a deterministic number of steps, a property well suited to a parallel implementation. It is simple and small, accurate and scalable. We propose a Graphics Processing Unit (GPU) implementation and validate our algorithm with simulated propensity distributions and the propensity distribution of a large model of yeast iron metabolism. We show that our algorithm can handle thousands of selections of next reaction to occur in parallel on the GPU, paving the way to massive SSA. Availability: We present our GPU-AR algorithm that focuses on the very heart of the SSA. We do not embed our algorithm within a full implementation in order to stay pedagogical and allows its rapid implementation in existing software. We hope that it will enable stochastic modelers to implement our algorithm with the benefits of their own optimizations.

Recently there has been renewed interests in derivative free approaches to stochastic optimization. In this paper, we examine the rates of convergence for the Kiefer-Wolfowitz algorithm and the mirror descent algorithm, under various updating schemes using finite differences as gradient approximations. It is shown that the convergence of these algorithms can be accelerated by controlling the implementation of the finite differences. Particularly, it is shown that the rate can be increased to $n^{-2/5}$ in general and to $n^{-1/2}$ in Monte Carlo optimization for a broad class of problems, in the iteration number n.

Since the introduction of the Black-Scholes model stochastic processes have played an increasingly important role in mathematical finance. In many cases prices, volatility and other quantities can be modeled using stochastic ordinary differential equations. Available methods for solving such equations have until recently been markedly inferior to analogous methods for deterministic ordinary differential equations. Recently, a number of methods which employ variable stepsizes to control local error have been developed which appear to offer greatly improved speed and accuracy. Here we conduct a comparative study of the performance of these algorithms for problems taken from the mathematical finance literature.

Construction of splitting-step methods and properties of related non-negativity and boundary preserving numerical algorithms for solving stochastic differential equations (SDEs) of Ito-type are discussed. We present convergence proofs for a newly designed splitting-step algorithm and simulation studies for numerous numerical examples ranging from stochastic dynamics occurring in asset pricing theory in mathematical finance (SDEs of CIR and CEV models) to measure-valued diffusion and superBrownian motion (SPDEs) as met in biology and physics.

This paper contributes to a development of randomized methods for neural networks. The proposed learner model is generated incrementally by stochastic configuration (SC) algorithms, termed as Stochastic Configuration Networks (SCNs). In contrast to the existing randomised learning algorithms for single layer feed-forward neural networks (SLFNNs), we randomly assign the input weights and biases of the hidden nodes in the light of a supervisory mechanism, and the output weights are analytically evaluated in either constructive or selective manner. As fundamentals of SCN-based data modelling techniques, we establish some theoretical results on the universal approximation property. Three versions of SC algorithms are presented for regression problems (applicable for classification problems as well) in this work. Simulation results concerning both function approximation and real world data regression indicate some remarkable merits of our proposed SCNs in terms of less human intervention on the network size setting, the scope adaptation of random parameters, fast learning and sound generalization.

This paper contributes to a development of randomized methods for neural networks. The proposed learner model is generated incrementally by stochastic configuration (SC) algorithms, termed as Stochastic Configuration Networks (SCNs). In contrast to the existing randomised learning algorithms for single layer feed-forward neural networks (SLFNNs), we randomly assign the input weights and biases of the hidden nodes in the light of a supervisory mechanism, and the output weights are analytically evaluated in either constructive or selective manner. As fundamentals of SCN-based data modelling techniques, we establish some theoretical results on the universal approximation property. Three versions of SC algorithms are presented for regression problems (applicable for classification problems as well) in this work. Simulation results concerning both function approximation and real world data regression indicate some remarkable merits of our proposed SCNs in terms of less human intervention on the network size setting, the scope adaptation of random parameters, fast learning and sound generalization.

I propose a large class of stochastic Markov processes associated with probability distributions analogous to that of lattice gauge theory with dynamical fermions. The construction incorporates the idea of approximate spectral split of the determinant through local loop action, and the idea of treating the infrared part of the split through explicit diagonalizations. I suggest that exact algorithms of practical relevance might be based on the Markov processes so constructed.

Background: Hidden Markov models are widely employed by numerous bioinformatics programs used today. Applications range widely from comparative gene prediction to time-series analyses of micro-array data. The parameters of the underlying models need to be adjusted for specific data sets, for example the genome of a particular species, in order to maximize the prediction accuracy. Computationally efficient algorithms for parameter training are thus key to maximizing the usability of a wide range of bioinformatics applications. Results: We introduce two computationally efficient training algorithms, one for Viterbi training and one for stochastic expectation maximization (EM) training, which render the memory requirements independent of the sequence length. Unlike the existing algorithms for Viterbi and stochastic EM training which require a two-step procedure, our two new algorithms require only one step and scan the input sequence in only one direction. We also implement these two new algorithms and the already published linear-memory algorithm for EM training into the hidden Markov model compiler HMM-Converter and examine their respective practical merits for three small example models. Conclusions: Bioinformatics applications employing hidden Markov models can use the two algorithms in order to make Viterbi training and stochastic EM training more computationally efficient. Using these algorithms, parameter training can thus be attempted for more complex models and longer training sequences. The two new algorithms have the added advantage of being easier to implement than the corresponding default algorithms for Viterbi training and stochastic EM training.

We study the convergence properties of the VR-PCA algorithm introduced by \cite{shamir2015stochastic} for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observations of independent interest, such as how pre-initializing with just a single exact power iteration can significantly improve the runtime of stochastic methods, and what are the convexity and non-convexity properties of the underlying optimization problem.

We consider unconstrained optimization problems where only "stochastic" estimates of the objective function are observable as replicates from a Monte Carlo oracle. The Monte Carlo oracle is assumed to provide no direct observations of the function gradient. We present ASTRO-DF --- a class of derivative-free trust-region algorithms, where a stochastic local interpolation model is constructed, optimized, and updated iteratively. Function estimation and model construction within ASTRO-DF is adaptive in the sense that the extent of Monte Carlo sampling is determined by continuously monitoring and balancing metrics of sampling error (or variance) and structural error (or model bias) within ASTRO-DF. Such balancing of errors is designed to ensure that Monte Carlo effort within ASTRO-DF is sensitive to algorithm trajectory, sampling more whenever an iterate is inferred to be close to a critical point and less when far away. We demonstrate the almost-sure convergence of ASTRO-DF's iterates to a first-order critical point when using linear or quadratic stochastic interpolation models. The question of using more complicated models, e.g., regression or stochastic kriging, in combination with adaptive sampling is worth further investigation and will benefit from the methods of proof presented here. We speculate that ASTRO-DF's iterates achieve the canonical Monte Carlo convergence rate, although a proof remains elusive.

In this study, we describe the algebraic computations required to implement the stochastic finite element method for solving problems in which uncertainty is restricted to right hand side data coming from forcing functions or boundary conditions. We show that the solution can be represented in a compact outer product form which leads to efficiencies in both work and sorage, and we demonstrate that block iterative methods for algebraic systems with multiple right hand sides can be used to advantage to compute this solution. Finally, we examine the behavior of these statistical quantities in one setting derived from a model of acoustic scattering.

This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov decision processes.

Stochastic convex optimization, where the objective is the expectation of a random convex function, is an important and widely used method with numerous applications in machine learning, statistics, operations research and other areas. We study the complexity of stochastic convex optimization given only statistical query (SQ) access to the objective function. We show that well-known and popular first-order iterative methods can be implemented using only statistical queries. For many cases of interest we derive nearly matching upper and lower bounds on the estimation (sample) complexity including linear optimization in the most general setting. We then present several consequences for machine learning, differential privacy and proving concrete lower bounds on the power of convex optimization based methods. The key ingredient of our work is SQ algorithms and lower bounds for estimating the mean vector of a distribution over vectors supported on a convex body in $\mathbb{R}^d$. This natural problem has not been previously studied and we show that our solutions can be used to get substantially improved SQ versions of Perceptron and other online algorithms for learning halfspaces.

We propose a semi-stochastic Frank-Wolfe algorithm with away-steps for regularized empirical risk minimization and extend it to problems with block-coordinate structure. Our algorithms use adaptive step-size and we show that they converge linearly in expectation. The proposed algorithms can be applied to many important problems in statistics and machine learning including regularized generalized linear models, support vector machines and many others. In preliminary numerical tests on structural SVM and graph-guided fused LASSO, our algorithms outperform other competing algorithms in both iteration cost and total number of data passes.

During the period April 15 1989, the Air Force Office of Scientific Research supported research work on the stochastic behavior of the LMS and related adaptive algorithms has yielded results in two major areas: Digital Implementation of Stochastic Gradient Type adaptive Algorithms; and LMS and RLS Performance Comparison for Tracking a Chirped Sinusoid in Noise. Keywords: Mathematical models, Electrical engineering, Echo cancellation, Abstracts.

We propose and study a novel stochastic inertial primal-dual approach to solve composite optimization problems. These latter problems arise naturally when learning with penalized regularization schemes. Our analysis provide convergence results in a general setting, that allows to analyze in a unified framework a variety of special cases of interest. Key in our analysis is considering the framework of splitting algorithm for solving a monotone inclusions in suitable product spaces and for a specific choice of preconditioning operators.

As part of our ONR sponsored research in the subtopic 'Linear-Time Stochastic Algorithms', we began investigating an optimization technique known as 'simulated annealing'. We have succeeded in giving a necessary and sufficient condition for the annealing algorithms to converge. New algorithms have been developed for open-loop computation of optimal state-dependent routine strategies for a fluid-approximation communication network model with a single destination. One of the algorithms is an efficient combinatorial algorithm based on the solution of max-flow problems for networks the same size as the original network. A new theory of distributed resource allocation was proposed. The work addresses problems of route selection and scheduling in communication networks. In a large distributed communication network, the decisions that individual stations can make should sometimes be purposely limited a priori in order to facilitate the coordination of such decisions. Such limitations might be placed at one layer of protocol by mechanisms operating at a higher protocol level.

As part of our ONR sponsored research in the subtopic 'Linear-Time Stochastic Algorithms', we began investigating an optimization technique known as 'simulated annealing'. We have succeeded in giving a necessary and sufficient condition for the annealing algorithms to converge. New algorithms have been developed for open-loop computation of optimal state-dependent routine strategies for a fluid-approximation communication network model with a single destination. One of the algorithms is an efficient combinatorial algorithm based on the solution of max-flow problems for networks the same size as the original network. A new theory of distributed resource allocation was proposed. The work addresses problems of route selection and scheduling in communication networks. In a large distributed communication network, the decisions that individual stations can make should sometimes be purposely limited a priori in order to facilitate the coordination of such decisions. Such limitations might be placed at one layer of protocol by mechanisms operating at a higher protocol level.

As part of our ONR sponsored research in the subtopic 'Linear-Time Stochastic Algorithms', we began investigating an optimization technique known as 'simulated annealing'. We have succeeded in giving a necessary and sufficient condition for the annealing algorithms to converge. New algorithms have been developed for open-loop computation of optimal state-dependent routine strategies for a fluid-approximation communication network model with a single destination. One of the algorithms is an efficient combinatorial algorithm based on the solution of max-flow problems for networks the same size as the original network. A new theory of distributed resource allocation was proposed. The work addresses problems of route selection and scheduling in communication networks. In a large distributed communication network, the decisions that individual stations can make should sometimes be purposely limited a priori in order to facilitate the coordination of such decisions. Such limitations might be placed at one layer of protocol by mechanisms operating at a higher protocol level.

Stochastic Algorithms: Foundations and Applications: International Symposium, SAGA 2001 Berlin, Germany, December 13–14, 2001 Proceedings Author: Kathleen Steinhöfel Published by Springer Berlin Heidelberg ISBN: 978-3-540-43025-4 DOI: 10.1007/3-540-45322-9 Table of Contents: Randomized Communication Protocols Optimal Mutation Rate Using Bayesian Priors for Estimation of Distribution Algorithms An Experimental Assessment of a Stochastic, Anytime, Decentralized, Soft Colourer for Sparse Graphs Randomized Branching Programs Yet Another Local Search Method for Constraint Solving An Evolutionary Algorithm for the Sequence Coordination in Furniture Production Evolutionary Search for Smooth Maps in Motor Control Unit Calibration Some Notes on Random Satisfiability Prospects for Simulated Annealing Algorithms in Automatic Differentiation Optimization and Simulation: Sequential Packing of Flexible Objects Using Evolutionary Algorithms Stochastic Finite Learning Sequential Sampling Algorithms: Unified Analysis and Lower Bounds Approximate Location of Relevant Variables under the Crossover Distribution

The authors present and discuss a class of continuous operations on the family of discrete time stochastic processes, which serves as a guide to construct qualitatively robust operations for a given class of processes, namely the one induced by a nominal process and a substitutive contaminating process. The results are general enough to help develop any robust statistical procedure, but the authors have concentrated their attention on detection of a change from one class of processes to another (disjoint) class of processes, while both classes consist of not necessarily Markov processes and satisfy certain mixing conditions in addition to stationarity and ergodicity. Two quantitative measures of robustness, breakdown point and influence functions are also developed for few examples.

Regularized empirical risk minimization (R-ERM) is an important branch of machine learning, since it constrains the capacity of the hypothesis space and guarantees the generalization ability of the learning algorithm. Two classic proximal optimization algorithms, i.e., proximal stochastic gradient descent (ProxSGD) and proximal stochastic coordinate descent (ProxSCD) have been widely used to solve the R-ERM problem. Recently, variance reduction technique was proposed to improve ProxSGD and ProxSCD, and the corresponding ProxSVRG and ProxSVRCD have better convergence rate. These proximal algorithms with variance reduction technique have also achieved great success in applications at small and moderate scales. However, in order to solve large-scale R-ERM problems and make more practical impacts, the parallel version of these algorithms are sorely needed. In this paper, we propose asynchronous ProxSVRG (Async-ProxSVRG) and asynchronous ProxSVRCD (Async-ProxSVRCD) algorithms, and prove that Async-ProxSVRG can achieve near linear speedup when the training data is sparse, while Async-ProxSVRCD can achieve near linear speedup regardless of the sparse condition, as long as the number of block partitions are appropriately set. We have conducted experiments on a regularized logistic regression task. The results verified our theoretical findings and demonstrated the practical efficiency of the asynchronous stochastic proximal algorithms with variance reduction.

In this paper we consider the Stochastic Matching problem, which is motivated by applications in kidney exchange and online dating. We are given an undirected graph in which every edge is assigned a probability of existence and a positive profit, and each node is assigned a positive integer called timeout. We know whether an edge exists or not only after probing it. On this random graph we are executing a process, which one-by-one probes the edges and gradually constructs a matching. The process is constrained in two ways: once an edge is taken it cannot be removed from the matching, and the timeout of node $v$ upper-bounds the number of edges incident to $v$ that can be probed. The goal is to maximize the expected profit of the constructed matching. For this problem Bansal et al. (Algorithmica 2012) provided a $3$-approximation algorithm for bipartite graphs, and a $4$-approximation for general graphs. In this work we improve the approximation factors to $2.845$ and $3.709$, respectively. We also consider an online version of the bipartite case, where one side of the partition arrives node by node, and each time a node $b$ arrives we have to decide which edges incident to $b$ we want to probe, and in which order. Here we present a $4.07$-approximation, improving on the $7.92$-approximation of Bansal et al. The main technical ingredient in our result is a novel way of probing edges according to a random but non-uniform permutation. Patching this method with an algorithm that works best for large probability edges (plus some additional ideas) leads to our improved approximation factors.

Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and the stochastic gradient thermostat. While finite-time convergence properties of the SGLD with a 1st-order Euler integrator have recently been studied, corresponding theory for general SG-MCMCs has not been explored. In this paper we consider general SG-MCMCs with high-order integrators, and develop theory to analyze finite-time convergence properties and their asymptotic invariant measures. Our theoretical results show faster convergence rates and more accurate invariant measures for SG-MCMCs with higher-order integrators. For example, with the proposed efficient 2nd-order symmetric splitting integrator, the {\em mean square error} (MSE) of the posterior average for the SGHMC achieves an optimal convergence rate of $L^{-4/5}$ at $L$ iterations, compared to $L^{-2/3}$ for the SGHMC and SGLD with 1st-order Euler integrators. Furthermore, convergence results of decreasing-step-size SG-MCMCs are also developed, with the same convergence rates as their fixed-step-size counterparts for a specific decreasing sequence. Experiments on both synthetic and real datasets verify our theory, and show advantages of the proposed method in two large-scale real applications.

We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple parallel processors to operate on a randomly chosen subset of blocks of the feature vector. We call the algorithm stochastic because processors choose training subsets uniformly at random. Algorithms that are parallel in either of these dimensions exist, but RAPSA is the first attempt at a methodology that is parallel in both the selection of blocks and the selection of elements of the training set. In RAPSA, processors utilize the randomly chosen functions to compute the stochastic gradient component associated with a randomly chosen block. The technical contribution of this paper is to show that this minimally coordinated algorithm converges to the optimal classifier when the training objective is convex. Moreover, we present an accelerated version of RAPSA (ARAPSA) that incorporates the objective function curvature information by premultiplying the descent direction by a Hessian approximation matrix. We further extend the results for asynchronous settings and show that if the processors perform their updates without any coordination the algorithms are still convergent to the optimal argument. RAPSA and its extensions are then numerically evaluated on a linear estimation problem and a binary image classification task using the MNIST handwritten digit dataset.

We prove a moderate deviation principle for the continuous time interpolation of discrete time recursive stochastic processes. The methods of proof are somewhat different from the corresponding large deviation result, and in particular the proof of the upper bound is more complicated. The results can be applied to the design of accelerated Monte Carlo algorithms for certain problems, where schemes based on moderate deviations are easier to construct and in certain situations provide performance comparable to those based on large deviations.

In this paper we study different algorithms for backward stochastic differential equations (BSDE in short) basing on random walk framework for 1-dimensional Brownian motion. Implicit and explicit schemes for both BSDE and reflected BSDE are introduced. Then we prove the convergence of different algorithms and present simulation results for different types of BSDEs.

Nested multi-step stochastic correction offers a possibility to improve updating algorithms for numerical simulations of lattice gauge theories with fermions. The corresponding generalisations of the two-step multi-boson (TSMB) algorithm as well as some applications with hybrid Monte Carlo (HMC) algorithms are considered.

In this paper we present distributed and adaptive algorithms for motion coordination of a group of m autonomous vehicles. The vehicles operate in a convex environment with bounded velocity and must service demands whose time of arrival, location and on-site service are stochastic; the objective is to minimize the expected system time (wait plus service) of the demands. The general problem is known as the m-vehicle Dynamic Traveling Repairman Problem (m-DTRP). The best previously known control algorithms rely on centralized a-priori task assignment and are not robust against changes in the environment, e.g. changes in load conditions; therefore, they are of limited applicability in scenarios involving ad-hoc networks of autonomous vehicles operating in a time-varying environment. First, we present a new class of policies for the 1-DTRP problem that: (i) are provably optimal both in light- and heavy-load condition, and (ii) are adaptive, in particular, they are robust against changes in load conditions. Second, we show that partitioning policies, whereby the environment is partitioned among the vehicles and each vehicle follows a certain set of rules in its own region, are optimal in heavy-load conditions. Finally, by combining the new class of algorithms for the 1-DTRP with suitable partitioning policies, we design distributed algorithms for the m-DTRP problem that (i) are spatially distributed, scalable to large networks, and adaptive to network changes, (ii) are within a constant-factor of optimal in heavy-load conditions and stabilize the system in any load condition. Simulation results are presented and discussed.

We discuss new global optimization algorithms that are related to the stochastic methods of Rinnooy Kan and Timmer, and to our previous static, synchronous parallel version of this method. The new algorithms have two main new features. First, they adaptively concentrate the computation in the areas of the domain space that appear most likely to produce the global minimum. Secondly, on parallel computers, they use an asynchronous approach, combined with a central work scheduler, to avoid load balancing problems. We investigate several mechanisms for deciding when and how to make the adaptive adjustments. We also describe both algorithmic and implementation considerations involved in constructing the parallel asynchronous algorithm. Computational tests on sequential and parallel computers show that the adaptive and asynchronous features of our new method can substantially reduce the number of function evaluations, and the execution time, required by previous stochastic methods to solve global optimization problems.

The classic randomized load balancing model is the so-called supermarket model, which describes a system in which customers arrive to a service center with n parallel servers according to a Poisson process with rate lamdba-n, where lamba is less than 1. Upon arrival, each customer samples d queues independently and uniformly at random before joining the shortest of those sampled. Customers are served according to a first-in first-out (FIFO) scheduling rule, and their service times are assumed to be mutually independent and exponentially distributed with unit mean mu = 1. Any ties that may occur are broken randomly. When d = 1, the model reduces to a system of n independent M/M/1 queues, for which it is a classical result that the stationary queue length distribution at a single queue is geometric with parameter lambda, and thus has an exponential decay rate. When d greater than or equal to 2, the model is not exactly solvable, but asymptotic results show that as n, the number of servers, goes to infinity, the limiting stationary distribution of a queue decays superexponentially. Moreover, the majority of this gain in performance is already obtained when d = 2. In particular, this shows that with just a slight increase in sampling cost, from d = 1 to d = 2, the performance is almost as good as in the case when all queues are sampled (that is, the Join-the-Shortest-Queue system where d = n). This phenomenon is referred to as the power of two choices, and this classic model is well studied.

The objectives of this project were to explore decomposition algorithms that solve optimization models under uncertainty. In order to accommodate a variety of future scenarios, our algorithms are designed to address large scale models. The main accomplishments of the project can be summarized as follows. 1) design and evaluate decomposition methods for stochastic mixed-integer programming (SMIP) problems (Yuan and Sen [2008]); 2) accelerate stochastic decomposition (SD) as a prelude to using SD for SMIP as well as a multi-stage version of SD (Sen et al [2007], Zhou and Sen [2008]); 3) develop a theory for parametric analysis of mixed-integer programs, and provide economically justifiable estimates of shadow prices from mixed-integer linear programming models (Sen and Genc [2008]). The first two relate to stochastic programming, whereas the last addresses one of the long-standing open questions in discrete optimization, namely, parametric analysis in MILP models. This paper (listed as [1]) is likely to have a long term impact on a variety of fields including discrete optimization, operations research, and computational economics.

Stochastic Mixed Integer Programs (SMIP) are recognized as one of the most formidable classes of mathematical programming problems. Not only are there significant challenges due to potentially large number of scenarios, but, SMIP with integers in the second stage give rise to a non-convex and discontinuous recourse function that may be difficult to optimize. As a result of this project, there have been significant advances in the design of algorithms for solving SMIP problems. Thanks to this project, we are able to report on solution to SMIP problems with over a million binary variables in less than 3 hours of computing on a desk-top machine! These problems arise in a variety of Air Force applications, such as adaptive command and control (AC2) under uncertainty. For instance, consider a situation in which assignments of pilots/aircrafts to targets may be contingent of sensor data revealed during the mission. In such situations, a preliminary set of assignments are made, recognizing that these will be revised once more reliable observations (regarding targets) are available. While such adaptive methods can enhance the effectiveness of C2, uncertainties can vastly enlarge the set of choices, and new algorithmic tools are necessary. Our algorithms solve such problems.

Biochemical reactions can happen on different time scales and also the abundance of species in these reactions can be very different from each other. Classical approaches, such as deterministic or stochastic approach, fail to account for or to exploit this multi-scale nature, respectively. In this paper, we propose a jump-diffusion approximation for multi-scale Markov jump processes that couples the two modeling approaches. An error bound of the proposed approximation is derived and used to partition the reactions into fast and slow sets, where the fast set is simulated by a stochastic differential equation and the slow set is modeled by a discrete chain. The error bound leads to a very efficient dynamic partitioning algorithm which has been implemented for several multi-scale reaction systems. The gain in computational efficiency is illustrated by a realistically sized model of a signal transduction cascade coupled to a gene expression dynamics.