Downloads & Free Reading Options - Results

Use of Chance-Constrained Programming to Account for Stochastic Variation in the A-Matrix of Large-Scale Linear Programs: A Forestry Application

Title from cover

Title from cover

In this paper, we study one kind of stochastic recursive optimal control problem with the obstacle constraints for the cost function where the cost function is described by the solution of one reflected backward stochastic differential equations. We will give the dynamic programming principle for this kind of optimal control problem and show that the value function is the unique viscosity solution of the obstacle problem for the corresponding Hamilton-Jacobi-Bellman equations.

In this paper, we study one kind of stochastic recursive optimal control problem with the obstacle constraints for the cost function where the cost function is described by the solution of one reflected backward stochastic differential equations. We will give the dynamic programming principle for this kind of optimal control problem and show that the value function is the unique viscosity solution of the obstacle problem for the corresponding Hamilton-Jacobi-Bellman equations.

This paper studies several stochastic programming formulations of dynamic oligopolistic games under uncertainty. We argue that one of the models, namely Games with Probabilistic Scenarios (GPS), provides an appropriate formulation. For such games, we show that symmetric players earn greater expected profits as demand volatility increases. This result suggests that even in an increasingly volatile market players may have an incentive to participate in the market. The key to our approach is the so-called scenario formulation of stochastic programming. In addition to several modeling insights, we also discuss the application of GPS to the electricity market in Ontario, Canada. The examples presented in this paper illustrate that this approach can address dynamic games that are clearly out of reach for dynamic programming, a common approach in the literature on dynamic games.

This paper studies several stochastic programming formulations of dynamic oligopolistic games under uncertainty. We argue that one of the models, namely Games with Probabilistic Scenarios (GPS), provides an appropriate formulation. For such games, we show that symmetric players earn greater expected profits as demand volatility increases. This result suggests that even in an increasingly volatile market players may have an incentive to participate in the market. The key to our approach is the so-called scenario formulation of stochastic programming. In addition to several modeling insights, we also discuss the application of GPS to the electricity market in Ontario, Canada. The examples presented in this paper illustrate that this approach can address dynamic games that are clearly out of reach for dynamic programming, a common approach in the literature on dynamic games.

The paper is a summary of some applications of the theory of dynamic programming to various classes of multi-stage decision problems of stochastic type.

The applicability of known stochastic programming models and methods for the solution of problems in classical statistics and probability is shown by a number of examples. These concern testing of hypotheses, constructing of tolerance regions, planning of optimal sampling and the Moran model for the dam. (Author)

To model combinatorial decision problems involving uncertainty and probability, we introduce scenario based stochastic constraint programming. Stochastic constraint programs contain both decision variables, which we can set, and stochastic variables, which follow a discrete probability distribution. We provide a semantics for stochastic constraint programs based on scenario trees. Using this semantics, we can compile stochastic constraint programs down into conventional (non-stochastic) constraint programs. This allows us to exploit the full power of existing constraint solvers. We have implemented this framework for decision making under uncertainty in stochastic OPL, a language which is based on the OPL constraint modelling language [Hentenryck et al., 1999]. To illustrate the potential of this framework, we model a wide range of problems in areas as diverse as portfolio diversification, agricultural planning and production/inventory management.

A question which combines statistics and linear programming considerations was first raised by G. Tintner (Econometrica 28:2, 490-5, April 60). It concerns the distribution of optimum functional values when a linear programming problem has probabilistic constraints. It is proposed to accord a chance constrained programming formulation to this kind of problem and to deal with it in a way that bears on project scheduling of the kind that is usually associated with critical path analysis, for instance, in PERT. The main focus of this paper is on the statistical distributions of the project completion (and subcompletion) times. The question of total time distributions that we deal with can therefore be given a managerial policy flavor by assuming that, ab initio, a management is considering a contract for a certain project. The task sequences are known but the times are not known except in probability. Before contracting for a target completion date--with resulting delay penalties--this management would like to know the likely distribution of total times in order to decide whether to accept an offered contract or else bargain further on the completion dates, penalty rates and progress payments and prices.

Mixed-Integer Programming has traditionally been restricted to deterministic models. Recent research has opened the door to stochastic optimization models, which are typically dynamic in nature. This project lays the foundation for stochastic dynamic mixed-integer and linear programming (SD-MIP). This project has produced several new ideas in connection with a) convexification of two-stage mixed-integer sets and b) multi-stage (including two-stage) stochastic linear programming. Together a) and b) provide the foundations for SD-MIP problems. From new concepts and algorithms to applications and software, this project has made significant breakthroughs in all aspects. This report provides a synopsis of both theoretical and computational results. As a preview, we mention that currently available deterministic MIP solvers, as powerful as they are known to be, are unable to solve SD-MIP models of modest size within an hour of computing. In contrast, our decomposition approach provides provably optimal solutions within the hour time-limit.

We propose a hybrid algorithmic strategy for complex stochastic optimization problems, which combines the use of scenario trees from multistage stochastic programming with machine learning techniques for learning a policy in the form of a statistical model, in the context of constrained vector-valued decisions. Such a policy allows one to run out-of-sample simulations over a large number of independent scenarios, and obtain a signal on the quality of the approximation scheme used to solve the multistage stochastic program. We propose to apply this fast simulation technique to choose the best tree from a set of scenario trees. A solution scheme is introduced, where several scenario trees with random branching structure are solved in parallel, and where the tree from which the best policy for the true problem could be learned is ultimately retained. Numerical tests show that excellent trade-offs can be achieved between run times and solution quality.

Fashion discounters face the problem of ordering the right amount of pieces in each size of a product. The product is ordered in pre-packs containing a certain size-mix of a product. For this so-called lot-type design problem, a stochastic mixed integer linear programm was developed, in which price cuts serve as recourse action for oversupply. Our goal is to answer the question, whether the resulting supply strategy leads to a supply that is significantly more consistent with the demand for sizes compared to the original manual planning. Since the total profit is influenced by too many factors unrelated to sizes (like the popularity of the product, the weather or a changing economic situation), we suggest a comparison method which excludes many outer effects by construction. We apply the method to a real-world field study: The improvements in the size distributions of the supply are significant.

We study Graver test sets for families of linear multi-stage stochastic integer programs with varying number of scenarios. We show that these test sets can be decomposed into finitely many ``building blocks'', independent of the number of scenarios, and we give an effective procedure to compute these building blocks. The paper includes an introduction to Nash-Williams' theory of better-quasi-orderings, which is used to show termination of our algorithm. We also apply this theory to finiteness results for Hilbert functions.

This thesis proposes new methods, based on dynamic programming, for solving certain single-stage and multi-stage integer stochastic knapsack problems. These problems model stochastic portfolio optimization problems (SPOPs) which assume deterministic unit weight, and normally distributed unit return with known mean and variance for each item type. Given an initial wealth, the objective is to select a portfolio that maximizes the probability of achieving or exceeding a specified final return threshold; the multi-stage problem allows revisions of the portfolio at regular time intervals. An exact method is developed to solve a single-stage SPOP with independence of returns among item types. For a problem from the literature with 11 item types, this method obtains an optimal solution in a fraction of a second on a laptop computer. An approximation method, based on discretization of possible wealth values, is developed to solve a multi-stage SPOP with inter- and intra-stage independence of returns among item types. Running on a desktop computer, this approximation method solves a 3-stage problem with 6 item types in under 12 minutes. With finer discretization in a 3-stage problem with 8 item types, the solution time is about 46 minutes.

The paper concerns the investigation of nonconvex and nondifferentiable integral functionals on general Banach spaces, which may not be reflexive and/or separable. Considering two major subdifferentials of variational analysis, we derive nonsmooth versions of the Leibniz rule on subdifferentiation under the integral sign, where the integral of the subdifferential set-valued mappings generated by Lipschitzian integrands is understood in the Gelfand sense. Besides examining integration over complete measure spaces and also over those with nonatomic measures, our special attention is drawn to a stronger version of measure nonatomicity, known as saturation, to invoke the recent results of the Lyapunov convexity theorem type for the Gelfand integral of the subdifferential mappings. The main results are applied to the subdifferential study of the optimal value functions and deriving the corresponding necessary optimality conditions in nonconvex problems of stochastic dynamic programming with discrete time on the infinite horizon.

A new decision-theoretic approach to Nonlinear Programming Problems with stochastic constraints is introduced. The Stochastic Program (SP) is replaced by a Deterministic Program (DP) in which a term is added to the objective function to penalize solutions which are not feasible in the mean. The special feature of the author's approach is the choice of the penalty function P sub E, which is given in terms if the relative entropy functional, and is accordingly called entropic penalty. It is shown that P sub E has properties which make it suitable to treat stochastic programs. Some of these properties are derived via a dual representation independent. The dual representation is also used to express the Deterministric Problem (DP) as a saddle function problem. For problems in which the randomness occurs in the rhs of the constraints, it shown that the dual problem of (DP) is equivalent to Expected Utility Maximization of the classical Lagrangian dual function of (SP), with the utility being of the constant-risk-aversion type. Finally, mean-variance approximations of P sub E and the induced Approximate Deterministic Program are considered.

A new decision-theoretic approach to Nonlinear Programming Problems with stochastic constraints is introduced. The Stochastic Program (SP) is replaced by a Deterministic Program (DP) in which a term is added to the objective function to penalize solutions which are not feasible in the mean. The special feature of the author's approach is the choice of the penalty function P sub E, which is given in terms if the relative entropy functional, and is accordingly called entropic penalty. It is shown that P sub E has properties which make it suitable to treat stochastic programs. Some of these properties are derived via a dual representation independent. The dual representation is also used to express the Deterministric Problem (DP) as a saddle function problem. For problems in which the randomness occurs in the rhs of the constraints, it shown that the dual problem of (DP) is equivalent to Expected Utility Maximization of the classical Lagrangian dual function of (SP), with the utility being of the constant-risk-aversion type. Finally, mean-variance approximations of P sub E and the induced Approximate Deterministic Program are considered.

We are concerned with the linear-quadratic optimal stochastic control problem with random coefficients. Under suitable conditions, we prove that the value field $V(t,x,\omega), (t,x,\omega)\in [0,T]\times R^n\times \Omega$, is quadratic in $x$, and has the following form: $V(t,x)=\langle K_tx, x\rangle$ where $K$ is an essentially bounded nonnegative symmetric matrix-valued adapted processes. Using the dynamic programming principle (DPP), we prove that $K$ is a continuous semi-martingale of the form $$K_t=K_0+\int_0^t \, dk_s+\sum_{i=1}^d\int_0^tL_s^i\, dW_s^i, \quad t\in [0,T]$$ with $k$ being a continuous process of bounded variation and $$E\left[\left(\int_0^T|L_s|^2\, ds\right)^p\right]

This research investigates a special class of stochastic linear programs known as two-stage stochastic linear programming with relatively complete and fixed recourse. These models characterize a two-phase process where the first- stage decision (itself subject to a separate set of first-stage linear constraints) allocates a set of resources to the second-stage linear program prior to the realization of random variables affecting second-stage resource availability. Since the second-stage decision deterministically follows both first-stage allocation and random variable realization, the first-stage variables constitute the only true decision. The expected cost of the two-stage recourse problem is also a piecewise convex function of the first-stage decision variables, thus allowing a global optimal solution that minimizes the total expected cost.

This research investigates a special class of stochastic linear programs known as two-stage stochastic linear programming with relatively complete and fixed recourse. These models characterize a two-phase process where the first- stage decision (itself subject to a separate set of first-stage linear constraints) allocates a set of resources to the second-stage linear program prior to the realization of random variables affecting second-stage resource availability. Since the second-stage decision deterministically follows both first-stage allocation and random variable realization, the first-stage variables constitute the only true decision. The expected cost of the two-stage recourse problem is also a piecewise convex function of the first-stage decision variables, thus allowing a global optimal solution that minimizes the total expected cost.

The energetic optimization problem, e.g., searching for the optimal switch- ing protocol of certain system parameters to minimize the input work, has been extensively studied by stochastic thermodynamics. In current work, we study this problem numerically with iterative dynamic programming. The model systems under investigation are toy actuators consisting of spring-linked beads with loading force imposed on both ending beads. For the simplest case, i.e., a one-spring actuator driven by tuning the stiffness of the spring, we compare the optimal control protocol of the stiffness for both the overdamped and the underdamped situations, and discuss how inertial effects alter the irreversibility of the driven process and thus modify the optimal protocol. Then, we study the systems with multiple degrees of freedom by constructing oligomer actuators, in which the harmonic interaction between the two ending beads is tuned externally. With the same rated output work, actuators of different constructions demand different minimal input work, reflecting the influence of the internal degrees of freedom on the performance of the actuators.

We prove the dynamic programming principe for uniformly nondegenerate stochastic differential games in the framework of time-homogeneous diffusion processes considered up to the first exit time from a domain. The zeroth-order "coefficient" and the "free" term are only assumed to be measurable. In contrast with previous results established for constant stopping times we allow arbitrary stopping times and randomized ones as well. The main assumption, which will be removed in a subsequent article, is that there exists a sufficiently regular solution of the Isaacs equation.

Motivated by applications to multi-antenna wireless networks, we propose a distributed and asynchronous algorithm for stochastic semidefinite programming. This algorithm is a stochastic approximation of a continous- time matrix exponential scheme regularized by the addition of an entropy-like term to the problem's objective function. We show that the resulting algorithm converges almost surely to an $\varepsilon$-approximation of the optimal solution requiring only an unbiased estimate of the gradient of the problem's stochastic objective. When applied to throughput maximization in wireless multiple-input and multiple-output (MIMO) systems, the proposed algorithm retains its convergence properties under a wide array of mobility impediments such as user update asynchronicities, random delays and/or ergodically changing channels. Our theoretical analysis is complemented by extensive numerical simulations which illustrate the robustness and scalability of the proposed method in realistic network conditions.

This paper formed part of a preliminary research report for a risk consultancy and academic research. Stochastic Programming models provide a powerful paradigm for decision making under uncertainty. In these models the uncertainties are represented by a discrete scenario tree and the quality of the solutions obtained is governed by the quality of the scenarios generated. We propose a new technique to generate scenarios based on Gaussian Mixture Hidden Markov Modelling. We show that our approach explicitly captures important time varying dynamics of stochastic processes (such as autoregression and jumps) as well as non-Gaussian distribution characteristics (such as skewness and kurtosis). Our scenario generation method enables richer robustness and scenario analysis through exploiting the tractable properties of Markov models and Gaussian mixture distributions. We demonstrate the benefits of our scenario generation method by conducting numerical experiments on FTSE-100 data.

To model combinatorial decision problems involving uncertainty and probability, we extend the stochastic constraint programming framework proposed in [Walsh, 2002] along a number of important dimensions (e.g. to multiple chance constraints and to a range of new objectives). We also provide a new (but equivalent) semantics based on scenarios. Using this semantics, we can compile stochastic constraint programs down into conventional (nonstochastic) constraint programs. This allows us to exploit the full power of existing constraint solvers. We have implemented this framework for decision making under uncertainty in stochastic OPL, a language which is based on the OPL constraint modelling language [Hentenryck et al., 1999]. To illustrate the potential of this framework, we model a wide range of problems in areas as diverse as finance, agriculture and production.

Bibliography: l. 66

Bibliography: l. 66

Bibliography: l. 66

The shortage of water resources and the increasing competition among water users have highlighted the importance of the water allocation problem. Water availability is crucial for water resource allocation and changes frequently, leading to the necessity to predict available water. This paper develops a framework aimed to plan regional water allocations under different representative concentration pathways (RCP) scenarios using an interval parameter conditional value-at-risk (CVaR) two-stage stochastic programming model. This framework combines prediction and optimization to reflect climate change, the uncertainty of water system and the coordination between water resources allocation and risk. The feasibility and practicality of the framework are demonstrated by its application in a real-world case study in the Lower Songhua River Basin in northeast China. Comparison between the results of the developed model and actual conditions show that 11.61 × 108 m3 volume of water supply can be saved after optimization, indicating that the developed model tends to allocate water in a more efficient way. The ratio of surface water to groundwater is reduced from 2:1 to 1.62:1. The proposed model has practical relevance for saving water and alleviating groundwater overexploitation. The approach is applicable to most areas with severe water shortages and groundwater overexploitation, and decision makers can determine the appropriate options for water resources allocation based on risk preferences and actual conditions.

The shortage of water resources and the increasing competition among water users have highlighted the importance of the water allocation problem. Water availability is crucial for water resource allocation and changes frequently, leading to the necessity to predict available water. This paper develops a framework aimed to plan regional water allocations under different representative concentration pathways (RCP) scenarios using an interval parameter conditional value-at-risk (CVaR) two-stage stochastic programming model. This framework combines prediction and optimization to reflect climate change, the uncertainty of water system and the coordination between water resources allocation and risk. The feasibility and practicality of the framework are demonstrated by its application in a real-world case study in the Lower Songhua River Basin in northeast China. Comparison between the results of the developed model and actual conditions show that 11.61 × 108 m3 volume of water supply can be saved after optimization, indicating that the developed model tends to allocate water in a more efficient way. The ratio of surface water to groundwater is reduced from 2:1 to 1.62:1. The proposed model has practical relevance for saving water and alleviating groundwater overexploitation. The approach is applicable to most areas with severe water shortages and groundwater overexploitation, and decision makers can determine the appropriate options for water resources allocation based on risk preferences and actual conditions.

We show that the variance of the Monte Carlo estimator that is importance sampled from an exponential family is a convex function of the natural parameter of the distribution. With this insight, we propose an adaptive importance sampling algorithm that simultaneously improves the choice of sampling distribution while accumulating a Monte Carlo estimate. Exploiting convexity, we prove that the method's unbiased estimator has variance that is asymptotically optimal over the exponential family.

We show that the variance of the Monte Carlo estimator that is importance sampled from an exponential family is a convex function of the natural parameter of the distribution. With this insight, we propose an adaptive importance sampling algorithm that simultaneously improves the choice of sampling distribution while accumulating a Monte Carlo estimate. Exploiting convexity, we prove that the method's unbiased estimator has variance that is asymptotically optimal over the exponential family.

In networks, there are often more than one source of capacity. The capacities can be permanently or temporarily owned by the decision maker. Depending on the nature of sources, we identify the permanent capacity, spot market capacity and contract capacity. We use a scenario tree to model the uncertainty, and build a multi-stage stochastic integer program that can incorporate multiple sources and multiple types of capacities in a general network. We propose two solution methodologies for the problem. Firstly, we design an asymptotically convergent approximation algorithm. Secondly, we design a cutting plane algorithm based on Benders decomposition to find tight bounds for the problem. The numerical experiments show superb performance of the proposed algorithms compared with commercial software.

We study the optimal control of general stochastic McKean-Vlasov equation. Such problem is motivated originally from the asymptotic formulation of cooperative equilibrium for a large population of particles (players) in mean-field interaction under common noise. Our first main result is to state a dynamic programming principle for the value function in the Wasserstein space of probability measures, which is proved from a flow property of the conditional law of the controlled state process. Next, by relying on the notion of differentiability with respect to probability measures due to P.L. Lions [32], and It{\^o}'s formula along a flow of conditional measures, we derive the dynamic programming Hamilton-Jacobi-Bellman equation, and prove the viscosity property together with a uniqueness result for the value function. Finally, we solve explicitly the linear-quadratic stochastic McKean-Vlasov control problem and give an application to an interbank systemic risk model with common noise.

In this paper, we study a stochastic recursive optimal control problem in which the cost functional is described by the solution of a backward stochastic differential equation driven by G-Brownian motion. Under standard assumptions, we establish the dynamic programming principle and the related fully nonlinear HJB equation in the framework of G-expectation. Finally, we show that the value function is the viscosity solution of the obtained HJB equation.

We study a combined optimal control/stopping problem under a nonlinear expectation ${\cal E}^f$ induced by a BSDE with jumps, in a Markovian framework. The terminal reward function is only supposed to be Borelian. The value function $u$ associated with this problem is generally irregular. We first establish a {\em sub- (resp. super-) optimality principle of dynamic programming} involving its {\em upper- (resp. lower-) semicontinuous envelope} $u^*$ (resp. $u_*$). This result, called {\em weak} dynamic programming principle (DPP), extends that obtained in \cite{BT} in the case of a classical expectation to the case of an ${\cal E}^f$-expectation and Borelian terminal reward function. Using this {\em weak} DPP, we then prove that $u^*$ (resp. $u_*$) is a {\em viscosity sub- (resp. super-) solution} of a nonlinear Hamilton-Jacobi-Bellman variational inequality.

We describe algorithms for two-stage stochastic linear programming with recourse and their implementation on a grid computing platform. In particular, we examine serial and asynchronous versions of the L-shaped method and a trust-region method. The parallel platform of choice is the dynamic, heterogeneous, opportunistic platform provided by the Condor system. The algorithms are of master-worker type (with the workers being used to solve second-stage problems, and the MW runtime support library (which supports master-worker computations) is key to the implementation. Computational results are presented on large sample average approximations of problems from the literature.

In this work we study the stochastic recursive control problem, in which the aggregator (or called generator) of the backward stochastic differential equation describing the running cost is continuous but not necessarily Lipschitz with respect to the first unknown variable and the control, and monotonic with respect to the first unknown variable. The dynamic programming principle and the connection between the value function and the viscosity solution of the associated Hamilton-Jacobi-Bellman equation are established in this setting by the generalized comparison theorem of backward stochastic differential equations and the stability of viscosity solutions. Finally we take the control problem of continuous-time Epstein-Zin utility with non-Lipschitz aggregator as an example to demonstrate the application of our study.

Multi-stage stochastic linear programs (MSLPs) are notoriously hard to solve in general. Linear decision rules (LDRs) yield an approximation of an MSLP by restricting the decisions at each stage to be an affine function of the observed uncertain parameters. Finding an optimal LDR is a static optimization problem that provides an upper bound on the optimal value of the MSLP, and, under certain assumptions, can be formulated as an explicit linear program. Similarly, as proposed by Kuhn, Wiesemann, and Georghiou (Math. Program., 130, 177-209, 2011) a lower bound for an MSLP can be obtained by restricting decisions in the dual of the MSLP to follow an LDR. We propose a new approximation approach for MSLPs, two-stage LDRs. The idea is to require only the state variables in an MSLP to follow an LDR, which is sufficient to obtain an approximation of an MSLP that is a two-stage stochastic linear program (2SLP). We similarly propose to apply LDR only to a subset of the variables in the dual of the MSLP, which yields a 2SLP approximation of the dual that provides a lower bound on the optimal value of the MSLP. Although solving the corresponding 2SLP approximations exactly is intractable in general, we investigate how approximate solution approaches that have been developed for solving 2SLP can be applied to solve these approximation problems, and derive statistical upper and lower bounds on the optimal value of the MSLP. In addition to potentially yielding better policies and bounds, this approach requires many fewer assumptions than are required to obtain an explicit reformulation when using the standard static LDR approach. As an illustrative example we apply our approach to a capacity expansion model, and find that the two-stage LDR policy has expected cost between 20% and 34% lower than the static LDR policy, and in the dual yields lower bounds that are between 0.1% and 3.3% better.

Multi-stage stochastic linear programs (MSLPs) are notoriously hard to solve in general. Linear decision rules (LDRs) yield an approximation of an MSLP by restricting the decisions at each stage to be an affine function of the observed uncertain parameters. Finding an optimal LDR is a static optimization problem that provides an upper bound on the optimal value of the MSLP, and, under certain assumptions, can be formulated as an explicit linear program. Similarly, as proposed by Kuhn, Wiesemann, and Georghiou (Math. Program., 130, 177-209, 2011) a lower bound for an MSLP can be obtained by restricting decisions in the dual of the MSLP to follow an LDR. We propose a new approximation approach for MSLPs, two-stage LDRs. The idea is to require only the state variables in an MSLP to follow an LDR, which is sufficient to obtain an approximation of an MSLP that is a two-stage stochastic linear program (2SLP). We similarly propose to apply LDR only to a subset of the variables in the dual of the MSLP, which yields a 2SLP approximation of the dual that provides a lower bound on the optimal value of the MSLP. Although solving the corresponding 2SLP approximations exactly is intractable in general, we investigate how approximate solution approaches that have been developed for solving 2SLP can be applied to solve these approximation problems, and derive statistical upper and lower bounds on the optimal value of the MSLP. In addition to potentially yielding better policies and bounds, this approach requires many fewer assumptions than are required to obtain an explicit reformulation when using the standard static LDR approach. As an illustrative example we apply our approach to a capacity expansion model, and find that the two-stage LDR policy has expected cost between 20% and 34% lower than the static LDR policy, and in the dual yields lower bounds that are between 0.1% and 3.3% better.

Multi-stage stochastic linear programs (MSLPs) are notoriously hard to solve in general. Linear decision rules (LDRs) yield an approximation of an MSLP by restricting the decisions at each stage to be an affine function of the observed uncertain parameters. Finding an optimal LDR is a static optimization problem that provides an upper bound on the optimal value of the MSLP, and, under certain assumptions, can be formulated as an explicit linear program. Similarly, as proposed by Kuhn, Wiesemann, and Georghiou (Math. Program., 130, 177-209, 2011) a lower bound for an MSLP can be obtained by restricting decisions in the dual of the MSLP to follow an LDR. We propose a new approximation approach for MSLPs, two-stage LDRs. The idea is to require only the state variables in an MSLP to follow an LDR, which is sufficient to obtain an approximation of an MSLP that is a two-stage stochastic linear program (2SLP). We similarly propose to apply LDR only to a subset of the variables in the dual of the MSLP, which yields a 2SLP approximation of the dual that provides a lower bound on the optimal value of the MSLP. Although solving the corresponding 2SLP approximations exactly is intractable in general, we investigate how approximate solution approaches that have been developed for solving 2SLP can be applied to solve these approximation problems, and derive statistical upper and lower bounds on the optimal value of the MSLP. In addition to potentially yielding better policies and bounds, this approach requires many fewer assumptions than are required to obtain an explicit reformulation when using the standard static LDR approach. As an illustrative example we apply our approach to a capacity expansion model, and find that the two-stage LDR policy has expected cost between 20% and 34% lower than the static LDR policy, and in the dual yields lower bounds that are between 0.1% and 3.3% better.

Few if any validated guidelines exist for making decisions about the design, media, or format of new instructional products. This study examined strings of programmed learning responses to create general guidelines for making such decisions. Using a Markov model, tables were developed relating the expected proportion of students to be in a solution state at a given accuracy level and at a given level of confidence with respect to the length of response strings.

Stochastic Constraint Programming (SCP) is an extension of Constraint Programming (CP) used for modelling and solving problems involving constraints and uncertainty. SCP inherits excellent modelling abilities and filtering algorithms from CP, but so far it has not been applied to large problems. Reinforcement Learning (RL) extends Dynamic Programming to large stochastic problems, but is problem-specific and has no generic solvers. We propose a hybrid combining the scalability of RL with the modelling and constraint filtering methods of CP. We implement a prototype in a CP system and demonstrate its usefulness on SCP problems.

We present a new primal-dual algorithm for computing the value of the Lagrangian dual of a stochastic mixed-integer program (SMIP) formed by relaxing its nonanticipativity constraints. This dual is widely used in decomposition methods for the solution of SMIPs. The algorithm relies on the well-known progressive hedging method, but unlike previous progressive hedging approaches for SMIP, our algorithm can be shown to converge to the optimal Lagrangian dual value. The key improvement in the new algorithm is an inner loop of optimized linearization steps, similar to those taken in the classical Frank-Wolfe method. Numerical results demonstrate that our new algorithm empirically outperforms the standard implementation of progressive hedging for obtaining bounds in SMIP.

Ariyawansa and Zhu [3] have proposed a new class of optimization problems termed stochastic semidefinite programs (SSDPs) to handle data uncertainty in applications leading to (deterministic) semidefinite programs (DSDPs). For the case where the event space of the random variables in an SSDP is finite they have also derived a class of volumetric barrier decomposition algorithms, and proved polynomial complexity of the short-step and long-step members of the class [2]. When the event space of the random variables in an SSDP is finite, the SSDP is equivalent to a large scale DSDP with special structure. Polynomial homogeneous self-dual algorithms [11] are an important class of algorithms that have been introduced for solving (general) DSDPs. It is therefore possible to solve SSDPs by applying homogeneous self-dual algorithms to their DSDP equivalents. However, such algorithms, while polynomial, will still have high computational complexities in comparison to decomposition algorithms. In this paper, we show how the special structure in DSDP equivalents of SSDPs can be exploited to design homogeneous self-dual algorithms with computation

This paper presents a method to approximately solve stochastic optimal control problems in which the cost function and the system dynamics are polynomial. For stochastic systems with polynomial dynamics, the moments of the state can be expressed as a, possibly infinite, system of deterministic linear ordinary differential equations. By casting the problem as a deterministic control problem in moment space, semidefinite programming is used to find a lower bound on the optimal solution. The constraints in the semidefinite program are imposed by the ordinary differential equations for moment dynamics and semidefiniteness of the outer product of moments. From the solution to the semidefinite program, an approximate optimal control strategy can be constructed using a least squares method. In the linear quadratic case, the method gives an exact solution to the optimal control problem. In more complex problems, an infinite number of moment differential equations would be required to compute the optimal control law. In this case, we give a procedure to increase the size of the semidefinite program, leading to increasingly accurate approximations to the true optimal control strategy.

We consider stochastic influence maximization problems arising in social networks. In contrast to existing studies that involve greedy approximation algorithms with a 63% performance guarantee, our work focuses on solving the problem optimally. To this end, we introduce a new class of problems that we refer to as two-stage stochastic submodular optimization models. We propose a delayed constraint generation algorithm to find the optimal solution to this class of problems with a finite number of samples. The influence maximization problems of interest are special cases of this general problem class. We show that the submodularity of the influence function can be exploited to develop strong optimality cuts that are more effective than the standard optimality cuts available in the literature. Finally, we report our computational experiments with large-scale real-world datasets for two fundamental influence maximization problems, independent cascade and linear threshold, and show that our proposed algorithm outperforms the greedy algorithm.

We consider stochastic influence maximization problems arising in social networks. In contrast to existing studies that involve greedy approximation algorithms with a 63% performance guarantee, our work focuses on solving the problem optimally. To this end, we introduce a new class of problems that we refer to as two-stage stochastic submodular optimization models. We propose a delayed constraint generation algorithm to find the optimal solution to this class of problems with a finite number of samples. The influence maximization problems of interest are special cases of this general problem class. We show that the submodularity of the influence function can be exploited to develop strong optimality cuts that are more effective than the standard optimality cuts available in the literature. Finally, we report our computational experiments with large-scale real-world datasets for two fundamental influence maximization problems, independent cascade and linear threshold, and show that our proposed algorithm outperforms the greedy algorithm.