Downloads & Free Reading Options - Results

We study nonlinear m-term approximation in a Banach space with regard to a basis. It is known that in the case of greedy basis (like the Haar basis H in L[sub p][0,1], 1 p infinity) a greedy type algorithm realizes near best m-term approximation for any individual function (element). In this paper we generalize the known result in two directions.

An elementary algorithm is used to simulate the industrial production of a fiber of a 3-dimensional nonwoven fabric. The algorithm simulates the fiber as a polyline where the direction of each segment is stochastically drawn based on a given probability density function (PDF) on the unit sphere. This PDF is obtained from data of directions of fiber fragments which originate from computer tomography scans of a real non-woven fabric. However, the simulation algorithm requires numerous evaluations of the PDF. Since the established technique of a kernel density estimator leads to very high computational costs, a novel greedy algorithm for estimating a sparse representation of the PDF is introduced. Numerical tests for a synthetic and a real example are presented. In a realistic scenario, the introduced sparsity ansatz leads to a reduction of the computation time for 100 fibers from nearly 40 days to 41 minutes.

The approximation of a discrete probability distribution $\mathbf{t}$ by an $M$-type distribution $\mathbf{p}$ is considered. The approximation error is measured by the informational divergence $\mathbb{D}(\mathbf{t}\Vert\mathbf{p})$, which is an appropriate measure, e.g., in the context of data compression. Properties of the optimal approximation are derived and bounds on the approximation error are presented, which are asymptotically tight. It is shown that $M$-type approximations that minimize either $\mathbb{D}(\mathbf{t}\Vert\mathbf{p})$, or $\mathbb{D}(\mathbf{p}\Vert\mathbf{t})$, or the variational distance $\Vert\mathbf{p}-\mathbf{t}\Vert_1$ can all be found by using specific instances of the same general greedy algorithm.

While greedy algorithms have long been observed to perform well on a wide variety of problems, up to now approximation ratios have only been known for their application to problems having submodular objective functions $f$. Since many practical problems have non-submodular $f$, there is a critical need to devise new techniques to bound the performance of greedy algorithms in the case of non-submodularity. Our primary contribution is the introduction of a novel technique for estimating the approximation ratio of the greedy algorithm for maximization of monotone non-decreasing functions based on the curvature of $f$ without relying on the submodularity constraint. We show that this technique reduces to the classical $(1 - 1/e)$ ratio for submodular functions. Furthermore, we develop an extension of this ratio to the adaptive greedy algorithm, which allows applications to non-submodular stochastic maximization problems. This notably extends support to applications modeling incomplete data with uncertainty. Finally we use this new technique to derive a $(1 - 1/\sqrt{e})$ ratio for a popular problem, Robust Influence Maximization, which is non-submodular and $1/2$ for Adaptive Max-Crawling, which is adaptive non-submodular.

We study sparse approximation by greedy algorithms. Our contribution is two-fold. First, we prove exact recovery with high probability of random $K$-sparse signals within $\lceil K(1+\e)\rceil$ iterations of the Orthogonal Matching Pursuit (OMP). This result shows that in a probabilistic sense the OMP is almost optimal for exact recovery. Second, we prove the Lebesgue-type inequalities for the Weak Chebyshev Greedy Algorithm, a generalization of the Weak Orthogonal Matching Pursuit to the case of a Banach space. The main novelty of these results is a Banach space setting instead of a Hilbert space setting. However, even in the case of a Hilbert space our results add some new elements to known results on the Lebesque-type inequalities for the RIP dictionaries. Our technique is a development of the recent technique created by Zhang.

Kernel-based methods provide flexible and accurate algorithms for the reconstruction of functions from meshless samples. A major question in the use of such methods is the influence of the samples locations on the behavior of the approximation, and feasible optimal strategies are not known for general problems. Nevertheless, efficient and greedy point-selection strategies are known. This paper gives a proof of the convergence rate of the data-independent \textit{$P$-greedy} algorithm, based on the application of the convergence theory for greedy algorithms in reduced basis methods. The resulting rate of convergence is shown to be near-optimal in the case of kernels generating Sobolev spaces. As a consequence, this convergence rate proves that, for kernels of Sobolev spaces, the points selected by the algorithm are asymptotically uniformly distributed, as conjectured in the paper where the algorithm has been introduced.

The stochastic matching problem was first introduced by Chen, Immorlica, Karlin, Mahdian, and Rudra (ICALP 2009). They presented greedy algorithm together with an analysis showing that this is a 4-approximation. They also presented modification of this problem called multiple-rounds matching, and gave O(log n)-approximation algorithm. Many questions were remaining after this work: is the greedy algorithm a 2-approximation, is there a constant-ratio algorithm for multiple-rounds matching, and what about weighted graphs? For the last two problems constant-factor approximations were given in the work of Bansal, Gupta, Nagarajan, and Rudra, and in the work of Li and Mestre. In this paper we are answering to the first question by showing that the greedy algorithm is in fact a 2-approximation.

We prove our lower estimate for the rate of convergence of Pure Greedy Algorithm with regard to a general dictionary and another lower estimate for the rate of convergence of Weak Greedy Algorithm with a special weakness sequence pie = {t}, 0 , t 1, with regard to a general dictionary. The second lower estimate combined with the known upper estimate gives the right (in the sense of order) dependence of the exponent in the rate of convergence on the parameter t when t yield 0.

We consider the approximation capability of orthogonal super greedy algorithms (OSGA) and its applications in supervised learning. OSGA is concerned with selecting more than one atoms in each iteration step, which, of course, greatly reduces the computational burden when compared with the conventional orthogonal greedy algorithm (OGA). We prove that even for function classes that are not the convex hull of the dictionary, OSGA does not degrade the approximation capability of OGA provided the dictionary is incoherent. Based on this, we deduce a tight generalization error bound for OSGA learning. Our results show that in the realm of supervised learning, OSGA provides a possibility to further reduce the computational burden of OGA in the premise of maintaining its prominent generalization capability.

We investigate the convergence of a nonlinear approximation method introduced by Ammar et al. (J. Non-Newtonian Fluid Mech. 139:153-176, 2006) for the numerical solution of high-dimensional Fokker-Planck equations featuring in Navier-Stokes-Fokker-Planck systems that arise in kinetic models of dilute polymers. In the case of Poisson's equation on a rectangular domain in R^2, subject to a homogeneous Dirichlet boundary condition, the mathematical analysis of the algorithm was carried out recently by Le Bris, Leli\`evre and Maday (Const. Approx. 30:621-651, 2009), by exploiting its connection to greedy algorithms from nonlinear approximation theory, explored, for example, by DeVore and Temlyakov (Adv. Comput. Math. 5:173-187, 1996); hence, the variational version of the algorithm, based on the minimization of a sequence of Dirichlet energies, was shown to converge. Here, we extend the convergence analysis of the pure greedy and orthogonal greedy algorithms considered by Le Bris et al. to a technically more complicated situation, where the Laplace operator is replaced by an Ornstein-Uhlenbeck operator of the kind that appears in Fokker-Planck equations that arise in bead-spring chain type kinetic polymer models with finitely extensible nonlinear elastic potentials, posed on a high-dimensional Cartesian product configuration space D = D_1 x ... x D_N contained in R^(N d), where each set D_i, i = 1, ..., N, is a bounded open ball in R^d, d = 2, 3.

For an edge-weighted connected undirected graph, the minimum $k$-way cut problem is to find a subset of edges of minimum total weight whose removal separates the graph into $k$ connected components. The problem is NP-hard when $k$ is part of the input and W[1]-hard when $k$ is taken as a parameter. A simple algorithm for approximating a minimum $k$-way cut is to iteratively increase the number of components of the graph by $h-1$, where $2 \le h \le k$, until the graph has $k$ components. The approximation ratio of this algorithm is known for $h \le 3$ but is open for $h \ge 4$. In this paper, we consider a general algorithm that iteratively increases the number of components of the graph by $h_i-1$, where $h_1 \le h_2 \le ... \le h_q$ and $\sum_{i=1}^q (h_i-1) = k-1$. We prove that the approximation ratio of this general algorithm is $2 - (\sum_{i=1}^q {h_i \choose 2})/{k \choose 2}$, which is tight. Our result implies that the approximation ratio of the simple algorithm is $2-h/k + O(h^2/k^2)$ in general and $2-h/k$ if $k-1$ is a multiple of $h-1$.

For an edge-weighted connected undirected graph, the minimum $k$-way cut problem is to find a subset of edges of minimum total weight whose removal separates the graph into $k$ connected components. The problem is NP-hard when $k$ is part of the input and W[1]-hard when $k$ is taken as a parameter. A simple algorithm for approximating a minimum $k$-way cut is to iteratively increase the number of components of the graph by $h-1$, where $2 \le h \le k$, until the graph has $k$ components. The approximation ratio of this algorithm is known for $h \le 3$ but is open for $h \ge 4$. In this paper, we consider a general algorithm that iteratively increases the number of components of the graph by $h_i-1$, where $h_1 \le h_2 \le ... \le h_q$ and $\sum_{i=1}^q (h_i-1) = k-1$. We prove that the approximation ratio of this general algorithm is $2 - (\sum_{i=1}^q {h_i \choose 2})/{k \choose 2}$, which is tight. Our result implies that the approximation ratio of the simple algorithm is $2-h/k + O(h^2/k^2)$ in general and $2-h/k$ if $k-1$ is a multiple of $h-1$.

We study sparse approximation by greedy algorithms. We prove the Lebesgue-type inequalities for the Weak Chebyshev Greedy Algorithm (WCGA), a generalization of the Weak Orthogonal Matching Pursuit to the case of a Banach space. The main novelty of these results is a Banach space setting instead of a Hilbert space setting. The results are proved for redundant dictionaries satisfying certain conditions. Then we apply these general results to the case of bases. In particular, we prove that the WCGA provides almost optimal sparse approximation for the trigonometric system in $L_p$, $2\le p

We consider the problem of approximating a given element $f$ from a Hilbert space $\mathcal{H}$ by means of greedy algorithms and the application of such procedures to the regression problem in statistical learning theory. We improve on the existing theory of convergence rates for both the orthogonal greedy algorithm and the relaxed greedy algorithm, as well as for the forward stepwise projection algorithm. For all these algorithms, we prove convergence results for a variety of function classes and not simply those that are related to the convex hull of the dictionary. We then show how these bounds for convergence rates lead to a new theory for the performance of greedy algorithms in learning. In particular, we build upon the results in [IEEE Trans. Inform. Theory 42 (1996) 2118--2132] to construct learning algorithms based on greedy approximations which are universally consistent and provide provable convergence rates for large classes of functions. The use of greedy algorithms in the context of learning is very appealing since it greatly reduces the computational burden when compared with standard model selection using general dictionaries.

We study the problem of selecting a subset of k random variables from a large set, in order to obtain the best linear prediction of another variable of interest. This problem can be viewed in the context of both feature selection and sparse approximation. We analyze the performance of widely used greedy heuristics, using insights from the maximization of submodular functions and spectral analysis. We introduce the submodularity ratio as a key quantity to help understand why greedy algorithms perform well even when the variables are highly correlated. Using our techniques, we obtain the strongest known approximation guarantees for this problem, both in terms of the submodularity ratio and the smallest k-sparse eigenvalue of the covariance matrix. We further demonstrate the wide applicability of our techniques by analyzing greedy algorithms for the dictionary selection problem, and significantly improve the previously known guarantees. Our theoretical analysis is complemented by experiments on real-world and synthetic data sets; the experiments show that the submodularity ratio is a stronger predictor of the performance of greedy algorithms than other spectral parameters.

This paper deals with nonlinear approximation in Banach spaces.

A major enterprise in compressed sensing and sparse approximation is the design and analysis of computationally tractable algorithms for recovering sparse, exact or approximate, solutions of underdetermined linear systems of equations. Many such algorithms have now been proven to have optimal-order uniform recovery guarantees using the ubiquitous Restricted Isometry Property (RIP). However, it is unclear when the RIP-based sufficient conditions on the algorithm are satisfied. We present a framework in which this task can be achieved; translating these conditions for Gaussian measurement matrices into requirements on the signal's sparsity level, length, and number of measurements. We illustrate this approach on three of the state-of-the-art greedy algorithms: CoSaMP, Subspace Pursuit (SP), and Iterative Hard Thresholding (IHT). Designed to allow a direct comparison of existing theory, our framework implies that, according to the best known bounds, IHT requires the fewest number of compressed sensing measurements and has the lowest per iteration computational cost of the three algorithms compared here.

In this paper we study nonlinear approximation. The basic idea behind nonlinear approximation is that the elements used in the approximation do not come from a fixed linear space but are allowed to depend on the function being approximated. The classical problem in this regard is the problem of m-term approximation where one fixed a basis in the space, and seeks to approximate a target function f by a linear combination of m terms from that basis. When the basis is a wavelet basis or a basis of other waveforms, then this type of approximation is the starting point for compression algorithms.

This paper describes a simple greedy D-approximation algorithm for any covering problem whose objective function is submodular and non-decreasing, and whose feasible region can be expressed as the intersection of arbitrary (closed upwards) covering constraints, each of which constrains at most D variables of the problem. (A simple example is Vertex Cover, with D = 2.) The algorithm generalizes previous approximation algorithms for fundamental covering problems and online paging and caching problems.

This paper is a survey which also contains some new results on the nonlinear approximation with regard to a basis or, more generally, with regard to a minimal system. Approximation takes place in a Banach or in a quasi-Banach space. The last decade was very successful in studying nonlinear approximation. This was motivated by numerous applications. Nonlinear approximation is important in applications because of its increased efficiency. Two types of nonlinear approximation are employed frequently in applications. Adaptive methods are used in PDE solvers. The m-term approximation considered here is used in image and signal processing as well as the design of neural networks. The basic idea behind nonlinear approximation is that the elements used in the approximation do not come from a fixed linear space but are allowed to depend on the function being approximated. The fundamental question of nonlinear approximation is how to construct good methods (algorithms) of nonlinear approximation. In this paper we discuss greedy type and thresholding type algorithms.

We provide new approximation guarantees for greedy low rank matrix estimation under standard assumptions of restricted strong convexity and smoothness. Our novel analysis also uncovers previously unknown connections between the low rank estimation and combinatorial optimization, so much so that our bounds are reminiscent of corresponding approximation bounds in submodular maximization. Additionally, we also provide statistical recovery guarantees. Finally, we present empirical comparison of greedy estimation with established baselines on two important real-world problems.

It is a survey on recent results in constructive sparse approximation. Three directions are discussed here: (1) Lebesgue-type inequalities for greedy algorithms with respect to a special class of dictionaries, (2) constructive sparse approximation with respect to the trigonometric system, (3) sparse approximation with respect to dictionaries with tensor product structure. In all three cases constructive ways are provided for sparse approximation. The technique used is based on fundamental results from the theory of greedy approximation. In particular, results in the direction (1) are based on deep methods developed recently in compressed sensing. We present some of these results with detailed proofs.

We analyze the inverse problem of identifying the diffusivity coefficient of a scalar elliptic equation as a function of the resolvent operator. We prove that, within the class of measurable coefficients, bounded above and below by positive constants, the resolvent determines the diffusivity in an unique manner. Furthermore we prove that the inverse mapping from resolvent to the coefficient is Lipschitz in suitable topologies. This result plays a key role when applying greedy algorithms to the approximation of parameter-dependent elliptic problems in an uniform and robust manner, independent of the given source terms. In one space dimension the results can be improved using the explicit expression of solutions, which allows to link distances between one resolvent and a linear combination of finitely many others and the corresponding distances on coefficients. These results are also extended to multi-dimensional elliptic equations with variable density coefficients. We also point out towards some possible extensions and open problems.

We analyze the inverse problem of identifying the diffusivity coefficient of a scalar elliptic equation as a function of the resolvent operator. We prove that, within the class of measurable coefficients, bounded above and below by positive constants, the resolvent determines the diffusivity in an unique manner. Furthermore we prove that the inverse mapping from resolvent to the coefficient is Lipschitz in suitable topologies. This result plays a key role when applying greedy algorithms to the approximation of parameter-dependent elliptic problems in an uniform and robust manner, independent of the given source terms. In one space dimension the results can be improved using the explicit expression of solutions, which allows to link distances between one resolvent and a linear combination of finitely many others and the corresponding distances on coefficients. These results are also extended to multi-dimensional elliptic equations with variable density coefficients. We also point out towards some possible extensions and open problems.

The maximum genus $\gamma_M(G)$ of a graph G is the largest genus of an orientable surface into which G has a cellular embedding. Combinatorially, it coincides with the maximum number of disjoint pairs of adjacent edges of G whose removal results in a connected spanning subgraph of G. In this paper we prove that removing pairs of adjacent edges from G arbitrarily while retaining connectedness leads to at least $\gamma_M(G)/2$ pairs of edges removed. This allows us to describe a greedy algorithm for the maximum genus of a graph; our algorithm returns an integer k such that $\gamma_M(G)/2\le k \le \gamma_M(G)$, providing a simple method to efficiently approximate maximum genus. As a consequence of our approach we obtain a 2-approximate counterpart of Xuong's combinatorial characterisation of maximum genus.

We study Matching and other related problems in a partial information setting where the agents' utilities for being matched to other agents are hidden and the mechanism only has access to ordinal preference information. Our model is motivated by the fact that in many settings, agents cannot express the numerical values of their utility for different outcomes, but are still able to rank the outcomes in their order of preference. Specifically, we study problems where the ground truth exists in the form of a weighted graph, and look to design algorithms that approximate the true optimum matching using only the preference orderings for each agent (induced by the hidden weights) as input. If no restrictions are placed on the weights, then one cannot hope to do better than the simple greedy algorithm, which yields a half optimal matching. Perhaps surprisingly, we show that by imposing a little structure on the weights, we can improve upon the trivial algorithm significantly: we design a 1.6-approximation algorithm for instances where the hidden weights obey the metric inequality. Using our algorithms for matching as a black-box, we also design new approximation algorithms for other closely related problems: these include a a 3.2-approximation for the problem of clustering agents into equal sized partitions, a 4-approximation algorithm for Densest k-subgraph, and a 2.14-approximation algorithm for Max TSP. These results are the first non-trivial ordinal approximation algorithms for such problems, and indicate that we can design robust algorithms even when we are agnostic to the precise agent utilities.

We prove that the approximation ratio of the greedy algorithm for the metric Traveling Salesman Problem is $\Theta(\log n)$. Moreover, we prove that the same result also holds for graphic, Euclidean, and rectilinear instances of the Traveling Salesman Problem. Finally we show that the approximation ratio of the Clarke-Wright savings heuristic for the metric Traveling Salesman Problem is $\Theta(\log n)$.