Downloads & Free Reading Options - Results

This report develops fast algorithms for computing filters for linear least squares estimation of one, two, and three dimensional random fields. The algorithms generalize the split Levinson and Schur algorithms to two and three dimensions; however, they are applicable to a more general Toeplitz plus Hankel structure in the covariance function. A discrete version of the Bellman Siegert Krein resolvent identity is developed for smoothing problems in one and two dimensions. Applications to linear predictive coding, and restoration and smoothing, of isotropic random fields on a polar raster are demonstrated. In addition, two new algorithms are developed for spectral estimation on a two- dimensional polar raster. Both use the Radon transform to map the two dimensional problem into one dimensional problems. Interpolating functions for computing the Radon transform, positive definite covariance extensions, and correlation matching are all considered.

This report develops fast algorithms for computing filters for linear least squares estimation of one, two, and three dimensional random fields. The algorithms generalize the split Levinson and Schur algorithms to two and three dimensions; however, they are applicable to a more general Toeplitz plus Hankel structure in the covariance function. A discrete version of the Bellman Siegert Krein resolvent identity is developed for smoothing problems in one and two dimensions. Applications to linear predictive coding, and restoration and smoothing, of isotropic random fields on a polar raster are demonstrated. In addition, two new algorithms are developed for spectral estimation on a two- dimensional polar raster. Both use the Radon transform to map the two dimensional problem into one dimensional problems. Interpolating functions for computing the Radon transform, positive definite covariance extensions, and correlation matching are all considered.

In this talk I describe MAGIC, an efficient approach to covariance estimation and signal reconstruction for Gaussian random fields (MAGIC Allows Global Inference of Covariance). It solves a long-standing problem in the field of cosmic microwave background (CMB) data analysis but is in fact a general technique that can be applied to noisy, contaminated and incomplete or censored measurements of either spatial or temporal Gaussian random fields. In this talk I will phrase the method in a way that emphasizes its general structure and applicability but I comment on applications in the CMB context. The method allows the exploration of the full non-Gaussian joint posterior density of the signal and parameters in the covariance matrix (such as the power spectrum) given the data. It generalizes the familiar Wiener filter in that it automatically discovers signal correlations in the data as long as a noise model is specified and priors encode what is known about potential contaminants. The key methodological difference is that instead of attempting to evaluate the likelihood (or posterior density) or its derivatives, this method generates an asymptotically exact Monte Carlo sample from it. I present example applications to power spectrum estimation and signal reconstruction from measurements of the CMB. For these applications the method achieves speed-ups of many orders of magnitude compared to likelihood maximization techniques, while offering greater flexibility in modeling and a full characterization of the uncertainty in the estimates.

This document presents two deterministic algorithms for the maximum a posteriori estimation of a one dimensional, binary Markov random field from noisy observations. Extensions to other related problems, such as one dimensional signal matching, and estimation of continuous valued Markov random fields are also discussed. Finally, the author presents an experimental comparison of the performance of optimal algorithms with a stochastic approximation scheme (simulated annealing). Additional keywords: Mathematical models, Dynamic programming, Gaussian noise, White noise, Army research. (Author)

This paper considers the problem of estimating a two-dimensional isotropic random field given some noisy observations of this field over a disk of finite radius. By expanding the field and observations in Fourier series, and exploiting the covariance structure of the resulting Fourier coefficient processes, some vibrating equations are obtained for estimating the random field. These equations provide a set of recursions for constructing the field estimates as the radius of the observation disk increases. In the spectral domain, these recursions take the form of Schrodinger equations which can be viewed as being associated to an inverse scattering problem. (Author)

We provide new methods for estimation of the one-point specification probabilities in general discrete random fields. Our procedures are based on model selection by minimization of a penalized empirical criterion. The selected estimators satisfy sharp oracle inequalities without any assumption on the random field for both $L_{2}$-risk and K\"ullback loss. We also prove the validity of slope heuristic for the specification probabilities estimation problem. We finally show in simulation studies the practical performances of our methods.

Gibbs random fields play an important role in statistics. However they are complicated to work with due to an intractability of the likelihood function and there has been much work devoted to finding computational algorithms to allow Bayesian inference to be conducted for such so-called doubly intractable distributions. This paper extends this work and addresses the issue of estimating the evidence and Bayes factor for such models. The approach which we develop is shown to yield good performance.

The aim of this paper is to propose diffusion strategies for distributed estimation over adaptive networks, assuming the presence of spatially correlated measurements distributed according to a Gaussian Markov random field (GMRF) model. The proposed methods incorporate prior information about the statistical dependency among observations, while at the same time processing data in real-time and in a fully decentralized manner. A detailed mean-square analysis is carried out in order to prove stability and evaluate the steady-state performance of the proposed strategies. Finally, we also illustrate how the proposed techniques can be easily extended in order to incorporate thresholding operators for sparsity recovery applications. Numerical results show the potential advantages of using such techniques for distributed learning in adaptive networks deployed over GMRF.

The Potts model is frequently used to describe the behavior of image classes, since it allows to incorporate contextual information linking neighboring pixels in a simple way. Its isotropic version has only one real parameter beta, known as smoothness parameter or inverse temperature, which regulates the classes map homogeneity. The classes are unavailable, and estimating them is central in important image processing procedures as, for instance, image classification. Methods for estimating the classes which stem from a Bayesian approach under the Potts model require to adequately specify a value for beta. The estimation of such parameter can be efficiently made solving the Pseudo Maximum likelihood (PML) equations in two different schemes, using the prior or the posterior model. Having only radiometric data available, the first scheme needs the computation of an initial segmentation, while the second uses both the segmentation and the radiometric data to make the estimation. In this paper, we compare these two PML estimators by computing the mean square error (MSE), bias, and sensitivity to deviations from the hypothesis of the model. We conclude that the use of extra data does not improve the accuracy of the PML, moreover, under gross deviations from the model, this extra information introduces unpredictable distortions and bias.

This study considers the 3D human pose estimation problem in a single RGB image by proposing a conditional random field (CRF) model over 2D poses, in which the 3D pose is obtained as a byproduct of the inference process. The unary term of the proposed CRF model is defined based on a powerful heat-map regression network, which has been proposed for 2D human pose estimation. This study also presents a regression network for lifting the 2D pose to 3D pose and proposes the prior term based on the consistency between the estimated 3D pose and the 2D pose. To obtain the approximate solution of the proposed CRF model, the N-best strategy is adopted. The proposed inference algorithm can be viewed as sequential processes of bottom-up generation of 2D and 3D pose proposals from the input 2D image based on deep networks and top-down verification of such proposals by checking their consistencies. To evaluate the proposed method, we use two large-scale datasets: Human3.6M and HumanEva. Experimental results show that the proposed method achieves the state-of-the-art 3D human pose estimation performance.

We consider random fields defined by finite-region conditional probabilities depending on a neighborhood of the region which changes with the boundary conditions. To predict the symbols within any finite region it is necessary to inspect a random number of neighborhood symbols which might change according to the value of them. In analogy to the one dimensional setting we call these neighborhood symbols the context of the region. This framework is a natural extension, to d-dimensional fields, of the notion of variable-length Markov chains introduced by Rissanen (1983) in his classical paper. We define an algorithm to estimate the radius of the smallest ball containing the context based on a realization of the field. We prove the consistency of this estimator. Our proofs are constructive and yield explicit upper bounds for the probability of wrong estimation of the radius of the context.

We consider random fields defined by finite-region conditional probabilities depending on a neighborhood of the region which changes with the boundary conditions. To predict the symbols within any finite region it is necessary to inspect a random number of neighborhood symbols which might change according to the value of them. In analogy to the one dimensional setting we call these neighborhood symbols the context of the region. This framework is a natural extension, to d-dimensional fields, of the notion of variable-length Markov chains introduced by Rissanen (1983) in his classical paper. We define an algorithm to estimate the radius of the smallest ball containing the context based on a realization of the field. We prove the consistency of this estimator. Our proofs are constructive and yield explicit upper bounds for the probability of wrong estimation of the radius of the context.

Gaussian random fields are a powerful tool for modeling environmental processes. For high dimensional samples, classical approaches for estimating the covariance parameters require highly challenging and massive computations, such as the evaluation of the Cholesky factorization or solving linear systems. Recently, Anitescu, Chen and Stein \cite{M.Anitescu} proposed a fast and scalable algorithm which does not need such burdensome computations. The main focus of this article is to study the asymptotic behavior of the algorithm of Anitescu et al. (ACS) for regular and irregular grids in the increasing domain setting. Consistency, minimax optimality and asymptotic normality of this algorithm are proved under mild differentiability conditions on the covariance function. Despite the fact that ACS's method entails a non-concave maximization, our results hold for any stationary point of the objective function. A numerical study is presented to evaluate the efficiency of this algorithm for large data sets.

Gaussian random fields are a powerful tool for modeling environmental processes. For high dimensional samples, classical approaches for estimating the covariance parameters require highly challenging and massive computations, such as the evaluation of the Cholesky factorization or solving linear systems. Recently, Anitescu, Chen and Stein \cite{M.Anitescu} proposed a fast and scalable algorithm which does not need such burdensome computations. The main focus of this article is to study the asymptotic behavior of the algorithm of Anitescu et al. (ACS) for regular and irregular grids in the increasing domain setting. Consistency, minimax optimality and asymptotic normality of this algorithm are proved under mild differentiability conditions on the covariance function. Despite the fact that ACS's method entails a non-concave maximization, our results hold for any stationary point of the objective function. A numerical study is presented to evaluate the efficiency of this algorithm for large data sets.

Gaussian random fields are a powerful tool for modeling environmental processes. For high dimensional samples, classical approaches for estimating the covariance parameters require highly challenging and massive computations, such as the evaluation of the Cholesky factorization or solving linear systems. Recently, Anitescu, Chen and Stein \cite{M.Anitescu} proposed a fast and scalable algorithm which does not need such burdensome computations. The main focus of this article is to study the asymptotic behavior of the algorithm of Anitescu et al. (ACS) for regular and irregular grids in the increasing domain setting. Consistency, minimax optimality and asymptotic normality of this algorithm are proved under mild differentiability conditions on the covariance function. Despite the fact that ACS's method entails a non-concave maximization, our results hold for any stationary point of the objective function. A numerical study is presented to evaluate the efficiency of this algorithm for large data sets.

We consider the estimation of high-dimensional network structures from partially observed Markov random field data using a penalized pseudo-likelihood approach. We fit a misspecified model obtained by ignoring the missing data problem. We study the consistency of the estimator and derive a bound on its rate of convergence. The results obtained relate the rate of convergence of the estimator to the extent of the missing data problem. We report some simulation results that empirically validate some of the theoretical findings.

We investigate the nonparametric estimation for regression in a fixed-design setting when the errors are given by a field of dependent random variables. Sufficient conditions for kernel estimators to converge uniformly are obtained. These estimators can attain the optimal rates of uniform convergence and the results apply to a large class of random fields which contains martingale-difference random fields and mixing random fields.

We present a general M-estimation framework for inference on the wavelet variance. This framework generalizes the results on the scale-wise properties of the standard estimator and extends them to deliver the joint asymptotic properties of the estimated wavelet variance vector. Moreover, this is achieved by extending the estimation of the wavelet variance to multidimensional random fields and by stating the necessary conditions for these properties to hold when the size of the wavelet variance vector goes to infinity with the sample size. Finally, these results generally hold when using bounded estimating functions thereby delivering a robust framework for the estimation of this quantity which improves over existing methods both in terms of asymptotic properties and in terms of its finite sample performance. The proposed estimator is investigated in simulation studies and different applications highlighting its good properties.

We consider the problem of interaction neighborhood estimation from the partial observation of a finite number of realizations of a random field. We introduce a model selection rule to choose estimators of conditional probabilities among natural candidates. Our main result is an oracle inequality satisfied by the resulting estimator. We use then this selection rule in a two-step procedure to evaluate the interacting neighborhoods. The selection rule selects a small prior set of possible interacting points and a cutting step remove from this prior set the irrelevant points. We also prove that the Ising models satisfy the assumptions of the main theorems, without restrictions on the temperature, on the structure of the interacting graph or on the range of the interactions. It provides therefore a large class of applications for our results. We give a computationally efficient procedure in these models. We finally show the practical efficiency of our approach in a simulation study.

We study in this paper the sufficient conditions for enhanced continuity of random fields, i.e. such that the modulus of its continuity allows the factorable representation by the product of random variable on the deterministic module of continuity. We estimate also the ordinary and (possible) exponential moments of these random variables. We consider also the case of random fields with heavy tails of distribution and the so-called rectangle its continuity.

In this paper, under natural and easily verifiable conditions, we prove the $\mathbb{L}^1$-convergence and the asymptotic normality of the Parzen-Rosenblatt density estimator for stationary random fields of the form $X_k = g(\varepsilon_{k-s}, s \in \Z^d)$, $k\in\Z^d$, where $(\varepsilon_i)_{i\in\Z^d}$ are i.i.d real random variables and $g$ is a measurable function defined on $\R^{\Z^d}$. Such kind of processes provides a general framework for stationary ergodic random fields. A Berry-Esseen's type central limit theorem is also given for the considered estimator.

The paper deals with the basic integral equation of random field estimation theory by the criterion of minimum of variance of the error estimate. This integral equation is of the first kind. The corresponding integra$ operator over a bounded domain $\Omega $ in ${\Bbb R}^{n}$ is weakly singular. This operator is an isomorphism between appropriate Sobolev spaces. This is proved by a reduction of the integral equ$ an elliptic boundary value problem in the domain exterior to $\Omega .$ Extra difficulties arise due to the fact that the exterior boundary value problem should be solved in the Sobolev spaces of negative order.

The paper deals with the basic integral equation of random field estimation theory by the criterion of minimum of variance of the error estimate. This integral equation is of the first kind. The corresponding integra$ operator over a bounded domain $\Omega $ in ${\Bbb R}^{n}$ is weakly singular. This operator is an isomorphism between appropriate Sobolev spaces. This is proved by a reduction of the integral equ$ an elliptic boundary value problem in the domain exterior to $\Omega .$ Extra difficulties arise due to the fact that the exterior boundary value problem should be solved in the Sobolev spaces of negative order.

The paper deals with the basic integral equation of random field estimation theory by the criterion of minimum of variance of the error estimate. This integral equation is of the first kind. The corresponding integra$ operator over a bounded domain $\Omega $ in ${\Bbb R}^{n}$ is weakly singular. This operator is an isomorphism between appropriate Sobolev spaces. This is proved by a reduction of the integral equ$ an elliptic boundary value problem in the domain exterior to $\Omega .$ Extra difficulties arise due to the fact that the exterior boundary value problem should be solved in the Sobolev spaces of negative order.

In this paper we present a novel slanted-plane MRF model which reasons jointly about occlusion boundaries as well as depth. We formulate the problem as the one of inference in a hybrid MRF composed of both continuous (i.e., slanted 3D planes) and discrete (i.e., occlusion boundaries) random variables. This allows us to define potentials encoding the ownership of the pixels that compose the boundary between segments, as well as potentials encoding which junctions are physically possible. Our approach outperforms the state-of-the-art on Middlebury high resolution imagery as well as in the more challenging KITTI dataset, while being more efficient than existing slanted plane MRF-based methods, taking on average 2 minutes to perform inference on high resolution imagery.

In this paper we present a novel slanted-plane MRF model which reasons jointly about occlusion boundaries as well as depth. We formulate the problem as the one of inference in a hybrid MRF composed of both continuous (i.e., slanted 3D planes) and discrete (i.e., occlusion boundaries) random variables. This allows us to define potentials encoding the ownership of the pixels that compose the boundary between segments, as well as potentials encoding which junctions are physically possible. Our approach outperforms the state-of-the-art on Middlebury high resolution imagery as well as in the more challenging KITTI dataset, while being more efficient than existing slanted plane MRF-based methods, taking on average 2 minutes to perform inference on high resolution imagery.

First an investigation of modeling stochastic processes by difference equations (Markov process) was undertaken. The starting point of the modeling procedure is the knowledge of the spectrum of the process. Two methods are discussed. One is based on optimal estimation theory and leads in most cases to a high-order (perhaps infinite) Markov process. The second method, based on linear system theory, leads to a first order Markov process (in matrix representation). Both methods have been extended to two-dimensional processes. Secondly, recursive estimation (filtering) of two-dimensional random fields was addressed. It was shown that a two-dimensional recursive filter cannot be optimal. Therefore, only a sub-optimal solution is available. This solution minimizes the mean square error for a specific structure of a filter. Finally, applications of modeling and recursive filtering are discussed. An image that includes a target, correlated noise and random noise was processed. Some methods of target enhancement (also called 'restoration' are discussed. (Author)

In this paper we discuss a method, which we call Minimum Conditional Description Length (MCDL), for estimating the parameters of a subset of sites within a Markov random field. We assume that the edges are known for the entire graph $G=(V,E)$. Then, for a subset $U\subset V$, we estimate the parameters for nodes and edges in $U$ as well as for edges incident to a node in $U$, by finding the exponential parameter for that subset that yields the best compression conditioned on the values on the boundary $\partial U$. Our estimate is derived from a temporally stationary sequence of observations on the set $U$. We discuss how this method can also be applied to estimate a spatially invariant parameter from a single configuration, and in so doing, derive the Maximum Pseudo-Likelihood (MPL) estimate.

For Markov random fields on $\mathbb{Z}^d$ with finite state space, we address the statistical estimation of the basic neighborhood, the smallest region that determines the conditional distribution at a site on the condition that the values at all other sites are given. A modification of the Bayesian Information Criterion, replacing likelihood by pseudo-likelihood, is proved to provide strongly consistent estimation from observing a realization of the field on increasing finite regions: the estimated basic neighborhood equals the true one eventually almost surely, not assuming any prior bound on the size of the latter. Stationarity of the Markov field is not required, and phase transition does not affect the results.

Consider a random vector with finite second moments. If its precision matrix is an M-matrix, then all partial correlations are non-negative. If that random vector is additionally Gaussian, the corresponding Markov random field (GMRF) is called attractive. We study estimation of M-matrices taking the role of inverse second moment or precision matrices using sign-constrained log-determinant divergence minimization. We also treat the high-dimensional case with the number of variables exceeding the sample size. The additional sign-constraints turn out to greatly simplify the estimation problem: we provide evidence that explicit regularization is no longer required. To solve the resulting convex optimization problem, we propose an algorithm based on block coordinate descent, in which each sub-problem can be recast as non-negative least squares problem. Illustrations on both simulated and real world data are provided.

An extension of the prior density for path (Onsager-Machlup functional) is defined and shown to exist for Gaussian fields generated by solutions of elliptic Partial Differential Equations (PDEs) driven by white noise. This functional is then used to define and solve the MAP estimation of such fields observed via nonlinear noisy sensors. Existence results and a representation of the estimator are derived for this model

An extension of the prior density for path (Onsager-Machlup functional) is defined and shown to exist for Gaussian fields generated by solutions of elliptic Partial Differential Equations (PDEs) driven by white noise. This functional is then used to define and solve the MAP estimation of such fields observed via nonlinear noisy sensors. Existence results and a representation of the estimator are derived for this model

ADA045179

In this work, we establish the asymptotic normality of the deconvolution kernel density estimator in the context of strongly mixing random fields. Only minimal conditions on the bandwidth parameter are required and a simple criterion on the strong mixing coefficients is provided. Our approach is based on the Lindeberg's method rather than on Bernstein's technique and coupling arguments widely used in previous works on nonparametric estimation for spatial processes. We deal also with nonmixing random fields which can be written as a (nonlinear) functional of i.i.d. random fields by considering the physical dependence measure coefficients introduced by Wu (2005).

Consider the problem of joint parameter estimation and prediction in a Markov random field: i.e., the model parameters are estimated on the basis of an initial set of data, and then the fitted model is used to perform prediction (e.g., smoothing, denoising, interpolation) on a new noisy observation. Working under the restriction of limited computation, we analyze a joint method in which the \emph{same convex variational relaxation} is used to construct an M-estimator for fitting parameters, and to perform approximate marginalization for the prediction step. The key result of this paper is that in the computation-limited setting, using an inconsistent parameter estimator (i.e., an estimator that returns the ``wrong'' model even in the infinite data limit) can be provably beneficial, since the resulting errors can partially compensate for errors made by using an approximate prediction technique. En route to this result, we analyze the asymptotic properties of M-estimators based on convex variational relaxations, and establish a Lipschitz stability property that holds for a broad class of variational methods. We show that joint estimation/prediction based on the reweighted sum-product algorithm substantially outperforms a commonly used heuristic based on ordinary sum-product.

In this paper, we address the Multi-Instance-Learning (MIL) problem when bag labels are naturally represented as ordinal variables (Multi--Instance--Ordinal Regression). Moreover, we consider the case where bags are temporal sequences of ordinal instances. To model this, we propose the novel Multi-Instance Dynamic Ordinal Random Fields (MI-DORF). In this model, we treat instance-labels inside the bag as latent ordinal states. The MIL assumption is modelled by incorporating a high-order cardinality potential relating bag and instance-labels,into the energy function. We show the benefits of the proposed approach on the task of weakly-supervised pain intensity estimation from the UNBC Shoulder-Pain Database. In our experiments, the proposed approach significantly outperforms alternative non-ordinal methods that either ignore the MIL assumption, or do not model dynamic information in target data.

Undirected graphical models are widely used in statistics, physics and machine vision. However Bayesian parameter estimation for undirected models is extremely challenging, since evaluation of the posterior typically involves the calculation of an intractable normalising constant. This problem has received much attention, but very little of this has focussed on the important practical case where the data consists of noisy or incomplete observations of the underlying hidden structure. This paper specifically addresses this problem, comparing two alternative methodologies. In the first of these approaches particle Markov chain Monte Carlo (Andrieu et al., 2010) is used to efficiently explore the parameter space, combined with the exchange algorithm (Murray et al., 2006) for avoiding the calculation of the intractable normalising constant (a proof showing that this combination targets the correct distribution in found in a supplementary appendix online). This approach is compared with approximate Bayesian computation (Pritchard et al., 1999). Applications to estimating the parameters of Ising models and exponential random graphs from noisy data are presented. Each algorithm used in the paper targets an approximation to the true posterior due to the use of MCMC to simulate from the latent graphical model, in lieu of being able to do this exactly in general. The supplementary appendix also describes the nature of the resulting approximation.