Downloads & Free Reading Options - Results

Unsupervised learning of linguistic structure is a difficult task. Frequently, standard techniques such as maximum-likelihood estimation yield poor results or are simply inappropriate (as when the class of models under consideration includes models of varying complexity). In this talk, I discuss how Bayesian statistical methods can be applied to the problem of unsupervised language learning to develop principled model-based systems and improve results. I first present some work on word segmentation, showing that maximum-likelihood estimation is inappropriate for this task and discussing a nonparametric Bayesian modeling solution. I then argue, using part-of-speech tagging as an example, that a Bayesian approach provides advantages even when maximum-likelihood (or maximum a posteriori) estimation is possible. I conclude by discussing some of the challenges that remain in pursuing a Bayesian approach to language learning. ©2007 Microsoft Corporation. All rights reserved.

Online survey built via Qualtrics and disseminated via Prolific to gather data relating to the measurement of aggression/aggressive behaviour in the general population

ADA753627

Introduction In genomic selection, genetic values of individuals are predicted using genetic markers that are distributed all across the genome and are in linkage disequilibrium with quantitative trait locus. Different methods have been introduced to predict genomic breeding values. These methods take into account different assumptions. Non-parametric methods, including artificial neural networks, have fewer assumptions than parametric methods, and can apply nonlinear relationships in genomic predictions so, in theory these approaches are more robust against genetic architecture changes and are able to provide better predictions.   Materials and Methods In current study, the prediction ability of Bayesian neural networks with different architectures (1 to 5 neurons in the hidden layer) and parametric methods (GBLUP, Bayes RR, Bayes A, Bayes B, Bayes C Bayes L) in four simulated genetic architectures and four real traits of mouse (six weeks weight, growth slope, body mass index and body length) were compared using the correlation coefficient between predicted and expected values, mean square error of prediction and computation time. All simulated genetic architectures were additive and the gene effects followed a normal distribution. The number of QTLs in the first and third genetic architecture was 50 and it was 500 for second and fourth genetic architecture. The heritability of the first and second genetic architectures was 0.3 and the heritability of the third and the fourth genetic architectures was 0.7. The real data consisted of 1,296 mice which were genotyped with 9,265 SNP markers.   Results and Discussion The highest prediction accuracy of Bayesian neural networks were 0.640 (4 neuron in the hidden layer), 0.664 (4 neuron in the hidden layer), 0.800 (1 neuron in the hidden layer) and 0.810 (1 neuron in the hidden layer), and the highest prediction accuracy of parametric methods were 0.711(Bayes B), 0.685 (Bayes A), 0.903(Bayes B) and 0.836 (Bayes B) respectively for one to four simulated genetic architectures. These results showed the superiority of parametric methods to Bayesian neural networks in terms of prediction accuracy in genetic architectures with additive effects. In additive genetic architectures, the allelic effects of genetic variations are independent. In parametric models, these effects are assumed to be independent, therefore in additive genetic architectures can be expected that parametric methods are able to provide better predictions than nonparametric methods.   The maximum predictive abilities of Bayesian neural networks to predict six weeks weight, growth slope, body mass index and body length were 0.474 (1 neuron in the hidden layer), 0.349 (4 neuron in the hidden layer), 0.154 (1 neuron in the hidden layer) and 0.214 (4 neuron in the hidden layer). The predictive abilities of parametric methods to predict these traits were similar and equal to 0.477, 0.336, 0.170, and 0.221 in average. The results showed that the predictive abilities of Bayesian neural networks and parametric methods were similar on real data as the difference between the best predictive ability of Bayesian neural networks and parametric methods for Six weeks weight, growth slope and body length were less than 1%. The difference was slightly higher for the body mass index and equal to 1.8%. The mean squared error of prediction of Bayesian Neural Networks was slightly less than parametric methods in the simulated genetic architectures. The results indicate a slight superiority of Bayesian neural networks compared to parametric methods in terms of mean squared error of prediction as an indicator of overall fit. The mean square prediction error is an appropriate criterion for evaluating the prediction performance of different methods because it contains both accuracy and bias. Considering table (3) and table (5), it can be concluded that the prediction of the Bayesian neural network are less accurate but more unbiased than the parametric methods. This could be due to more applied penalty in parametric methods compared to Bayesian neural networks, which can lead to an increase in the average mean squared error of prediction. In real data, the mean squared error of prediction of the Bayesian neural networks and parametric methods were similar. The computation time of Bayesian neural networks was increased with an increase in the number of neurons in the hidden layer. The computation time of the parametric methods was the same with the exception of GBLUP. The GBLUP method took more computation time. The computation time of neural the networks with 1 to 2 neurons in the hidden layer were less than GBLUP. Genomic prediction using Bayesian Neural Networks with a greater number of neurons is really challenging, and improving their performance in terms of computational cost is necessary before applying them in genomic selection. Conclusion Although parametric methods had better predictive accuracy and predictive ability due to the additive genetic architecture of the studied traits, it can be concluded that Bayesian neural networks are powerful tools in genomic enabled prediction that can predict genomic breeding values with acceptable accuracy. The genomic prediction ability of the neural networks depends on target traits, the animal species, and neural network architecture. Before using Bayesian neural networks in genomic prediction, it is better to compare the results with parametric methods. It is also necessary to improve the computation time of the Bayesian neural networks with a greater number of neurons in hidden layer before applying them in real application of genomic selection. 

In an era of large spectroscopic surveys of stars and big data, sophisticated statistical methods become more and more important in order to infer fundamental stellar parameters such as mass and age. Bayesian techniques are powerful methods because they can match all available observables simultaneously to stellar models while taking prior knowledge properly into account. However, in most cases it is assumed that observables are uncorrelated which is generally not the case. Here, we include correlations in the Bayesian code BONNSAI by incorporating the covariance matrix in the likelihood function. We derive a parametrisation of the covariance matrix that, in addition to classical uncertainties, only requires the specification of a correlation parameter that describes how observables co-vary. Our correlation parameter depends purely on the method with which observables have been determined and can be analytically derived in some cases. This approach therefore has the advantage that correlations can be accounted for even if information for them are not available in specific cases but are known in general. Because the new likelihood model is a better approximation of the data, the reliability and robustness of the inferred parameters are improved. We find that neglecting correlations biases the most likely values of inferred stellar parameters and affects the precision with which these parameters can be determined. For example, we apply our technique to massive OB stars, but emphasise that it is valid for any type of stars. For effective temperatures and surface gravities determined from atmosphere modelling, we find that masses can be underestimated on average by $0.5\sigma$ and mass uncertainties overestimated by a factor of about 2 when neglecting correlations. At the same time, the age precisions are underestimated over a wide range of stellar parameters. [abridged]

It takes years of effort employing the best telescopes and instruments to obtain high-quality stellar photometry, astrometry, and spectroscopy. Stellar evolution models contain the experience of lifetimes of theoretical calculations and testing. Yet most astronomers fit these valuable models to these precious datasets by eye. We show that a principled Bayesian approach to fitting models to stellar data yields substantially more information over a range of stellar astrophysics. We highlight advances in determining the ages of star clusters, mass ratios of binary stars, limitations in the accuracy of stellar models, post-main-sequence mass loss, and the ages of individual white dwarfs. We also outline a number of unsolved problems that would benefit from principled Bayesian analyses.

This article is from BMC Bioinformatics , volume 12 . Abstract Background: A Bayesian approach based on a Dirichlet process (DP) prior is useful for inferring genetic population structures because it can infer the number of populations and the assignment of individuals simultaneously. However, the properties of the DP prior method are not well understood, and therefore, the use of this method is relatively uncommon. We characterized the DP prior method to increase its practical use. Results: First, we evaluated the usefulness of the sequentially-allocated merge-split (SAMS) sampler, which is a technique for improving the mixing of Markov chain Monte Carlo algorithms. Although this sampler has been implemented in a preceding program, HWLER, its effectiveness has not been investigated. We showed that this sampler was effective for population structure analysis. Implementation of this sampler was useful with regard to the accuracy of inference and computational time. Second, we examined the effect of a hyperparameter for the prior distribution of allele frequencies and showed that the specification of this parameter was important and could be resolved by considering the parameter as a variable. Third, we compared the DP prior method with other Bayesian clustering methods and showed that the DP prior method was suitable for data sets with unbalanced sample sizes among populations. In contrast, although current popular algorithms for population structure analysis, such as those implemented in STRUCTURE, were suitable for data sets with uniform sample sizes, inferences with these algorithms for unbalanced sample sizes tended to be less accurate than those with the DP prior method. Conclusions: The clustering method based on the DP prior was found to be useful because it can infer the number of populations and simultaneously assign individuals into populations, and it is suitable for data sets with unbalanced sample sizes among populations. Here we presented a novel program, DPART, that implements the SAMS sampler and can consider the hyperparameter for the prior distribution of allele frequencies to be a variable.

A series system is simulated to obtain lower confidence limits on system reliability using Bayesian techniques. A comparison between classical and Bayesian methods is made. Random beta variate generators are developed and used in the simulation. The results of the simulation are tabulated for easy comparison of the Bayesian and classical methods. The values of lower confidence limits that are realized using the Bayesian method decrease as the number of components increase. In most cases, as the number of components increase, the Bayesian method appears to yield lower values of lower confidence limits than the classical method.

We present a parameter estimation procedure based on a Bayesian framework by applying a Markov Chain Monte Carlo algorithm to the calibration of the dynamical parameters of a space based gravitational wave detector. The method is based on the Metropolis-Hastings algorithm and a two-stage annealing treatment in order to ensure an effective exploration of the parameter space at the beginning of the chain. We compare two versions of the algorithm with an application to a LISA Pathfinder data analysis problem. The two algorithms share the same heating strategy but with one moving in coordinate directions using proposals from a multivariate Gaussian distribution, while the other uses the natural logarithm of some parameters and proposes jumps in the eigen-space of the Fisher Information matrix. The algorithm proposing jumps in the eigen-space of the Fisher Information matrix demonstrates a higher acceptance rate and a slightly better convergence towards the equilibrium parameter distributions in the application to LISA Pathfinder data . For this experiment, we return parameter values that are all within $\sim1\sigma$ of the injected values. When we analyse the accuracy of our parameter estimation in terms of the effect they have on the force-per-unit test mass noise estimate, we find that the induced errors are three orders of magnitude less than the expected experimental uncertainty in the power spectral density.

After a derivation of the quantum Bayes theorem, and a discussion of the reconstruction of the unknown state of identical spin systems by repeated measurements, the main part of this paper treats the problem of determining the unknown phase difference of two coherent sources by photon measurements. While the approach of this paper is based on computing correlations of actual measurements (photon detections), it is possible to derive indirectly a probability distribution for the phase difference. In this approach, the quantum phase is not an observable, but a parameter of an unknown quantum state. Photon measurements determine a probability distribution for the phase difference. The approach used in this paper takes into account both photon statistics and the finite efficiency of the detectors.

Bayesian methods for statistical analysis is a book on statistical methods for analysing a wide variety of data. The book consists of 12 chapters, starting with basic concepts and covering numerous topics, including Bayesian estimation, decision theory, prediction, hypothesis testing, hierarchical models, Markov chain Monte Carlo methods, finite population inference, biased sampling and nonignorable nonresponse. The book contains many exercises, all with worked solutions, including complete computer code. It is suitable for self-study or a semester-long course, with three hours of lectures and one tutorial per week for 13 weeks.

Producers and consumers of test scores are increasingly concerned about fraudulent behavior before and during the test. There exist several statistical or psychometric methods for detecting fraudulent behavior on tests. This paper provides a review of the Bayesian approaches among them. Four hitherto-unpublished real data examples are provided to demonstrate the application of Bayesian approaches to detect various types of fraudulent behavior on tests. The examples show that Bayesian methods can be useful in detecting several types of test fraud. [This paper was published in "Measurement: Interdisciplinary Research and Perspectives" (EJ1174692).]

Learning the network structure of a large graph is computationally demanding, and dynamically monitoring the network over time for any changes in structure threatens to be more challenging still. This paper presents a two-stage method for anomaly detection in dynamic graphs: the first stage uses simple, conjugate Bayesian models for discrete time counting processes to track the pairwise links of all nodes in the graph to assess normality of behavior; the second stage applies standard network inference tools on a greatly reduced subset of potentially anomalous nodes. The utility of the method is demonstrated on simulated and real data sets.

One of the major test requirements for study of equipment that increases their efficiency is to reduce the number of tests at a given accuracy of statistical information. Based on the Bayesian method, which takes into account a priori data of statistical studies, the algorithm for estimates of the results for statistical tests has been made. This algorithm provides the required accuracy with less number of test

One of the major test requirements for study of equipment that increases their efficiency is to reduce the number of tests at a given accuracy of statistical information. Based on the Bayesian method, which takes into account a priori data of statistical studies, the algorithm for estimates of the results for statistical tests has been made. This algorithm provides the required accuracy with less number of test

One of the major test requirements for study of equipment that increases their efficiency is to reduce the number of tests at a given accuracy of statistical information. Based on the Bayesian method, which takes into account a priori data of statistical studies, the algorithm for estimates of the results for statistical tests has been made. This algorithm provides the required accuracy with less number of test

This is an individual participant data meta-analysis (IPD) or retrospective pooled study. Six unpublished studies from the Computational Psychiatry lab at Melbourne University were previously carried out using variations of the Coin Estimation Task (described below) which modelled participants’ weight of sensory reliance relative to prior expectations. All studies were carried out online and participants were sourced from a mixed online university population and the general population. All studies previously investigated a variety of different research questions to the current study, with the primary outcomes variable of interest being sensory reliance (i.e., slope), as well as subjective prior and likelihood variance. Part of this data is published in (Goodwin et al, 2022). As a part of their exploratory analyses, two studies observed a significant effect of a shifted subjective prior mean to the right of the true prior (true prior = 0.0 screen coordinate) and one study showed a non-significant shift. The present study will attempt to collate all previous data from six researchers to investigate whether an overall effect of spatial asymmetry is observed.

Bayesian network meta-analysis will be conducted using STATA V.14.0 and WinBUGS V.1.4.3 to compare the efficacy of different acupuncture methods for diabetic peripheral neuropathy (DPN).

Minimum mean squared error (MMSE) estimators of signals from samples corrupted by jitter (timing noise) and additive noise are nonlinear, even when the signal prior and additive noise have normal distributions. This paper develops a stochastic algorithm based on Gibbs sampling and slice sampling to approximate the optimal MMSE estimator in this Bayesian formulation. Simulations demonstrate that this nonlinear algorithm can improve significantly upon the linear MMSE estimator, as well as the EM algorithm approximation to the maximum likelihood (ML) estimator used in classical estimation. Effective off-chip post-processing to mitigate jitter enables greater jitter to be tolerated, potentially reducing on-chip ADC power consumption.

The objectives of this scoping review are to: 1. Identify Stroke trials that used Bayesian Statistics. 2. Determine the Motivation for adopting Bayesian statistics. 3. Examine how the technical choices around the construction of the Prior and Posterior distributions illustrated the motivations for using Bayesian statistics.

Discussion of "Bayesian Models and Methods in Public Policy and Government Settings" by S. E. Fienberg [arXiv:1108.2177]

In this paper, we consider the so-called Shape Invariant Model which stands for the estimation of a function f0 submitted to a random translation of law g0 in a white noise model. We are interested in such a model when the law of the deformations is unknown. We aim to recover the law of the process P(f0,g0). In this perspective, we adopt a Bayesian point of view and find prior on f and g such that the posterior distribution concentrates at a polynomial rate around P(f0,g0) when n goes to infinity. We intensively use some Bayesian non parametric tools coupled with mixture models and believe that some of our results obtained on this mixture framework may be also of interest for frequentist point of view.

This is a network meta-analysis to investigate the efficacy and safety of acupuncture methods for nonspecific low back pain. We will compare acupuncture methods for nonspecific low back pain by bayesian network meta‑analysis and rank the priority of acupuncture methods to assess the efficacy and safety of diverse acupuncture methods for nonspecific low back pain treatment.

This is a network meta-analysis to investigate the efficacy and safety of acupuncture methods for nonspecific low back pain. We will compare acupuncture methods for nonspecific low back pain by bayesian network meta‑analysis and rank the priority of acupuncture methods to assess the efficacy and safety of diverse acupuncture methods for nonspecific low back pain treatment.

This is a network meta-analysis to investigate the efficacy and safety of acupuncture methods for nonspecific low back pain. We will compare acupuncture methods for nonspecific low back pain by bayesian network meta‑analysis and rank the priority of acupuncture methods to assess the efficacy and safety of diverse acupuncture methods for nonspecific low back pain treatment.

This is a network meta-analysis to investigate the efficacy and safety of acupuncture methods for nonspecific low back pain. We will compare acupuncture methods for nonspecific low back pain by bayesian network meta‑analysis and rank the priority of acupuncture methods to assess the efficacy and safety of diverse acupuncture methods for nonspecific low back pain treatment.

This survey reviews the existing literature on the most relevant Bayesian inference methods for univariate and multivariate GARCH models. The advantages and drawbacks of each procedure are outlined as well as the advantages of the Bayesian approach versus classical procedures. The paper makes emphasis on recent Bayesian non-parametric approaches for GARCH models that avoid imposing arbitrary parametric distributional assumptions. These novel approaches implicitly assume infinite mixture of Gaussian distributions on the standardized returns which have been shown to be more flexible and describe better the uncertainty about future volatilities. Finally, the survey presents an illustration using real data to show the flexibility and usefulness of the non-parametric approach.

We consider comparisons of statistical learning algorithms using multiple data sets, via leave-one-in cross-study validation: each of the algorithms is trained on one data set; the resulting model is then validated on each remaining data set. This poses two statistical challenges that need to be addressed simultaneously. The first is the assessment of study heterogeneity, with the aim of identifying a subset of studies within which algorithm comparisons can be reliably carried out. The second is the comparison of algorithms using the ensemble of data sets. We address both problems by integrating clustering and model comparison. We formulate a Bayesian model for the array of cross-study validation statistics, which defines clusters of studies with similar properties and provides the basis for meaningful algorithm comparison in the presence of study heterogeneity. We illustrate our approach through simulations involving studies with varying severity of systematic errors, and in the context of medical prognosis for patients diagnosed with cancer, using high-throughput measurements of the transcriptional activity of the tumor's genes.

Optimal experimental design (OED) seeks experiments expected to yield the most useful data for some purpose. In practical circumstances where experiments are time-consuming or resource-intensive, OED can yield enormous savings. We pursue OED for nonlinear systems from a Bayesian perspective, with the goal of choosing experiments that are optimal for parameter inference. Our objective in this context is the expected information gain in model parameters, which in general can only be estimated using Monte Carlo methods. Maximizing this objective thus becomes a stochastic optimization problem. This paper develops gradient-based stochastic optimization methods for the design of experiments on a continuous parameter space. Given a Monte Carlo estimator of expected information gain, we use infinitesimal perturbation analysis to derive gradients of this estimator. We are then able to formulate two gradient-based stochastic optimization approaches: (i) Robbins-Monro stochastic approximation, and (ii) sample average approximation combined with a deterministic quasi-Newton method. A polynomial chaos approximation of the forward model accelerates objective and gradient evaluations in both cases. We discuss the implementation of these optimization methods, then conduct an empirical comparison of their performance. To demonstrate design in a nonlinear setting with partial differential equation forward models, we use the problem of sensor placement for source inversion. Numerical results yield useful guidelines on the choice of algorithm and sample sizes, assess the impact of estimator bias, and quantify tradeoffs of computational cost versus solution quality and robustness.

In this paper, we consider the so-called Shape Invariant Model which stands for the estimation of a function f0 submitted to a random translation of law g0 in a white noise model. We are interested in such a model when the law of the deformations is unknown. We aim to recover the law of the process P(f0,g0) as well as f0 and g0. We first provide some identifiability result on this model and then adopt a Bayesian point of view. In this view, we find some prior on f and g such that the posterior distribution concentrates around the functions f0 and g0 when n goes to infinity, we then obtain a contraction rate of order a power of log(n)^(-1). We also obtain a lower bound on the model for the estimation of f0 and g0 in a frequentist paradigm which also decreases following a power of log(n)^(-1).

This paper provides a detailed theoretical analysis of methods to approximate the solutions of high-dimensional (>10^6) linear Bayesian problems. An optimal low-rank projection that maximizes the information content of the Bayesian inversion is proposed and efficiently constructed using a scalable randomized SVD algorithm. Useful optimality results are established for the associated posterior error covariance matrix and posterior mean approximations, which are further investigated in a numerical experiment consisting of a large-scale atmospheric tracer transport source-inversion problem. This method proves to be a robust and efficient approach to dimension reduction, as well as a natural framework to analyze the information content of the inversion. Possible extensions of this approach to the non-linear framework in the context of operational numerical weather forecast data assimilation systems based on the incremental 4D-Var technique are also discussed, and a detailed implementation of a new Randomized Incremental Optimal Technique (RIOT) for 4D-Var algorithms leveraging our theoretical results is proposed.

This paper provides a detailed theoretical analysis of methods to approximate the solutions of high-dimensional (>10^6) linear Bayesian problems. An optimal low-rank projection that maximizes the information content of the Bayesian inversion is proposed and efficiently constructed using a scalable randomized SVD algorithm. Useful optimality results are established for the associated posterior error covariance matrix and posterior mean approximations, which are further investigated in a numerical experiment consisting of a large-scale atmospheric tracer transport source-inversion problem. This method proves to be a robust and efficient approach to dimension reduction, as well as a natural framework to analyze the information content of the inversion. Possible extensions of this approach to the non-linear framework in the context of operational numerical weather forecast data assimilation systems based on the incremental 4D-Var technique are also discussed, and a detailed implementation of a new Randomized Incremental Optimal Technique (RIOT) for 4D-Var algorithms leveraging our theoretical results is proposed.

Stochastic approximation Monte Carlo (SAMC) has recently been proposed by Liang, Liu and Carroll [J. Amer. Statist. Assoc. 102 (2007) 305--320] as a general simulation and optimization algorithm. In this paper, we propose to improve its convergence using smoothing methods and discuss the application of the new algorithm to Bayesian model selection problems. The new algorithm is tested through a change-point identification example. The numerical results indicate that the new algorithm can outperform SAMC and reversible jump MCMC significantly for the model selection problems. The new algorithm represents a general form of the stochastic approximation Markov chain Monte Carlo algorithm. It allows multiple samples to be generated at each iteration, and a bias term to be included in the parameter updating step. A rigorous proof for the convergence of the general algorithm is established under verifiable conditions. This paper also provides a framework on how to improve efficiency of Monte Carlo simulations by incorporating some nonparametric techniques.

We describe work in progress by a collaboration of astronomers and statisticians developing a suite of Bayesian data analysis tools for extrasolar planet (exoplanet) detection, planetary orbit estimation, and adaptive scheduling of observations. Our work addresses analysis of stellar reflex motion data, where a planet is detected by observing the "wobble" of its host star as it responds to the gravitational tug of the orbiting planet. Newtonian mechanics specifies an analytical model for the resulting time series, but it is strongly nonlinear, yielding complex, multimodal likelihood functions; it is even more complex when multiple planets are present. The parameter spaces range in size from few-dimensional to dozens of dimensions, depending on the number of planets in the system, and the type of motion measured (line-of-sight velocity, or position on the sky). Since orbits are periodic, Bayesian generalizations of periodogram methods facilitate the analysis. This relies on the model being linearly separable, enabling partial analytical marginalization, reducing the dimension of the parameter space. Subsequent analysis uses adaptive Markov chain Monte Carlo methods and adaptive importance sampling to perform the integrals required for both inference (planet detection and orbit measurement), and information-maximizing sequential design (for adaptive scheduling of observations). We present an overview of our current techniques and highlight directions being explored by ongoing research.

We develop a generalized hybrid iterative approach for computing solutions to large-scale Bayesian inverse problems. We consider a hybrid algorithm based on the generalized Golub-Kahan bidiagonalization for computing Tikhonov regularized solutions to problems where explicit computation of the square root and inverse of the covariance kernel for the prior covariance matrix is not feasible. This is useful for large-scale problems where covariance kernels are defined on irregular grids or are only available via matrix-vector multiplication, e.g., those from the Mat\'{e}rn class. We show that iterates are equivalent to LSQR iterates applied to a directly regularized Tikhonov problem, after a transformation of variables, and we provide connections to a generalized singular value decomposition filtered solution. Our approach shares many benefits of standard hybrid methods such as avoiding semi-convergence and automatically estimating the regularization parameter. Numerical examples from image processing demonstrate the effectiveness of the described approaches.

We develop a generalized hybrid iterative approach for computing solutions to large-scale Bayesian inverse problems. We consider a hybrid algorithm based on the generalized Golub-Kahan bidiagonalization for computing Tikhonov regularized solutions to problems where explicit computation of the square root and inverse of the covariance kernel for the prior covariance matrix is not feasible. This is useful for large-scale problems where covariance kernels are defined on irregular grids or are only available via matrix-vector multiplication, e.g., those from the Mat\'{e}rn class. We show that iterates are equivalent to LSQR iterates applied to a directly regularized Tikhonov problem, after a transformation of variables, and we provide connections to a generalized singular value decomposition filtered solution. Our approach shares many benefits of standard hybrid methods such as avoiding semi-convergence and automatically estimating the regularization parameter. Numerical examples from image processing demonstrate the effectiveness of the described approaches.

Bibliography: p. 28

The following zero-sum game between nature and a statistician blends Bayesian methods with frequentist methods such as p-values and confidence intervals. Nature chooses a posterior distribution consistent with a set of possible priors. At the same time, the statistician selects a parameter distribution for inference with the goal of maximizing the minimum Kullback-Leibler information gained over a confidence distribution or other benchmark distribution. An application to testing a simple null hypothesis leads the statistician to report a posterior probability of the hypothesis that is informed by both Bayesian and frequentist methodology, each weighted according how well the prior is known. Since neither the Bayesian approach nor the frequentist approach is entirely satisfactory in situations involving partial knowledge of the prior distribution, the proposed procedure reduces to a Bayesian method given complete knowledge of the prior, to a frequentist method given complete ignorance about the prior, and to a blend between the two methods given partial knowledge of the prior. The blended approach resembles the Bayesian method rather than the frequentist method to the precise extent that the prior is known. The problem of testing a point null hypothesis illustrates the proposed framework. The blended probability that the null hypothesis is true is equal to the p-value or a lower bound of an unknown Bayesian posterior probability, whichever is greater. Thus, given total ignorance represented by a lower bound of 0, the p-value is used instead of any Bayesian posterior probability. At the opposite extreme of a known prior, the p-value is ignored. In the intermediate case, the possible Bayesian posterior probability that is closest to the p-value is used for inference. Thus, both the Bayesian method and the frequentist method influence the inferences made.

Discussion of "Impact of Frequentist and Bayesian Methods on Survey Sampling Practice: A Selective Appraisal" by J. N. K. Rao [arXiv:1108.2356]

Cognitive radio is an exciting wireless technology that has been introduced for the efficient used of spectrum. Using cognitive radios (CRs), the secondary users (unlicensed users) are allowed to use the spectrum which is originally allocated to primary users (PUs) as far as the active primary users are not using it temporarily. In order to prevent harmful interference to primary users, the SUs need to perform spectrum sensing before transmitting signal over the spectrum. In this paper we use an optimal Bayesian detector for digitally modulated primary user to improve the spectrum utilization, without prior knowledge of transmitted sequence of the primary signals. And further suboptimal detectors in low and high SNR regime. We provide the performance analysis in terms of Detection probability and False alarm probability. Abdul Hameed Ansari | Narode Sweety S."Performance Analysis of Bayesian Methods to for the Spectrum Utilization in Cognitive Radio" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-1 | Issue-5 , August 2017, URL: http://www.ijtsrd.com/papers/ijtsrd2385.pdf Article URL: http://www.ijtsrd.com/engineering/electronics-and-communication-engineering/2385/performance-analysis-of-bayesian-methods-to-for-the-spectrum-utilization-in-cognitive-radio/abdul-hameed-ansari

Current analysis of astronomical data are confronted with the daunting task of modeling the awkward features of astronomical data, among which heteroscedastic (point-dependent) errors, intrinsic scatter, non-ignorable data collection (selection effects), data structure, non-uniform populations (often called Malmquist bias), non-Gaussian data, and upper/lower limits. This chapter shows, by examples, how modeling all these features using Bayesian methods. In short, one just need to formalize, using maths, the logical link between the involved quantities, how the data arise and what we already known on the quantities we want to study. The posterior probability distribution summarizes what we known on the studied quantities after the data, and we should not be afraid about their actual numerical computation, because it is left to (special) Monte Carlo programs such as JAGS. As examples, we show how to predict the mass of a new object disposing of a calibrating sample, how to constraint cosmological parameters from supernovae data and how to check if the fitted data are in tension with the adopted fitting model. Examples are given with their coding. These examples can be easily used as template for completely different analysis, on totally unrelated astronomical objects, requiring to model the same awkward data features.

This thesis examines three methods for calculating the 100(1- α)% lower confidence limits for the reliability of a K-sized series system. Assuming that each component reliability has a Beta distribution, identical posterior parameters A and B are assigned for each component.

In genetic association analyses, it is often desired to analyze data from multiple potentially-heterogeneous subgroups. The amount of expected heterogeneity can vary from modest (as might typically be expected in a meta-analysis of multiple studies of the same phenotype, for example), to large (e.g. a strong gene-environment interaction, where the environmental exposure defines discrete subgroups). Here, we consider a flexible set of Bayesian models and priors that can capture these different levels of heterogeneity. We provide accurate numerical approaches to compute approximate Bayes Factors for these different models, and also some simple analytic forms which have natural interpretations and, in some cases, close connections with standard frequentist test statistics. These approximations also have the convenient feature that they require only summary-level data from each subgroup (in the simplest case, a point estimate for the genetic effect, and its standard error, from each subgroup). We illustrate the flexibility of these approaches on three examples: an analysis of a potential gene-environment interaction for a recombination phenotype, a large scale meta-analysis of genome-wide association data from the Global Lipids consortium, and a cross-population analysis for expression quantitative trait loci (eQTLs).

Unsupervised learning of linguistic structure is a difficult task. Frequently, standard techniques such as maximum-likelihood estimation yield poor results or are simply inappropriate (as when the class of models under consideration includes models of varying complexity). In this talk, I discuss how Bayesian statistical methods can be applied to the problem of unsupervised language learning to develop principled model-based systems and improve results. I first present some work on word segmentation, showing that maximum-likelihood estimation is inappropriate for this task and discussing a nonparametric Bayesian modeling solution. I then argue, using part-of-speech tagging as an example, that a Bayesian approach provides advantages even when maximum-likelihood (or maximum a posteriori) estimation is possible. I conclude by discussing some of the challenges that remain in pursuing a Bayesian approach to language learning. ©2007 Microsoft Corporation. All rights reserved.

Approximate Bayesian computation (ABC) methods make use of comparisons between simulated and observed summary statistics to overcome the problem of computationally intractable likelihood functions. As the practical implementation of ABC requires computations based on vectors of summary statistics, rather than full data sets, a central question is how to derive low-dimensional summary statistics from the observed data with minimal loss of information. In this article we provide a comprehensive review and comparison of the performance of the principal methods of dimension reduction proposed in the ABC literature. The methods are split into three nonmutually exclusive classes consisting of best subset selection methods, projection techniques and regularization. In addition, we introduce two new methods of dimension reduction. The first is a best subset selection method based on Akaike and Bayesian information criteria, and the second uses ridge regression as a regularization procedure. We illustrate the performance of these dimension reduction techniques through the analysis of three challenging models and data sets.

Approximate Bayesian computation (ABC) methods make use of comparisons between simulated and observed summary statistics to overcome the problem of computationally intractable likelihood functions. As the practical implementation of ABC requires computations based on vectors of summary statistics, rather than full data sets, a central question is how to derive low-dimensional summary statistics from the observed data with minimal loss of information. In this article we provide a comprehensive review and comparison of the performance of the principal methods of dimension reduction proposed in the ABC literature. The methods are split into three nonmutually exclusive classes consisting of best subset selection methods, projection techniques and regularization. In addition, we introduce two new methods of dimension reduction. The first is a best subset selection method based on Akaike and Bayesian information criteria, and the second uses ridge regression as a regularization procedure. We illustrate the performance of these dimension reduction techniques through the analysis of three challenging models and data sets.

In this paper, we develop a Bayesian multiscale approach based on a multiscale finite element method. Because of scale disparity in many multiscale applications, computational models can not resolve all scales. Various subgrid models are proposed to represent un-resolved scales. Here, we consider a probabilistic approach for modeling un-resolved scales using the Multiscale Finite Element Method (cf., [1, 2]). By representing dominant modes using the Generalized Multiscale Finite Element, we propose a Bayesian framework, which provides multiple inexpensive (computable) solutions for a deterministic problem. These approximate probabilistic solutions may not be very close to the exact solutions and, thus, many realizations are needed. In this way, we obtain a rigorous probabilistic description of approximate solutions. In the paper, we consider parabolic and wave equations in heterogeneous media. In each time interval, the domain is divided into subregions. Using residual information, we design appropriate prior and posterior distributions. The likelihood consists of the residual minimization. To sample from the resulting posterior distribution, we consider several sampling strategies. The sampling involves identifying important regions and important degrees of freedom beyond permanent basis functions, which are used in residual computation. Numerical results are presented. We consider two sampling algorithms. The first algorithm uses sequential sampling and is inexpensive. In the second algorithm, we perform full sampling using the Gibbs sampling algorithm, which is more accurate compared to the sequential sampling. The main novel ingredients of our approach consist of: defining appropriate permanent basis functions and the corresponding residual; setting up a proper posterior distribution; and sampling the posteriors.

Introduction Genomic Selection (GS) has been proved to be a powerful tool for estimating genetic values in livestock breeding. Newly developed sequencing technologies have dramatically reduced the cost of genotyping and significantly increased the scale of genotype data that used for GS. The estimation of breeding values in order to select the best animals as parents of the next generation is the main goal of animal breeding programs. Traditional methods of genetic evaluation were performed using a combination of phenotypic and pedigree information to produce estimated breeding values. Most simulation studies of genomic selection (GS) methods have considered genetic architectures in which the number and relative magnitudes of quantitative trait loci (QTL) have varied. Among the Bayesian methods, those using marker-specific shrinkage of effects (e.g., BayesA or BayesB of or the Bayesian LASSO are commonly used in animal breeding applications. The Bayesian methods proposed differ in the way of looking at the variances of parameters. In classical livestock breeding methods, selection for important economic traits using pedigree information with individual phenotypic records was performed and best Linear Prediction of Breeding Values (BLUP) is achieved. In genome selection, genomic breeding values of all individuals can be predicted with high accuracy using a linear model. Various factors can be affecting the accuracy of genomic breeding values. Therefore, the present study aimed to evaluate the accuracy of estimating genomic breeding values in different genetic architectures including different distributions of gene effects, different numbers of QTL, different levels of heritability and different marker densities using GBLUP and Bayesian methods including Bayes A, Bayes B, Bayes C and Bayes LASSO. In addition to comparing the performance of different methods in different genetic architectures, a marker density and QTL numbers were introduced for simulation programs of sheep populations.   Materials and Methods To create a basic population (G0), 100 heads of livestock, including 50 males and 50 females, were considered. The frequency of primary alleles for single-nucleotide polymorphisms in the basal generation was considered to be 0.5. To create the first generation (G1), the parents were randomly selected from the males and females of the G0 generation. Parental gametes were simulated based on the assumption of disconnection imbalance using the Halldan location function method, and then randomly generated gametes were randomly selected and mixed to create a new generation of G1 generation. A genome with a length of 300 cM was simulated and 500, 1000 and 1500 SNPs were equally spaced over the chromosome. Three different numbers of QTL (50, 100 and 150) were considered and QTLs were uniformly distributed over the chromosome. One hundred individuals, including 50 males and 50 females, were simulated for the base population. The first generation structure was followed through to the 50th generation of random mating to make linkage disequilibrium populations. Generation 51 was assumed as a training population and the other generations (52 to 60) as validation populations. Five methods, GBLUP, Bayes A, Bayes B, Bayes C and Bayesian LASSO, were used to estimate genomic breeding values.   Results and Discussion In all five methods, the accuracy of genomic values decreased as the number of QTLs increased from 50 to 150. The reason for this can be attributed to the limited amount of genetic variance distributed over many QTLs. Also predicting accuracy of all five methods increased with increasing marker density. Results showed that increasing marker density at low (0.1) and high (0.5) heritability levels, increased genomic accuracy but increasing at moderate heritability (0.3) traits did not affect the accuracy of genomic evaluation. Accuracy of genomic breeding values in the gamma distribution provides better gene effects to uniform distributions.   Conclusion The results showed that factors such as marker density, QTL numbers, distribution QTL effect and trait heritability were effective in estimating the accuracy of genomic breeding values. In high heritability traits, the higher markers density and lower QTL numbers, leading to increase accuracy of estimating genomic breeding values. In genomic studies, if the trait is affected by a small number of QTLs, estimation of breeding values by Bayes B method can yield a more favorable result. Marker densities did not affect the accuracy of genomic evaluation in traits of moderate heritability, and since most of the economic traits in native species of sheep are moderate heritability, 500 to 1000 markers can be used to estimate breeding values in simulation programs

In this paper, we propose a generalized scale mixture family of distributions, namely the Power Exponential Scale Mixture (PESM) family, to model the sparsity inducing priors currently in use for sparse signal recovery (SSR). We show that the successful and popular methods such as LASSO, Reweighted $\ell_1$ and Reweighted $\ell_2$ methods can be formulated in an unified manner in a maximum a posteriori (MAP) or Type I Bayesian framework using an appropriate member of the PESM family as the sparsity inducing prior. In addition, exploiting the natural hierarchical framework induced by the PESM family, we utilize these priors in a Type II framework and develop the corresponding EM based estimation algorithms. Some insight into the differences between Type I and Type II methods is provided and of particular interest in the algorithmic development is the Type II variant of the popular and successful reweighted $\ell_1$ method. Extensive empirical results are provided and they show that the Type II methods exhibit better support recovery than the corresponding Type I methods.

Explosive growth in data and availability of cheap computing resources have sparked increasing interest in Big learning, an emerging subfield that studies scalable machine learning algorithms, systems, and applications with Big Data. Bayesian methods represent one important class of statistic methods for machine learning, with substantial recent developments on adaptive, flexible and scalable Bayesian learning. This article provides a survey of the recent advances in Big learning with Bayesian methods, termed Big Bayesian Learning, including nonparametric Bayesian methods for adaptively inferring model complexity, regularized Bayesian inference for improving the flexibility via posterior regularization, and scalable algorithms and systems based on stochastic subsampling and distributed computing for dealing with large-scale applications.