Downloads & Free Reading Options - Results

This article is from Journal of Psychopharmacology (Oxford, England) , volume 26 . Abstract Evidence suggests that some aspects of schizophrenia can be induced in healthy volunteers through acute administration of the non-competitive NMDA-receptor antagonist, ketamine. In probabilistic inference tasks, patients with schizophrenia have been shown to ‘jump to conclusions’ (JTC) when asked to make a decision. We aimed to test whether healthy participants receiving ketamine would adopt a JTC response pattern resembling that of patients. The paradigmatic task used to investigate JTC has been the ‘urn’ task, where participants are shown a sequence of beads drawn from one of two ‘urns’, each containing coloured beads in different proportions. Participants make a decision when they think they know the urn from which beads are being drawn. We compared performance on the urn task between controls receiving acute ketamine or placebo with that of patients with schizophrenia and another group of controls matched to the patient group. Patients were shown to exhibit a JTC response pattern relative to their matched controls, whereas JTC was not evident in controls receiving ketamine relative to placebo. Ketamine does not appear to promote JTC in healthy controls, suggesting that ketamine does not affect probabilistic inferences.

The problem of optimal motion planing and control is fundamental in robotics. However, this problem is intractable for continuous-time stochastic systems in general and the solution is difficult to approximate if non-instantaneous nonlinear performance indices are present. In this work, we provide an efficient algorithm, PIPC (Probabilistic Inference for Planning and Control), that yields approximately optimal policies with arbitrary higher-order nonlinear performance indices. Using probabilistic inference and a Gaussian process representation of trajectories, PIPC exploits the underlying sparsity of the problem such that its complexity scales linearly in the number of nonlinear factors. We demonstrate the capabilities of our algorithm in a receding horizon setting with multiple systems in simulation.

In this project, we study how people make inferences about players in a tug of war game based on direct evidence about their performance (i.e. the outcome of tournament matches) as well as based on information by a commentator. We explore whether people's inferences are well-captured by a probabilistic program that captures people's domain knowledge using compositional concepts that combine productively.

The properties of the normal distribution under linear transformation, as well the easy way to compute the covariance matrix of marginals and conditionals, offer a unique opportunity to get an insight about several aspects of uncertainties in measurements. The way to build the overall covariance matrix in a few, but conceptually relevant cases is illustrated: several observations made with (possibly) different instruments measuring the same quantity; effect of systematics (although limited to offset, in order to stick to linear models) on the determination of the 'true value', as well in the prediction of future observations; correlations which arise when different quantities are measured with the same instrument affected by an offset uncertainty; inferences and predictions based on averages; inference about constrained values; fits under some assumptions (linear models with known standard deviations). Many numerical examples are provided, exploiting the ability of the R language to handle large matrices and to produce high quality plots. Some of the results are framed in the general problem of 'propagation of evidence', crucial in analyzing graphical models of knowledge.

We introduce a new approach to solving path-finding problems under uncertainty by representing them as probabilistic models and applying domain-independent inference algorithms to the models. This approach separates problem representation from the inference algorithm and provides a framework for efficient learning of path-finding policies. We evaluate the new approach on the Canadian Traveler Problem, which we formulate as a probabilistic model, and show how probabilistic inference allows high performance stochastic policies to be obtained for this problem.

Probabilistic inference offers a principled framework for understanding both behaviour and cortical computation. However, two basic and ubiquitous properties of cortical responses seem difficult to reconcile with probabilistic inference: neural activity displays prominent oscillations in response to constant input, and large transient changes in response to stimulus onset. Here we show that these dynamical behaviours may in fact be understood as hallmarks of the specific representation and algorithm that the cortex employs to perform probabilistic inference. We demonstrate that a particular family of probabilistic inference algorithms, Hamiltonian Monte Carlo (HMC), naturally maps onto the dynamics of excitatory-inhibitory neural networks. Specifically, we constructed a model of an excitatory-inhibitory circuit in primary visual cortex that performed HMC inference, and thus inherently gave rise to oscillations and transients. These oscillations were not mere epiphenomena but served an important functional role: speeding up inference by rapidly spanning a large volume of state space. Inference thus became an order of magnitude more efficient than in a non-oscillatory variant of the model. In addition, the network matched two specific properties of observed neural dynamics that would otherwise be difficult to account for in the context of probabilistic inference. First, the frequency of oscillations as well as the magnitude of transients increased with the contrast of the image stimulus. Second, excitation and inhibition were balanced, and inhibition lagged excitation. These results suggest a new functional role for the separation of cortical populations into excitatory and inhibitory neurons, and for the neural oscillations that emerge in such excitatory-inhibitory networks: enhancing the efficiency of cortical computations.

The use of standard statistical methods, such as maximum likelihood, is often justified based on their asymptotic properties. For suitably regular models, this theory is standard but, when the model is non-regular, e.g., the support depends on the parameter, these asymptotic properties may be difficult to assess. Recently, an inferential model (IM) framework has been developed that provides valid prior-free probabilistic inference without the need for asymptotic justification. In this paper, we construct an IM for a class of highly non-regular models with parameter-dependent support. This construction requires conditioning, which is facilitated through the solution of a particular differential equation. We prove that the plausibility intervals derived from this IM are exact confidence intervals, and we demonstrate their efficiency in a simulation study.

Non-deductive reasoning systems are often {\em representation dependent}: representing the same situation in two different ways may cause such a system to return two different answers. Some have viewed this as a significant problem. For example, the principle of maximum entropy has been subjected to much criticism due to its representation dependence. There has, however, been almost no work investigating representation dependence. In this paper, we formalize this notion and show that it is not a problem specific to maximum entropy. In fact, we show that any representation-independent probabilistic inference procedure that ignores irrelevant information is essentially entailment, in a precise sense. Moreover, we show that representation independence is incompatible with even a weak default assumption of independence. We then show that invariance under a restricted class of representation changes can form a reasonable compromise between representation independence and other desiderata, and provide a construction of a family of inference procedures that provides such restricted representation independence, using relative entropy.

The distribution semantics is one of the most prominent approaches for the combination of logic programming and probability theory. Many languages follow this semantics, such as Independent Choice Logic, PRISM, pD, Logic Programs with Annotated Disjunctions (LPADs) and ProbLog. When a program contains functions symbols, the distribution semantics is well-defined only if the set of explanations for a query is finite and so is each explanation. Well-definedness is usually either explicitly imposed or is achieved by severely limiting the class of allowed programs. In this paper we identify a larger class of programs for which the semantics is well-defined together with an efficient procedure for computing the probability of queries. Since LPADs offer the most general syntax, we present our results for them, but our results are applicable to all languages under the distribution semantics. We present the algorithm "Probabilistic Inference with Tabling and Answer subsumption" (PITA) that computes the probability of queries by transforming a probabilistic program into a normal program and then applying SLG resolution with answer subsumption. PITA has been implemented in XSB and tested on six domains: two with function symbols and four without. The execution times are compared with those of ProbLog, cplint and CVE, PITA was almost always able to solve larger problems in a shorter time, on domains with and without function symbols.

Probabilistic Latent Tensor Factorization (PLTF) is a recently proposed probabilistic framework for modelling multi-way data. Not only the common tensor factorization models but also any arbitrary tensor factorization structure can be realized by the PLTF framework. This paper presents full Bayesian inference via variational Bayes that facilitates more powerful modelling and allows more sophisticated inference on the PLTF framework. We illustrate our approach on model order selection and link prediction.

This article is from Nucleic Acids Research , volume 41 . Abstract Using nuclear factor-κB (NF-κB) ChIP-Seq data, we present a framework for iterative learning of regulatory networks. For every possible transcription factor-binding site (TFBS)-putatively regulated gene pair, the relative distance and orientation are calculated to learn which TFBSs are most likely to regulate a given gene. Weighted TFBS contributions to putative gene regulation are integrated to derive an NF-κB gene network. A de novo motif enrichment analysis uncovers secondary TFBSs (AP1, SP1) at characteristic distances from NF-κB/RelA TFBSs. Comparison with experimental ENCODE ChIP-Seq data indicates that experimental TFBSs highly correlate with predicted sites. We observe that RelA-SP1-enriched promoters have distinct expression profiles from that of RelA-AP1 and are enriched in introns, CpG islands and DNase accessible sites. Sixteen novel NF-κB/RelA-regulated genes and TFBSs were experimentally validated, including TANK, a negative feedback gene whose expression is NF-κB/RelA dependent and requires a functional interaction with the AP1 TFBSs. Our probabilistic method yields more accurate NF-κB/RelA-regulated networks than a traditional, distance-based approach, confirmed by both analysis of gene expression and increased informativity of Genome Ontology annotations. Our analysis provides new insights into how co-occurring TFBSs and local chromatin context orchestrate activation of NF-κB/RelA sub-pathways differing in biological function and temporal expression patterns.

Information geometric techniques and inductive inference methods hold great promise for solving computational problems of interest in classical and quantum physics, especially with regard to complexity characterization of dynamical systems in terms of their probabilistic description on curved statistical manifolds. In this article, we investigate the possibility of describing the macroscopic behavior of complex systems in terms of the underlying statistical structure of their microscopic degrees of freedom by use of statistical inductive inference and information geometry. We review the Maximum Relative Entropy (MrE) formalism and the theoretical structure of the information geometrodynamical approach to chaos (IGAC) on statistical manifolds. Special focus is devoted to the description of the roles played by the sectional curvature, the Jacobi field intensity and the information geometrodynamical entropy (IGE). These quantities serve as powerful information geometric complexity measures of information-constrained dynamics associated with arbitrary chaotic and regular systems defined on the statistical manifold. Finally, the application of such information geometric techniques to several theoretical models are presented.

Ambiguity results from having limited knowledge of the process that generates outcomes. It is argued that many real-world processes are perceived to be ambiguous; moreover, as Ellsberg demonstrated, this poses problems for theories of probability operationalized via choices amongst gambles. A descriptive model of how people make judgments under ambiguity in tasks where data come from a source of limited, but not exactly known reliability, is proposed. The model assumes an anchoring-and-adjustment process in which data provides the anchor, and adjustments are made for what might have been. The latter is modeled as the result of a mental simulation process that incorporates the unreliability of the source and one's attitude toward ambiguity in the circumstances. A two-parameter model of this process is shown to be consistent with: Keynes' idea of the weight of evidence, the non-additivity of complementary probabilities, current psychological theories of risk, and Ellsberg's original paradox. The model is tested in four experiments at both the individual and group levels. In experiments 1-3, the model is shown to predict judgments quite well; in experiment 4, the inference model is shown to predict choices between gambles. The results and model are then discussed with respect to the importance of ambiguity in assessing perceived uncertainty; the use of cognitive strategies in judgments under ambiguity; the role of ambiguity in risky choice; and extensions of the model. (Author)

This article is from PLoS Computational Biology , volume 10 . Abstract Humans have been shown to combine noisy sensory information with previous experience (priors), in qualitative and sometimes quantitative agreement with the statistically-optimal predictions of Bayesian integration. However, when the prior distribution becomes more complex than a simple Gaussian, such as skewed or bimodal, training takes much longer and performance appears suboptimal. It is unclear whether such suboptimality arises from an imprecise internal representation of the complex prior, or from additional constraints in performing probabilistic computations on complex distributions, even when accurately represented. Here we probe the sources of suboptimality in probabilistic inference using a novel estimation task in which subjects are exposed to an explicitly provided distribution, thereby removing the need to remember the prior. Subjects had to estimate the location of a target given a noisy cue and a visual representation of the prior probability density over locations, which changed on each trial. Different classes of priors were examined (Gaussian, unimodal, bimodal). Subjects' performance was in qualitative agreement with the predictions of Bayesian Decision Theory although generally suboptimal. The degree of suboptimality was modulated by statistical features of the priors but was largely independent of the class of the prior and level of noise in the cue, suggesting that suboptimality in dealing with complex statistical features, such as bimodality, may be due to a problem of acquiring the priors rather than computing with them. We performed a factorial model comparison across a large set of Bayesian observer models to identify additional sources of noise and suboptimality. Our analysis rejects several models of stochastic behavior, including probability matching and sample-averaging strategies. Instead we show that subjects' response variability was mainly driven by a combination of a noisy estimation of the parameters of the priors, and by variability in the decision process, which we represent as a noisy or stochastic posterior.

We identify various situations in probabilistic intelligent systems in which conditionals (rules) as mathematical entities as well as their conditional logic operations are needed. In discussing Bayesian updating procedure and belief function construction, we provide a new method for modeling if...then rules as Boolean elements, and yet, compatible with conditional probability quantifications.

This report describes how the Dirac algebra of quantum mechanics provides for a robust and self-consistent approach to probabilistic inference system modeling and processing. We call such systems Dirac networks and demonstrate how their use: (1) allows an efficient algebraic encoding of the probabilities and distributions for all possible combinations of truth values for the logical variable in an inference system; (2) employs unitary - rotation, time evolution, and translation operators to model influences upon system variable probabilities and their distributions; (3) guarantees system normalization; (4) admits unambiguously defined linear, as well as cyclic, cause and effect relationships; (5) enables the use of the von Neumann entropy as an informational uncertainty measure; and (6) allows for a variety of 'measurement' operators useful for quantifying probabilistic inferences. Dirac networks should have utility in such diverse application areas as data fusion and analysis, dynamic resource allocation, qualitative analysis of complex systems, automated medical diagnostics, and interactive/collaborative decision processes. The approach is illustrated by developing and applying simple Dirac networks to the following representative problems: (a) cruise missile - target allocation decision aiding; (b) genetic disease carrier identification using ancestral evidential information; (c) combat system control methodology trade-off analysis; (d) finding rotational symmetries in a digital image; and (e) fusing observational error profiles. Optical device implementations of several Dirac network components are also briefly discussed.

Probabilistic Logic Programming (PLP), exemplified by Sato and Kameya's PRISM, Poole's ICL, Raedt et al's ProbLog and Vennekens et al's LPAD, is aimed at combining statistical and logical knowledge representation and inference. A key characteristic of PLP frameworks is that they are conservative extensions to non-probabilistic logic programs which have been widely used for knowledge representation. PLP frameworks extend traditional logic programming semantics to a distribution semantics, where the semantics of a probabilistic logic program is given in terms of a distribution over possible models of the program. However, the inference techniques used in these works rely on enumerating sets of explanations for a query answer. Consequently, these languages permit very limited use of random variables with continuous distributions. In this paper, we present a symbolic inference procedure that uses constraints and represents sets of explanations without enumeration. This permits us to reason over PLPs with Gaussian or Gamma-distributed random variables (in addition to discrete-valued random variables) and linear equality constraints over reals. We develop the inference procedure in the context of PRISM; however the procedure's core ideas can be easily applied to other PLP languages as well. An interesting aspect of our inference procedure is that PRISM's query evaluation process becomes a special case in the absence of any continuous random variables in the program. The symbolic inference procedure enables us to reason over complex probabilistic models such as Kalman filters and a large subclass of Hybrid Bayesian networks that were hitherto not possible in PLP frameworks. (To appear in Theory and Practice of Logic Programming).

The abundance of data produced daily from large variety of sources has boosted the need of novel approaches on causal inference analysis from observational data. Observational data often contain noisy or missing entries. Moreover, causal inference studies may require unobserved high-level information which needs to be inferred from other observed attributes. In such cases, inaccuracies of the applied inference methods will result in noisy outputs. In this study, we propose a novel approach for causal inference when one or more key variables are noisy. Our method utilizes the knowledge about the uncertainty of the real values of key variables in order to reduce the bias induced by noisy measurements. We evaluate our approach in comparison with existing methods both on simulated and real scenarios and we demonstrate that our method reduces the bias and avoids false causal inference conclusions in most cases.

In this work we present a flexible, probabilistic and reference-free method of error correction for high throughput DNA sequencing data. The key is to exploit the high coverage of sequencing data and model short sequence outputs as independent realizations of a Hidden Markov Model (HMM). We pose the problem of error correction of reads as one of maximum likelihood sequence detection over this HMM. While time and memory considerations rule out an implementation of the optimal Baum-Welch algorithm (for parameter estimation) and the optimal Viterbi algorithm (for error correction), we propose low-complexity approximate versions of both. Specifically, we propose an approximate Viterbi and a sequential decoding based algorithm for the error correction. Our results show that when compared with Reptile, a state-of-the-art error correction method, our methods consistently achieve superior performances on both simulated and real data sets.

When dealing with incomplete data in statistical learning, or incomplete observations in probabilistic inference, one needs to distinguish the fact that a certain event is observed from the fact that the observed event has happened. Since the modeling and computational complexities entailed by maintaining this proper distinction are often prohibitive, one asks for conditions under which it can be safely ignored. Such conditions are given by the missing at random (mar) and coarsened at random (car) assumptions. In this paper we provide an in-depth analysis of several questions relating to mar/car assumptions. Main purpose of our study is to provide criteria by which one may evaluate whether a car assumption is reasonable for a particular data collecting or observational process. This question is complicated by the fact that several distinct versions of mar/car assumptions exist. We therefore first provide an overview over these different versions, in which we highlight the distinction between distributional and coarsening variable induced versions. We show that distributional versions are less restrictive and sufficient for most applications. We then address from two different perspectives the question of when the mar/car assumption is warranted. First we provide a static analysis that characterizes the admissibility of the car assumption in terms of the support structure of the joint probability distribution of complete data and incomplete observations. Here we obtain an equivalence characterization that improves and extends a recent result by Grunwald and Halpern. We then turn to a procedural analysis that characterizes the admissibility of the car assumption in terms of procedural models for the actual data (or observation) generating process. The main result of this analysis is that the stronger coarsened completely at random (ccar) condition is arguably the most reasonable assumption, as it alone corresponds to data coarsening procedures that satisfy a natural robustness property.

The relationship between ideological thinking and its neurocognitive underpinnings is an interesting one. But how do individuals evaluate new information from experience in order to form impressions about the world and make decisions, and how do ideological beliefs play in on how individuals evaluate new information and update their prior beliefs? Using a probabilistic inference behavioural paradigm, we study decision-making behaviour in relation to the ideological mind. More specifically, we investigate how evoking political affiliation and ideology influence judgment and inference processes. Furthermore, the study aims to determine neural underpinnings of the jumping-to-conclusions (JTC) bias, a behavior related to the tendency of some individuals to form judgements based on insufficient information. Results from a pilot study with nine participants show that information on political affiliation/ideology can trigger differences in the neural processing of stimulus information used to update beliefs.

This document examines probabilistic word problems. Such problems (for example, the Blue/Green Cab Problem) require the subjects to state the probability of a hypothesis H, when they are given information about the base rate P(H), evidence E, and the reliability of the evidence thus the formula p(E/ H). A frequent wrong answer to the problem is to cite the reliability. Data from a study in which the subjects answered after receiving each piece of information are used to evaluate three explanations for the use of the reliability as the answer. Production system models state each hypothesis unambiguously. The hypothesis that subjects consider the base rate to be irrelevant in principle is rejected. The data are consistent with two hypotheses: that subjects confuse P(E/H) with p(H/E), and that they interpolate between the base rate probability and 1.0, but select their response from among nearby numbers which are available in the word problem. Keywords: Statistical inference; Information integration; Production systems; Heuristic strategies.

It is important to distinguish probabilistic reasoning from probabilistic inference. Probabilistic reasoning may concern the manipulation of knowledge of probabilities in the context of decision theory, or it may involve the updating of probabilities in the light of new evidence via Bayes' theorem or some other procedure. Both of these operations are essentially deductive in character. Contrasted with these procedures of manipulating or computing with probabilities, is the use of probabilistic rules of inference: rules that lead from one sentence (or a set of sentences) to another sentence, but do so in a way that need not be truth preserving. One could attempt to get along without probabilistic inference in AI, but it would be very difficult and unnatural. Instances of such rules are several classes of inference rules associated with statistics, and some rules discussed by philosophers. In artificial intelligence the rules that fall into this category are (mainly) default rules; these are not generally construed probabilistic, but obviously default rules that more often lead you astray than to the truth would be poor ones.

Qualitative probabilistic networks (QPNs) are abstractions of influence diagrams that encode constraints on the probabilistic relation among variables rather than precise numeric distributions. Qualitative relations express monotonicity constraints on direct probabilistic relations between variables, or on interactions among the direct relations. Like their numeric counterpart, QPNs facilitate graphical inference: methods for deriving qualitative relations of interest via graphical transformations of the network model. However, query processing in QPNs exhibits computational properties quite different from basic influence diagrams. In particular, the potential for information loss due to the incomplete specification of probabilities poses the new challenge of minimizing ambiguity. Analysis of the properties of QPN transformations reveals several characteristics of admissible graphical inference procedures. Keywords: Networks, Interference, Variables.

Ambiguity results from having limited knowledge of the process that generates outcomes. It is argued that many real-world processes are perceived to be ambigious; moreover, as Ellsberg demonstrated, this poses problems for theories of probability operationalized via choices amongst gambles. A descriptive model of how people make judgments under ambiguity is proposed. The model assumes an anchoring-and-adjustment process in which an initial estimate provides the anchor, and adjustments are made for what might be. The latter is modeled as the result of a mental simulation process where the size of the simulation is a function of the amount of ambiguity, and differential weighting of imagined probabilities reflects one's attitude toward ambiguity. A two-parameter model of this process is shown to be consistent with: Ellsberg's original paradox, the non-additivity of complementary probabilities, current psycho-loical theories of risk, and Keynes' idea of the weight of evidence. The model is tested in four experiments involving boht individual and group analyses. In experiments 1 and 2, the model is shown to predict judgments quite well; in experiment 3, the inference model is shown to predict choices between gambles; experiment 4 shows how buying and selling prices for insurance are systematically influenced by one's attitude toward ambiguity.

A relatively large body of research indicates that people are conservative processors of probabilistic information. Recent attention has focussed on two possible explanations of this phenomenon. The misaggregation hypothesis depicts conservatism as an inability to properly combine the information in sequence of data. The other explanation suggests conservatism is the result of a response bias: the avoidance of extreme odds or probability judgments. Two experiments explored the use of a specific response, average certainty that was devised to thwart conservatism caused by either a response bias or a certain form of misaggregation. Use of appropriate instructions and response scales made the average certainty judgments good subjective assessments of the arithemetic mean likelihood ratio which could then be used in the appropriate form of Bayes' Theorem to calculate posterior odds. These judgments seemed unlikely to be affected by a response bias since extreme responses were not needed. In addition, research has suggested that people are more likely to aggregate information by averaging than by adding or multiplying, so misaggregation may be exhibited only in specific forms of aggregation and may not be present in averaging.

Many Artificial Intelligence (AI) tasks, such as natural language processing, commonsense reasoning and vision, could be naturally modeled by a language and associated inference engine using both relational (first-order) predicates and probabilistic information. While logic has been the basis for much AI development and is a powerful framework for using relational predicates, its lack of representation for probabilistic knowledge severely limits its application to many tasks. Graphical models and Machine Learning, on the other hand, can capture much of probabilistic reasoning but lack convenient means for using relational predicates. In the last fifteen years, many frameworks have been proposed for merging those two approaches but have mainly been probabilistic logic languages resorting to propositionalization of relational predicates (and, as a consequence, ordinary graphical models inference). This has the severe disadvantage of ignoring the relational structure of the model and potentially causing exponential blowups in inference time. I will talk about my work in integrating logic and probabilistic inference in a more seamless way. This includes Lifted First-Order Probabilistic Inference, a way of performing inference directly on first-order representation, without propositionalization, and work on DBLOG (Dynamic Bayesian Logic), an extension of BLOG (Bayesian Logic, by Milch and Russell) for temporal models such as data association and activity recognition. I will conclude with what I see as important future directions in this field. ©2008 Microsoft Corporation. All rights reserved.

Many Artificial Intelligence (AI) tasks, such as natural language processing, commonsense reasoning and vision, could be naturally modeled by a language and associated inference engine using both relational (first-order) predicates and probabilistic information. While logic has been the basis for much AI development and is a powerful framework for using relational predicates, its lack of representation for probabilistic knowledge severely limits its application to many tasks. Graphical models and Machine Learning, on the other hand, can capture much of probabilistic reasoning but lack convenient means for using relational predicates. In the last fifteen years, many frameworks have been proposed for merging those two approaches but have mainly been probabilistic logic languages resorting to propositionalization of relational predicates (and, as a consequence, ordinary graphical models inference). This has the severe disadvantage of ignoring the relational structure of the model and potentially causing exponential blowups in inference time. I will talk about my work in integrating logic and probabilistic inference in a more seamless way. This includes Lifted First-Order Probabilistic Inference, a way of performing inference directly on first-order representation, without propositionalization, and work on DBLOG (Dynamic Bayesian Logic), an extension of BLOG (Bayesian Logic, by Milch and Russell) for temporal models such as data association and activity recognition. I will conclude with what I see as important future directions in this field. ©2008 Microsoft Corporation. All rights reserved.

Since the appearance of the influential article by McCarthy and Hays, few people have tried to use probabilities as a basis for non-monotonic inference. One reason, perhaps the main one, is that probabilistic inference easily yields inconsistent bodies of knowledge, as is revealed by the lottery paradox. Here we establish three things : First that standard systems of non- monotonic reasoning (default logic, non-monotonic logic, and circumscription) fall prey to the same lottery-like difficulties as does probabilistic inference. Second, that probabilistic inference provides equally plausible treatment of the standard examples of non-monotonic reasoning. Third, that the inconsistency threatened by the lottery paradox is a petty hodgoblin, and need not in any way interfere with the use of beliefs in planning and design.

The promise of lifted probabilistic inference is to carry out probabilistic inference in a relational probabilistic model without needing to reason about each individual separately (grounding out the representation) by treating the undistinguished individuals as a block. Current exact methods still need to ground out in some cases, typically because the representation of the intermediate results is not closed under the lifted operations. We set out to answer the question as to whether there is some fundamental reason why lifted algorithms would need to ground out undifferentiated individuals. We have two main results: (1) We completely characterize the cases where grounding is polynomial in a population size, and show how we can do lifted inference in time polynomial in the logarithm of the population size for these cases. (2) For the case of no-argument and single-argument parametrized random variables where the grounding is not polynomial in a population size, we present lifted inference which is polynomial in the population size whereas grounding is exponential. Neither of these cases requires reasoning separately about the individuals that are not explicitly mentioned.

Exchangeability is a central notion in statistics and probability theory. The assumption that an infinite sequence of data points is exchangeable is at the core of Bayesian statistics. However, finite exchangeability as a statistical property that renders probabilistic inference tractable is less well-understood. We develop a theory of finite exchangeability and its relation to tractable probabilistic inference. The theory is complementary to that of independence and conditional independence. We show that tractable inference in probabilistic models with high treewidth and millions of variables can be understood using the notion of finite (partial) exchangeability. We also show that existing lifted inference algorithms implicitly utilize a combination of conditional independence and partial exchangeability.

Twitter provides an open and rich source of data for studying human behaviour at scale and is widely used in social and network sciences. However, a major criticism of Twitter data is that demographic information is largely absent. Enhancing Twitter data with user ages would advance our ability to study social network structures, information flows and the spread of contagions. Approaches toward age detection of Twitter users typically focus on specific properties of tweets, e.g., linguistic features, which are language dependent. In this paper, we devise a language-independent methodology for determining the age of Twitter users from data that is native to the Twitter ecosystem. The key idea is to use a Bayesian framework to generalise ground-truth age information from a few Twitter users to the entire network based on what/whom they follow. Our approach scales to inferring the age of 700 million Twitter accounts with high accuracy.

An objective operational theory of probabilistic parametric inference is formulated without invoking the so-called non-informative prior probability distributions.

Animals perform near-optimal probabilistic inference in a wide range of psychophysical tasks. Probabilistic inference requires trial-to-trial representation of the uncertainties associated with task variables and subsequent use of this representation. Previous work has implemented such computations using neural networks with hand-crafted and task-dependent operations. We show that generic neural networks trained with a simple error-based learning rule perform near-optimal probabilistic inference in nine common psychophysical tasks. In a probabilistic categorization task, error-based learning in a generic network simultaneously explains a monkey's learning curve and the evolution of qualitative aspects of its choice behavior. In all tasks, the number of neurons required for a given level of performance grows sub-linearly with the input population size, a substantial improvement on previous implementations of probabilistic inference. The trained networks develop a novel sparsity-based probabilistic population code. Our results suggest that probabilistic inference emerges naturally in generic neural networks trained with error-based learning rules.

We introduce and demonstrate a new approach to inference in expressive probabilistic programming languages based on particle Markov chain Monte Carlo. Our approach is simple to implement and easy to parallelize. It applies to Turing-complete probabilistic programming languages and supports accurate inference in models that make use of complex control flow, including stochastic recursion. It also includes primitives from Bayesian nonparametric statistics. Our experiments show that this approach can be more efficient than previously introduced single-site Metropolis-Hastings methods.

Probabilistic logic programs are logic programs in which some of the facts are annotated with probabilities. This paper investigates how classical inference and learning tasks known from the graphical model community can be tackled for probabilistic logic programs. Several such tasks such as computing the marginals given evidence and learning from (partial) interpretations have not really been addressed for probabilistic logic programs before. The first contribution of this paper is a suite of efficient algorithms for various inference tasks. It is based on a conversion of the program and the queries and evidence to a weighted Boolean formula. This allows us to reduce the inference tasks to well-studied tasks such as weighted model counting, which can be solved using state-of-the-art methods known from the graphical model and knowledge compilation literature. The second contribution is an algorithm for parameter estimation in the learning from interpretations setting. The algorithm employs Expectation Maximization, and is built on top of the developed inference algorithms. The proposed approach is experimentally evaluated. The results show that the inference algorithms improve upon the state-of-the-art in probabilistic logic programming and that it is indeed possible to learn the parameters of a probabilistic logic program from interpretations.

We study the extent to which we can infer users' geographical locations from social media. Location inference from social media can benefit many applications, such as disaster management, targeted advertising, and news content tailoring. In recent years, a number of algorithms have been proposed for identifying user locations on social media platforms such as Twitter and Facebook from message contents, friend networks, and interactions between users. In this paper, we propose a novel probabilistic model based on factor graphs for location inference that offers several unique advantages for this task. First, the model generalizes previous methods by incorporating content, network, and deep features learned from social context. The model is also flexible enough to support both supervised learning and semi-supervised learning. Second, we explore several learning algorithms for the proposed model, and present a Two-chain Metropolis-Hastings (MH+) algorithm, which improves the inference accuracy. Third, we validate the proposed model on three different genres of data - Twitter, Weibo, and Facebook - and demonstrate that the proposed model can substantially improve the inference accuracy (+3.3-18.5% by F1-score) over that of several state-of-the-art methods.

When doing inference in ProbLog, a probabilistic extension of Prolog, we extend SLD resolution with some additional bookkeeping. This additional information is used to compute the probabilistic results for a probabilistic query. In Prolog's SLD, goals are nested very naturally. In ProbLog's SLD, nesting probabilistic queries interferes with the probabilistic bookkeeping. In order to support nested probabilistic inference we propose the notion of a parametrised ProbLog engine. Nesting becomes possible by suspending and resuming instances of ProbLog engines. With our approach we realise several extensions of ProbLog such as meta-calls, negation, and answers of probabilistic goals.

Today, many different probabilistic programming languages exist and even more inference mechanisms for these languages. Still, most logic programming based languages use backward reasoning based on SLD resolution for inference. While these methods are typically computationally efficient, they often can neither handle infinite and/or continuous distributions, nor evidence. To overcome these limitations, we introduce distributional clauses, a variation and extension of Sato's distribution semantics. We also contribute a novel approximate inference method that integrates forward reasoning with importance sampling, a well-known technique for probabilistic inference. To achieve efficiency, we integrate two logic programming techniques to direct forward sampling. Magic sets are used to focus on relevant parts of the program, while the integration of backward reasoning allows one to identify and avoid regions of the sample space that are inconsistent with the evidence.

We describe a variational approximation method for efficient inference in large-scale probabilistic models. Variational methods are deterministic procedures that provide approximations to marginal and conditional probabilities of interest. They provide alternatives to approximate inference methods based on stochastic sampling or search. We describe a variational approach to the problem of diagnostic inference in the `Quick Medical Reference' (QMR) network. The QMR network is a large-scale probabilistic graphical model built on statistical and expert knowledge. Exact probabilistic inference is infeasible in this model for all but a small set of cases. We evaluate our variational inference algorithm on a large set of diagnostic test cases, comparing the algorithm to a state-of-the-art stochastic sampling method.

Contents: Optimal Inspection Policy for a Deteriorating Device; Some Multivariate Life Distributions.

Many Artificial Intelligence (AI) tasks, such as natural language processing, commonsense reasoning and vision, could be naturally modeled by a language and associated inference engine using both relational (first-order) predicates and probabilistic information. While logic has been the basis for much AI development and is a powerful framework for using relational predicates, its lack of representation for probabilistic knowledge severely limits its application to many tasks. Graphical models and Machine Learning, on the other hand, can capture much of probabilistic reasoning but lack convenient means for using relational predicates. In the last fifteen years, many frameworks have been proposed for merging those two approaches but have mainly been probabilistic logic languages resorting to propositionalization of relational predicates (and, as a consequence, ordinary graphical models inference). This has the severe disadvantage of ignoring the relational structure of the model and potentially causing exponential blowups in inference time. I will talk about my work in integrating logic and probabilistic inference in a more seamless way. This includes Lifted First-Order Probabilistic Inference, a way of performing inference directly on first-order representation, without propositionalization, and work on DBLOG (Dynamic Bayesian Logic), an extension of BLOG (Bayesian Logic, by Milch and Russell) for temporal models such as data association and activity recognition. I will conclude with what I see as important future directions in this field. ©2008 Microsoft Corporation. All rights reserved.

In this work, the task of wide-area indoor people detection in a network of depth sensors is examined. In particular, we investigate how the redundant and complementary multi-view information, including the temporal context, can be jointly leveraged to improve the detection performance. We recast the problem of multi-view people detection in overlapping depth images as an inverse problem and present a generative probabilistic framework to jointly exploit the temporal multi-view image evidence.

In this work, the task of wide-area indoor people detection in a network of depth sensors is examined. In particular, we investigate how the redundant and complementary multi-view information, including the temporal context, can be jointly leveraged to improve the detection performance. We recast the problem of multi-view people detection in overlapping depth images as an inverse problem and present a generative probabilistic framework to jointly exploit the temporal multi-view image evidence.

This article is from BMC Bioinformatics , volume 14 . Abstract Background: Since the advent of microarray technology, numerous methods have been devised to infer gene regulatory relationships from gene expression data. Many approaches that infer entire regulatory networks. This produces results that are rich in information and yet so complex that they are often of limited usefulness for researchers. One alternative unit of regulatory interactions is a linear path between genes. Linear paths are more comprehensible than networks and still contain important information. Such paths can be extracted from inferred regulatory networks or inferred directly. Since criteria for inferring networks generally differs from criteria for inferring paths, indirect and direct inference of paths may achieve different results. Results: This paper explores a strategy to infer linear pathways by converting the path inference problem into a shortest-path problem. The edge weights used are the negative log-transformed probabilities of directness derived from the posterior joint distributions of pairwise mutual information between gene expression levels. Directness is inferred using the data processing inequality. The method was designed with two goals. One is to achieve better accuracy in path inference than extraction of paths from inferred networks. The other is to facilitate priorization of interactions for laboratory validation. A method is proposed for achieving this by ranking paths according to the joint probability of directness of each path's edges. The algorithm is evaluated using simulated expression data and is compared to extraction of shortest paths from networks inferred by two alternative methods, ARACNe and a minimum spanning tree algorithm. Conclusions: Direct path inference appears to achieve accuracy competitive with that obtained by extracting paths from networks inferred by the other methods. Preliminary exploration of the use of joint edge probabilities to rank paths is largely inconclusive. Suggestions for a better framework for such comparisons are discussed.

We present SGDPLL(T), an algorithm that solves (among many other problems) probabilistic inference modulo theories, that is, inference problems over probabilistic models defined via a logic theory provided as a parameter (currently, propositional, equalities on discrete sorts, and inequalities, more specifically difference arithmetic, on bounded integers). While many solutions to probabilistic inference over logic representations have been proposed, SGDPLL(T) is simultaneously (1) lifted, (2) exact and (3) modulo theories, that is, parameterized by a background logic theory. This offers a foundation for extending it to rich logic languages such as data structures and relational data. By lifted, we mean algorithms with constant complexity in the domain size (the number of values that variables can take). We also detail a solver for summations with difference arithmetic and show experimental results from a scenario in which SGDPLL(T) is much faster than a state-of-the-art probabilistic solver.

We numerically investigate a mean-field Bayesian approach with the assistance of the Markov chain Monte Carlo method to estimate motion velocity fields and probabilistic models simultaneously in consecutive digital images described by spatio-temporal Markov random fields. Preliminary to construction of our procedure, we find that mean-field variables in the iteration diverge due to improper normalization factor of regularization terms appearing in the posterior. To avoid this difficulty, we rescale the regularization term by introducing a scaling factor and optimizing it by means of minimization of the mean-square error. We confirm that the optimal scaling factor stabilizes the mean-field iterative process of the motion velocity estimation. We next attempt to estimate the optimal values of hyper-parameters including the regularization term, which define our probabilistic model macroscopically, by using the Boltzmann-machine type learning algorithm based on gradient descent of marginal likelihood (type-II likelihood) with respect to the hyper-parameters. In our framework, one can estimate both the probabilistic model (hyper-parameters) and motion velocity fields simultaneously. We find that our motion estimation is much better than the result obtained by Zhang and Hanouer (1995) in which the hyper-parameters are set to some ad-hoc values without any theoretical justification.

Statistical relational models provide compact encodings of probabilistic dependencies in relational domains, but result in highly intractable graphical models. The goal of lifted inference is to carry out probabilistic inference without needing to reason about each individual separately, by instead treating exchangeable, undistinguished objects as a whole. In this paper, we study the domain recursion inference rule, which, despite its central role in early theoretical results on domain-lifted inference, has later been believed redundant. We show that this rule is more powerful than expected, and in fact significantly extends the range of models for which lifted inference runs in time polynomial in the number of individuals in the domain. This includes an open problem called S4, the symmetric transitivity model, and a first-order logic encoding of the birthday paradox. We further identify new classes S2FO2 and S2RU of domain-liftable theories, which respectively subsume FO2 and recursively unary theories, the largest classes of domain-liftable theories known so far, and show that using domain recursion can achieve exponential speedup even in theories that cannot fully be lifted with the existing set of inference rules.

This paper describes several applications in astronomy and cosmology that are addressed using probabilistic modelling and statistical inference.

This document, Bayesian Inference for NASA Probabilistic Risk and Reliability Analysis, is intended to provide guidelines for the collection and evaluation of risk and reliability-related data. It is aimed at scientists and engineers familiar with risk and reliability methods and provides a hands-on approach to the investigation and application of a variety of risk and reliability data assessment methods, tools, and techniques. This document provides both: A broad perspective on data analysis collection and evaluation issues. A narrow focus on the methods to implement a comprehensive information repository. The topics addressed herein cover the fundamentals of how data and information are to be used in risk and reliability analysis models and their potential role in decision making. Understanding these topics is essential to attaining a risk informed decision making environment that is being sought by NASA requirements and procedures such as 8000.4 (Agency Risk Management Procedural Requirements), NPR 8705.05 (Probabilistic Risk Assessment Procedures for NASA Programs and Projects), and the System Safety requirements of NPR 8715.3 (NASA General Safety Program Requirements).