Downloads & Free Reading Options - Results

This study investigated statistical methods for identifying errors in Bayesian networks (BN) with latent variables, as found in intelligent cognitive assessments. BN, commonly used in artificial intelligence systems, are promising mechanisms for scoring constructed-response examinations. The success of an intelligent assessment or tutoring system depends on the adequacy of the student model, representing the relationship between the unobservable cognitive variables of interest (thetas) and the observable features of task performance (x) with the probability model for x given theta expressed as a BN. The method for model fit analyses investigated in this study is appropriate for several uses in cognitive assessment. Data were generated for posited models to reflect the true BN model and several discrepancies from the true model. The study examined three indices: (1) Weaver's Surprise Index (Weaver, 1948); (2) Good's Logarithmic Score (Good, 1952); and (3) the Ranked Probability Score (Epstein, 1969). Simulation studies offer promise for the usefulness of the Ranked Probability Score and Weaver's Surprise Index as global measures and node measures to detect specific types of modeling errors in the latent structure of BNs. The introduction of this methodology and the emphasis on model criticism of BNs with latent variables provide a means of maximizing the accuracy and usefulness of BN models for a variety of applications. (Contains 4 tables, 9 figures, and 26 references.) (SLD)

The performed project focused on a new paradigm of planning systems that are based on a combination of Bayesian networks and structural equation models. We focused on theoretical issues that surround combining the two in a practical planning system, developing the foundations for, and building a prototype of such system. The approach and the system built allow for efficient, yet normatively correct, treatment of various types of information, uncertainty, and utility. It is especially powerful in complex situations where the available information is heterogeneous and consists of a mixture of deterministic and uncertain relationships among discrete and continuous variables. Our main contributions are: (1) several fast state of the art stochastic sampling algorithms for approximate inference in graphical models, (2) treatment of reversible causal mechanisms for causal reasoning in graphical models, (3) a scheme for interactive construction of causal graphical models based on causal mechanisms, (4) an algorithm for learning graphical models from data, and (5) a prototype of the system, used by over 2,300 people world-wide.

Our research question was whether we could develop a feasible technique, using Bayesian networks, to diagnose gaps in student knowledge. Thirty-four college-age participants completed tasks designed to measure conceptual knowledge, procedural knowledge, and problem-solving skills related to circuit analysis. A Bayesian network was used to model the knowledge dependencies among the circuit analysis concepts. Preliminary results suggested that the Bayesian network was generally working as intended. When high- and low-performing groups were formed on the basis of posterior probabilities, significant group differences were found favoring the high-performing group with respect to circuit definitions and circuit analysis problems, for both actual and self-assessments, and higher major GPA. The Bayesian network was able to predict participants' performance on a problem-solving item on average 75% of the time. The findings of this study are promising for our long-term goal of developing scalable and feasible online automated reasoning techniques to diagnose student knowledge gaps. (Contains 12 tables and 2 figures.) [Appended are: (1) Node-Voltage Analysis Problem-Solving Procedure (Kaiser, 2003); and (2) Bayesian Network.]

This research is about studying and comparing two different ways of building complex networks. The main goal of our study is to find an effective way to build networks, particularly when we have fewer observations than variables. We construct networks estimating the partial correlation coefficient on Classic Statistics (Inverse Method) and on Bayesian Statistics (Normal - Inverse Wishart conjugate prior). In this current work, in order to solve the problem of having less observations than variables, we propose a new methodology called local partial correlation, which consists of selecting, for each pair of variables, the other variables most correlated to the pair.We applied these methods on simulated data and compared them through ROC curves. The most attractive result is that, even though it has high computational costs, to use Bayesian inference on trees is better when we have less observations than variables. In other cases, both approaches present satisfactory results.

We are concerned with modeling the strength of links in networks by taking into account how often those links are used. Link usage is a strong indicator of how closely two nodes are related, but existing network models in Bayesian Statistics and Machine Learning are able to predict only wether a link exists at all. As priors for latent attributes of network nodes we explore the Chinese Restaurant Process (CRP) and a multivariate Gaussian with fixed dimensionality. The model is applied to a social network dataset and a word coocurrence dataset.

Bayesian graphical modeling provides an appealing way to obtain uncertainty estimates when inferring network structures, and much recent progress has been made for Gaussian models. These models have been used extensively in applications to gene expression data, even in cases where there appears to be significant deviations from the Gaussian model. For more robust inferences, it is natural to consider extensions to t-distribution models. We argue that the classical multivariate t-distribution, defined using a single latent Gamma random variable to rescale a Gaussian random vector, is of little use in highly multivariate settings, and propose other, more flexible t-distributions. Using an independent Gamma-divisor for each component of the random vector defines what we term the alternative t-distribution. The associated model allows one to extract information from highly multivariate data even when most experiments contain outliers for some of their measurements. However, the use of this alternative model comes at increased computational cost and imposes constraints on the achievable correlation structures, raising the need for a compromise between the classical and alternative models. To this end we propose the use of Dirichlet processes for adaptive clustering of the latent Gamma-scalars, each of which may then divide a group of latent Gaussian variables. Dirichlet processes are commonly used to cluster independent observations; here they are used instead to cluster the dependent components of a single observation. The resulting Dirichlet t-distribution interpolates naturally between the two extreme cases of the classical and alternative t-distributions and combines more appealing modeling of the multivariate dependence structure with favorable computational properties.

Unsupervised estimation of latent variable models is a fundamental problem central to numerous applications of machine learning and statistics. This work presents a principled approach for estimating broad classes of such models, including probabilistic topic models and latent linear Bayesian networks, using only second-order observed moments. The sufficient conditions for identifiability of these models are primarily based on weak expansion constraints on the topic-word matrix, for topic models, and on the directed acyclic graph, for Bayesian networks. Because no assumptions are made on the distribution among the latent variables, the approach can handle arbitrary correlations among the topics or latent factors. In addition, a tractable learning method via $\ell_1$ optimization is proposed and studied in numerical experiments.

Unsupervised estimation of latent variable models is a fundamental problem central to numerous applications of machine learning and statistics. This work presents a principled approach for estimating broad classes of such models, including probabilistic topic models and latent linear Bayesian networks, using only second-order observed moments. The sufficient conditions for identifiability of these models are primarily based on weak expansion constraints on the topic-word matrix, for topic models, and on the directed acyclic graph, for Bayesian networks. Because no assumptions are made on the distribution among the latent variables, the approach can handle arbitrary correlations among the topics or latent factors. In addition, a tractable learning method via $\ell_1$ optimization is proposed and studied in numerical experiments.

Directed acyclic graph (DAG) models, also called Bayesian networks, impose conditional independence constraints on a multivariate probability distribution, and are widely used in probabilistic reasoning, machine learning and causal inference. If latent variables are included in such a model, then the set of possible marginal distributions over the remaining (observed) variables is generally complex, and not represented by any DAG. Larger classes of mixed graphical models, which use multiple edge types, have been introduced to overcome this; however, these classes do not represent all the models which can arise as margins of DAGs. In this paper we show that this is because ordinary mixed graphs are fundamentally insufficiently rich to capture the variety of marginal models. We introduce a new class of hyper-graphs, called mDAGs, and a latent projection operation to obtain an mDAG from the margin of a DAG. We show that each distinct marginal of a DAG model is represented by at least one mDAG, and provide graphical results towards characterizing when two such marginal models are the same. Finally we show that mDAGs correctly capture the marginal structure of causally-interpreted DAGs under interventions on the observed variables.

An important part of the analysis of bio-molecular networks is to detect different functional units. Different functions are reflected in a different evolutionary dynamics, and hence in different statistical characteristics of network parts. In this sense, the {\em global statistics} of a biological network, e.g., its connectivity distribution, provides a background, and {\em local deviations} from this background signal functional units. In the computational analysis of biological networks, we thus typically have to discriminate between different statistical models governing different parts of the dataset. The nature of these models depends on the biological question asked. We illustrate this rationale here with three examples: identification of functional parts as highly connected \textit{network clusters}, finding \textit{network motifs}, which occur in a similar form at different places in the network, and the analysis of \textit{cross-species network correlations}, which reflect evolutionary dynamics between species.

A substantial focus of research in molecular biology are gene regulatory networks: the set of transcription factors and target genes which control the involvement of different biological processes in living cells. Previous statistical approaches for identifying gene regulatory networks have used gene expression data, ChIP binding data or promoter sequence data, but each of these resources provides only partial information. We present a Bayesian hierarchical model that integrates all three data types in a principled variable selection framework. The gene expression data are modeled as a function of the unknown gene regulatory network which has an informed prior distribution based upon both ChIP binding and promoter sequence data. We also present a variable weighting methodology for the principled balancing of multiple sources of prior information. We apply our procedure to the discovery of gene regulatory relationships in Saccharomyces cerevisiae (Yeast) for which we can use several external sources of information to validate our results. Our inferred relationships show greater biological relevance on the external validation measures than previous data integration methods. Our model also estimates synergistic and antagonistic interactions between transcription factors, many of which are validated by previous studies. We also evaluate the results from our procedure for the weighting for multiple sources of prior information. Finally, we discuss our methodology in the context of previous approaches to data integration and Bayesian variable selection.

Stochastic processes that involve the creation of objects and relations over time are widespread, but relatively poorly studied. For example, accurate fault diagnosis in factory assembly processes requires inferring the probabilities of erroneous assembly operations, but doing this efficiently and accurately is difficult. Modeled as dynamic Bayesian networks, these processes have discrete variables with very large domains and extremely high dimensionality. In this paper, we introduce relational dynamic Bayesian networks (RDBNs), which are an extension of dynamic Bayesian networks (DBNs) to first-order logic. RDBNs are a generalization of dynamic probabilistic relational models (DPRMs), which we had proposed in our previous work to model dynamic uncertain domains. We first extend the Rao-Blackwellised particle filtering described in our earlier work to RDBNs. Next, we lift the assumptions associated with Rao-Blackwellization in RDBNs and propose two new forms of particle filtering. The first one uses abstraction hierarchies over the predicates to smooth the particle filters estimates. The second employs kernel density estimation with a kernel function specifically designed for relational domains. Experiments show these two methods greatly outperform standard particle filtering on the task of assembly plan execution monitoring.

Powerful ideas recently appeared in the literature are adjusted and combined to design improved samplers for Bayesian exponential random graph models. Different forms of adaptive Metropolis-Hastings proposals (vertical, horizontal and rectangular) are tested and combined with the Delayed rejection (DR) strategy with the aim of reducing the variance of the resulting Markov chain Monte Carlo estimators for a given computational time. In the examples treated in this paper the best combination, namely horizontal adaptation with delayed rejection, leads to a variance reduction that varies between 92% and 144% relative to the adaptive direction sampling approximate exchange algorithm of Caimo and Friel (2011). These results correspond to an increased performance which varies from 10% to 94% if we take simulation time into account. The highest improvements are obtained when highly correlated posterior distributions are considered.

In this paper we consider a network of agents monitoring a spatially distributed arrival process. Each node measures the number of arrivals seen at its monitoring point in a given time-interval with the objective of estimating the unknown local arrival rate. We propose an asynchronous distributed approach based on a Bayesian model with unknown hyperparameter, where each node computes the minimum mean square error (MMSE) estimator of its local arrival rate in a distributed way. As a result, the estimation at each node "optimally" fuses the information from the whole network through a distributed optimization algorithm. Moreover, we propose an ad-hoc distributed estimator, based on a consensus algorithm for time-varying and directed graphs, which exhibits reduced complexity and exponential convergence. We analyze the performance of the proposed distributed estimators, showing that they: (i) are reliable even in presence of limited local data, and (ii) improve the estimation accuracy compared to the purely decentralized setup. Finally, we provide a statistical characterization of the proposed estimators. In particular, for the ad-hoc estimator, we show that as the number of nodes goes to infinity its mean square error converges to the optimal one. Numerical Monte Carlo simulations confirm the theoretical characterization and highlight the appealing performances of the estimators.

In this paper we consider a network of agents monitoring a spatially distributed arrival process. Each node measures the number of arrivals seen at its monitoring point in a given time-interval with the objective of estimating the unknown local arrival rate. We propose an asynchronous distributed approach based on a Bayesian model with unknown hyperparameter, where each node computes the minimum mean square error (MMSE) estimator of its local arrival rate in a distributed way. As a result, the estimation at each node "optimally" fuses the information from the whole network through a distributed optimization algorithm. Moreover, we propose an ad-hoc distributed estimator, based on a consensus algorithm for time-varying and directed graphs, which exhibits reduced complexity and exponential convergence. We analyze the performance of the proposed distributed estimators, showing that they: (i) are reliable even in presence of limited local data, and (ii) improve the estimation accuracy compared to the purely decentralized setup. Finally, we provide a statistical characterization of the proposed estimators. In particular, for the ad-hoc estimator, we show that as the number of nodes goes to infinity its mean square error converges to the optimal one. Numerical Monte Carlo simulations confirm the theoretical characterization and highlight the appealing performances of the estimators.

Recently developed techniques have made it possible to quickly learn accurate probability density functions from data in low-dimensional continuous spaces. In particular. mixtures of Gaussians can be fitted to data very quickly using an accelerated EM algorithm that employs multiresolution kd-trees (Moore 1999). In this paper, we propose a kind of Bayesian network in which low-dimensional mixtures of Gaussians over different subsets of the domain's variables are combined into a coherent joint probability model over the entire domain. The network is also capable of modelling complex dependencies between discrete variables and continuous variables without requiring discretization of the continuous variables. We present efficient heuristic algorithms for automatically learning these networks from data and perform comparative experiments illustrating how well these networks model real scientific data and synthetic data. We also briefly discuss some possible improvements to the networks. as well as their possible application to anomaly detection, classification probabilistic inference, and compression.

Correctly identifying tracks is a difficult but important capability for U.S. Navy ships and aircraft. It is difficult because of the inherent uncertainty, complexity, and short timelines involved. It is important because the price of failure is missed or civilian engagements and fratricide. Today, Navy ships and aircraft primarily use an If-Then rule-based system in evaluating radar and IFF information to perform Combat Identification (CID). To cope with the uncertainty and complexity of CID, Bayesian Networks have been suggested to integrate radar, IFF, and other lower quality sources to perform the identification determination. The goal of this project is to show that Bayesian Networks can be used to support CID investment decisions. Two investments, a new sensor and good maintenance, were compared in a difficult CID scenario in four different environments. The paper applies techniques from decision analysis and Bayesian networks to address the challenges of CID. The CID network was developed using good knowledge engineering practices.

The performed project focused on a new paradigm of planning systems that are based on a combination of Bayesian networks and structural equation models. We focused on theoretical issues that surround combining the two in a practical planning system, developing the foundations for, and building a prototype of such system. The approach and the system built allow for efficient, yet normatively correct, treatment of various types of information, uncertainty, and utility. It is especially powerful in complex situations where the available information is heterogeneous and consists of a mixture of deterministic and uncertain relationships among discrete and continuous variables. Our main contributions are: (1) two state of the art stochastic sampling algorithm for approximate inference in graphical models, both (2) analysis of problems related to combining probabilistic information, (3) an module for interactive construction of causal graphical models and search for opportunities, (4) algorithm for learning graphical models from small data sets, and (5) a prototype of the system, used by over 5,000 people world-wide.

Many organizations require quality improvement initiatives to be based on quantified business cases. This leads some organizations to start measurement programs to collect data about current performance-a lengthy and expensive process that requires a strong commitment from management. This report describes a collaboration between the Software Engineering Institute and Ericsson Research and Development, The Netherlands, to build a business case using high maturity measurement approaches that require limited measurement effort. For this project, a Bayesian belief network (BBN) and Monte Carlo simulation were combined to build a business case for quality improvement. Using a BBN gave quick insight into potential areas of improvement based on relevant quality factors and the current performance level of the organization. Monte Carlo simulation enabled a detailed calculation of the likely business results in the areas of potential improvement. This approach led to the decision to implement agile methods to improve the quality of requirements.

Complex interactions between genes or proteins contribute a substantial part to phenotypic evolution. Here we develop an evolutionarily grounded method for the cross-species analysis of interaction networks by {\em alignment}, which maps bona fide functional relationships between genes in different organisms. Network alignment is based on a scoring function measuring mutual similarities between networks taking into account their interaction patterns as well as sequence similarities between their nodes. High-scoring alignments and optimal alignment parameters are inferred by a systematic Bayesian analysis. We apply this method to analyze the evolution of co-expression networks between human and mouse. We find evidence for significant conservation of gene expression clusters and give network-based predictions of gene function. We discuss examples where cross-species functional relationships between genes do not concur with sequence similarity.

We propose a new inequality that we call the conditional ageing inequality (CAIN). The CAIN is a slight generalization to non-equilibrium situations of the Second Law of thermodynamics. The goal of this paper is to study the consequences of the CAIN. We use the CAIN to discuss Maxwell demon processes (i.e., thermodynamic processes with feedback.) In particular, we apply the CAIN to four cases of the Szilard engine: for a classical or a quantum system with either one or two correlated particles. Besides proposing this new inequality that we call the CAIN, another novel feature of this paper is that we use quantum Bayesian networks for our analysis of Maxwell demon processes.

Network data are increasingly collected along with other variables of interest. Our motivation is drawn from neurophysiology studies measuring brain connectivity networks for a sample of individuals along with their membership to a low or high creative reasoning group. It is of paramount importance to develop statistical methods for testing of global and local changes in the structural interconnections among brain regions across groups. We develop a general Bayesian procedure for inference and testing of group differences in the network structure, which relies on a nonparametric representation for the conditional probability mass function associated with a network-valued random variable. By leveraging a mixture of low-rank factorizations, we allow simple global and local hypothesis testing adjusting for multiplicity. An efficient Gibbs sampler is defined for posterior computation. We provide theoretical results on the flexibility of the model and assess testing performance in simulations. The approach is applied to provide novel insights on the relationships between human brain networks and creativity.

Since the use of information and communication technologies has now become indispensable for the development of human activity, governments have assumed the responsibility of ensuring access to them, as is the case in Mexico, generating a series of public policies aimed at that end. However, these politics have not generated the expected results since there are yet large differences to internet connectivity. This research presents an analysis of the availability and use of information and communication technologies. Diverse studies indicate that there is a verifiable inequality in terms of access to technologies, this difference is remarkable between municipalities and between regions. The analysis has allowed to know the main activities that the population makes through the ICT with the objective of determining the actions in matters of digital policies that must be considered by the local government. From the analysis performed it can be concluded that the majority of the population that Internet accesses does it through a desktop, laptop computer or a mobile phone. The most actions performed by the population through the internet are actions of entertainment and downloading software, as well as the search for general information, but participating in government affairs is not a priority issue. The population interested in public affairs is the one with the highest educational level, then that the institution requires the development of mechanisms to strengthen citizen participation in the taking of public affairs. It is also required to increase the communications infrastructure then that a greater percentage of the population can access the Internet. The programs on the use of technologies that must be developed by the government must be adapted. The study shows that the completion of procedures is not a priority issue, then that the implementation of electronic services may not have much impact on the Entity. It is necessary at first then to promote the development of the population's capacities to use of communication technologies. As a result, in this moment not have conditions to guarantee the success of an e-government policy such as the implementation of electronic payments and the use of mobile applications. It is necessary to overcome connectivity deficiencies and develop digital literacy actions to ensure the productive use of technologies. Elia Martínez | Guillermo De la Torre-Gea"Analysis of the ICT user Profile for e-goverment through Bayesian Networks" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-2 , February 2018, URL: http://www.ijtsrd.com/papers/ijtsrd2407.pdf  http://www.ijtsrd.com/other-scientific-research-area/other/2407/analysis-of-the-ict-user-profile-for-e-goverment--through-bayesian-networks/elia-martínez

Explains how Bayesian networks can tackle the limitations of pure data-driven statistical machine learning methods when applied to observational data. This is the lecture I was due to present at the NHS Health and Care Analytics Conference, 5 July 2023. For the back story on this see: https://wherearethenumbers.substack.com/p/an-update-on-my-nhs-conference-cancellation

Maternal mortality has shown a considerable decrease in countries where the rate of development is higher. This is not the case for Mexico where a large number of annual maternal deaths are still perceived, which turns out to be a public health problem in which the World Health Organization (WHO) considers indigenous populations with a higher rate of maternal deaths, as is the case in the state of Oaxaca. The aim of this paper is to carry out an analysis of maternal deaths during the period 2014-2016 in the municipalities of this state in order to identify the main causes that cause it, factors that intervene in it and municipalities with the highest rate of this problem. The analysis has revealed that the National Institute of Social Security (IMSS) is the site where the highest number of deaths occurred, as well as identifying the main causes that lead to maternal mortality, determining that it is not enough to have Public and Private Health institutions, but they must have a better care system and equipment for care, without leaving aside the cultural aspect of the inhabitants of the municipalities of the state of Oaxaca. Araceli Pinacho-Ríos | Guillermo De la Torre-Gea"Analysis of Maternal Deaths in Oaxaca through Bayesian Networks" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-2 , February 2018, URL: http://www.ijtsrd.com/papers/ijtsrd10758.pdf  http://www.ijtsrd.com/computer-science/data-miining/10758/analysis-of-maternal-deaths-in-oaxaca-through-bayesian-networks/araceli-pinacho-ríos

Migration has long been a topic of interest in Georgia, given its small economy, population, and unique history and culture. The main objective of this study is to examine the factors affecting emigration and immigration in Georgia and identify the dependencies among various macroeconomic variables, such as international remittances, trade, foreign direct investment (FDI), inflation, real interest rates, and employment. Using data spanning from 2002 to 2023, the study applies a machine learning technique, specifically Bayesian Networks, to analyze these relationships. The findings are discussed, and conclusions are drawn, along with recommendations for both the government and researchers for further exploration. To our knowledge, this is the first study to apply the Bayesian Network algorithm to investigate these dynamics in Georgia, filling an important research gap. The results indicate that both immigration and emigration are affected by remittances paid, with emigration also being dependent on employment. It was found that remittances received and exports are directly influenced by remittances paid, while imports are affected by both exports and employment. Additionally, remittances received are directly dependent on imports, and the real interest rate is influenced by both imports and inflation (CPI). FDI is shown to be dependent on inflation, imports, and remittances received. Furthermore, both emigration and immigration are dependent on exports, imports, and remittances received, with immigration also exhibiting a dependency on FDI.

Consistency-based diagnosis relies heavily on the assumption that discrepancies between model predictions and sensor observations can be detected accurately. When sources of uncertainty like sensor noise and model abstraction exist robust schemes have to be designed to make a binary decision on whether predictions are consistent with observations. This risks the occurrence of false alarms and missed alarms when an erroneous decision is made. Moreover when multiple sensors (with differing sensing properties) are available the degree of match between predictions and observations can be used to guide the search for fault candidates. In this paper we propose a novel approach to handle this problem using Bayesian networks. In the consistency- based diagnosis formulation, automatically generated Bayesian networks are used to encode a probabilistic measure of fit between predictions and observations. A Bayesian network inference algorithm is used to compute most probable fault candidates.

Portfolio methods support the combination of different algorithms and heuristics, including stochastic local search (SLS) heuristics, and have been identified as a promising approach to solve computationally hard problems. While successful in experiments, theoretical foundations and analytical results for portfolio-based SLS heuristics are less developed. This article aims to improve the understanding of the role of portfolios of heuristics in SLS. We emphasize the problem of computing most probable explanations (MPEs) in Bayesian networks (BNs). Algorithmically, we discuss a portfolio-based SLS algorithm for MPE computation, Stochastic Greedy Search (SGS). SGS supports the integration of different initialization operators (or initialization heuristics) and different search operators (greedy and noisy heuristics), thereby enabling new analytical and experimental results. Analytically, we introduce a novel Markov chain model tailored to portfolio-based SLS algorithms including SGS, thereby enabling us to analytically form expected hitting time results that explain empirical run time results. For a specific BN, we show the benefit of using a homogenous initialization portfolio. To further illustrate the portfolio approach, we consider novel additive search heuristics for handling determinism in the form of zero entries in conditional probability tables in BNs. Our additive approach adds rather than multiplies probabilities when computing the utility of an explanation. We motivate the additive measure by studying the dramatic impact of zero entries in conditional probability tables on the number of zero-probability explanations, which again complicates the search process. We consider the relationship between MAXSAT and MPE, and show that additive utility (or gain) is a generalization, to the probabilistic setting, of MAXSAT utility (or gain) used in the celebrated GSAT and WalkSAT algorithms and their descendants. Utilizing our Markov chain framework, we show that expected hitting time is a rational function - i.e. a ratio of two polynomials - of the probability of applying an additive search operator. Experimentally, we report on synthetically generated BNs as well as BNs from applications, and compare SGSs performance to that of Hugin, which performs BN inference by compilation to and propagation in clique trees. On synthetic networks, SGS speeds up computation by approximately two orders of magnitude compared to Hugin. In application networks, our approach is highly competitive in Bayesian networks with a high degree of determinism. In addition to showing that stochastic local search can be competitive with clique tree clustering, our empirical results provide an improved understanding of the circumstances under which portfolio-based SLS outperforms clique tree clustering and vice versa.

The IBIS system provides a means of correlating the images of two breech face or firing pin impressions. Cartridges fired by the same gun result in similar images and thus higher scores. The generated scores, together with related firearm and ammunition information were transformed into a Bayesian network. Bayesian networks allow for the assessment of evidence based upon two propositions (same gun ordifferent gun). This allows a forensic scientist to provide insight to courts and investigators as to the value of the evidence.The breech face (BF) and firing pin (FP) scores, and their product, were used to assess the ability of the system to classify an unknowncartridge case into a same-gun or different-gun category. The IBIS system does not provide for an easy means to use the combination of the BF and FP scores. Twenty sets of known and questioned cartridge cases, from a large collection which had been analyzed by operational firearms examiners, were examined and tested using the Bayesian networks. Out of the 20 comparisons, there were eight true positives, seven true negatives, five false negatives, and zero false positives. In all instances of eliminations, the support for the different-gun hypothesis was, at minimum, strong.

In many domains, we are interested in analyzing the structure of the underlying distribution e.g. whether one variable is a direct parent of the other. Bayesian model-selection attempts to find the MAP model and is its structure to answer these questions. However, when the amount of available data is modest, there might be many models that have non-negligible posterior. Thus, we want to compute the Bayeasian posterior of a feature, i.e. the total posterior probability of all models that contain it. In this paper, we propose a new approach for this task. We first show how to efficiently compute a sum over the exponential number of networks that are consistent with a fixed ordering over network variables. This allows us to compute, for a given ordering, both the marginal probability of the data and the posterior of a feature. We then use this result as a basis for an algorithm that approximates the Bayesian posterior of a feature. our approach uses an Markov Chain Monte Carlo (MCMC) method, but over orderings rather than over network structures. The space of orderings is much smaller and more regular than the space of structures, and has a smoother posterior landscape. We present empirical results on synthetic and real-life datasets that compare our approach to full model averaging ( when possible), to MCMC over network structures, and to a non-Bayesian bootstrap approach,

We implement a 2-time slice dynamic Bayesian network (2T-DBN) framework and make a 1-D state estimation simulation, an extension of the experiment in (v.d. Merwe et al., 2000) and compare different filtering techniques. Furthermore, we demonstrate experimentally that inference in a complex hybrid DBN is possible by simulating fault detection in a watertank system, an extension of the experiment in (Koller & Lerner, 2000) using a hybrid 2T-DBN. In both experiments, we perform approximate inference using standard filtering techniques, Monte Carlo methods and combinations of these. In the watertank simulation, we also demonstrate the use of 'non-strict' Rao-Blackwellisation. We show that the unscented Kalman filter (UKF) and UKF in a particle filtering framework outperform the generic particle filter, the extended Kalman filter (EKF) and EKF in a particle filtering framework with respect to accuracy in terms of estimation RMSE and sensitivity with respect to choice of network structure. Especially we demonstrate the superiority of UKF in a PF framework when our beliefs of how data was generated are wrong. Furthermore, we investigate the influence of data noise in the watertank simulation using UKF and PFUKD and show that the algorithms are more sensitive to changes in the measurement noise level that the process noise level. Theory and implementation is based on (v.d. Merwe et al., 2000).

This report demonstrates the application of Bayesian networks for modelling and reasoning about uncertainties. A scenario for naval anti-surface warfare is constructed and Bayesian networks are used to represent and update uncertainties encountered in the process of situation assessment. Concepts from information theory are used to provide a measure of uncertainty and understand its flow in a Bayesian network. This in turn yields analytical methods to formulate various effectiveness measures.

In this paper, we discuss how Bayesian networks can be used to develop automated situation-assessment tools suitable for use as decision aids in a command and control system. Inevitably, the introduction of a new technology raises a number of validation, systems integration and human-factors questions. Those issues pertinent to Bayesian network decision aids are identified and their implications discussed. We then describe in detail the implementation of such a system capable of providing Combat-ID and Threat Assessment advisories in the naval anti-air warfare role and its assessment within a realistic (synthetic) human-in-the-loop experiment. We discuss the experimental system, the experimental design and protocol and the experimental results. In a controlled experiment using 14 subjects with relevant military experience we found that the Bayes' net decision aid system was preferred by the majority of the experimental subjects and led to a number of operator performance improvements which could directly contribute to improved operational effectiveness.

In this paper, we discuss how Bayesian networks can be used to develop automated situation-assessment tools suitable for use as decision aids in a command and control system. Inevitably, the introduction of a new technology raises a number of validation, systems integration and human-factors questions. Those issues pertinent to Bayesian network decision aids are identified and their implications discussed. We then describe in detail the implementation of such a system capable of providing Combat-ID and Threat Assessment advisories in the naval anti-air warfare role and its assessment within a realistic (synthetic) human-in-the-loop experiment. We discuss the experimental system, the experimental design and protocol and the experimental results. In a controlled experiment using 14 subjects with relevant military experience we found that the Bayes' net decision aid system was preferred by the majority of the experimental subjects and led to a number of operator performance improvements which could directly contribute to improved operational effectiveness.

Real data often contains a mixture of discrete and continuous variables, but many Bayesian network structure learning and inference algorithms assume all random variables are discrete. Continuous variables are often discretized, but the choice of discretization policy has significant impact on the accuracy, speed, and interpretability of the resulting models. This paper introduces a principled Bayesian discretization method for continuous variables in Bayesian networks with quadratic complexity instead of the cubic complexity of other standard techniques. Empirical demonstrations show that the proposed method is superior to the state of the art. In addition, this paper shows how to incorporate existing methods into the structure learning process to discretize all continuous variables and simultaneously learn Bayesian network structures.

A plethora of networks is being collected in a growing number of fields, including disease transmission, international relations, social interactions, and others. As data streams continue to grow, the complexity associated with these highly multidimensional connectivity data presents novel challenges. In this paper, we focus on the time-varying interconnections among a set of actors in multiple contexts, called layers. Current literature lacks flexible statistical models for dynamic multilayer networks, which can enhance quality in inference and prediction by efficiently borrowing information within each network, across time, and between layers. Motivated by this gap, we develop a Bayesian nonparametric model leveraging latent space representations. Our formulation characterizes the edge probabilities as a function of shared and layer-specific actors positions in a latent space, with these positions changing in time via Gaussian processes. This representation facilitates dimensionality reduction and incorporates different sources of information in the observed data. In addition, we obtain tractable procedures for posterior computation, inference, and prediction. We provide theoretical results on the flexibility of our model. Our methods are tested on simulations and infection studies monitoring dynamic face-to-face contacts among individuals in multiple days, where we perform better than current methods in inference and prediction.

For hard computational problems, stochastic local search has proven to be a competitive approach to finding optimal or approximately optimal problem solutions. Two key research questions for stochastic local search algorithms are: Which algorithms are effective for initialization? When should the search process be restarted? In the present work we investigate these research questions in the context of approximate computation of most probable explanations (MPEs) in Bayesian networks (BNs). We introduce a novel approach, based on the Viterbi algorithm, to explanation initialization in BNs. While the Viterbi algorithm works on sequences and trees, our approach works on BNs with arbitrary topologies. We also give a novel formalization of stochastic local search, with focus on initialization and restart, using probability theory and mixture models. Experimentally, we apply our methods to the problem of MPE computation, using a stochastic local search algorithm known as Stochastic Greedy Search. By carefully optimizing both initialization and restart, we reduce the MPE search time for application BNs by several orders of magnitude compared to using uniform at random initialization without restart. On several BNs from applications, the performance of Stochastic Greedy Search is competitive with clique tree clustering, a state-of-the-art exact algorithm used for MPE computation in BNs.

In this article, we review a general theoretical framework of thermodynamics of information on the basis of Bayesian networks. This framework can describe a broad class of nonequilibrium dynamics of multiple interacting systems with complex information exchanges. For such situations, we discuss a generalization of the second law of thermodynamics including information contents. The key concept here is an informational quantity called the transfer entropy, which describes the directional information transfer in stochastic dynamics. The generalized second law gives the fundamental lower bound of the entropy production in nonequilibrium dynamics, and sheds modern light on the paradox of "Maxwell's demon" that performs measurements and feedback control at the level of thermal fluctuations.

In Bayesian networks, exact belief propagation is achieved through message passing algorithms. These algorithms (ex: inward and outward) provide only a recursive definition of the corresponding messages. In contrast, when working on hidden Markov models and variants, one classically first defines explicitly these messages (forward and backward quantities), and then derive all results and algorithms. In this paper, we generalize the hidden Markov model approach by introducing an explicit definition of the messages in Bayesian networks, from which we derive all the relevant properties and results including the recursive algorithms that allow to compute these messages. Two didactic examples (the precipitation hidden Markov model and the pedigree Bayesian network) are considered along the paper to illustrate the new formalism and standalone R source code is provided in the appendix.

Gene regulatory networks are collections of genes that interact with one other and with other substances in the cell. By measuring gene expression over time using high-throughput technologies, it may be possible to reverse engineer, or infer, the structure of the gene network involved in a particular cellular process. These gene expression data typically have a high dimensionality and a limited number of biological replicates and time points. Due to these issues and the complexity of biological systems, the problem of reverse engineering networks from gene expression data demands a specialized suite of statistical tools and methodologies. We propose a non-standard adaptation of a simulation-based approach known as Approximate Bayesian Computing based on Markov chain Monte Carlo sampling. This approach is particularly well suited for the inference of gene regulatory networks from longitudinal data. The performance of this approach is investigated via simulations and using longitudinal expression data from a genetic repair system in Escherichia coli.

When a posterior distribution has multiple modes, unconditional expectations, such as the posterior mean, may not offer informative summaries of the distribution. Motivated by this problem, we propose to decompose the sample space of a multimodal distribution into domains of attraction of local modes. Domain-based representations are defined to summarize the probability masses of and conditional expectations on domains of attraction, which are much more informative than the mean and other unconditional expectations. A computational method, the multi-domain sampler, is developed to construct domain-based representations for an arbitrary multimodal distribution. The multi-domain sampler is applied to structural learning of protein-signaling networks from high-throughput single-cell data, where a signaling network is modeled as a causal Bayesian network. Not only does our method provide a detailed landscape of the posterior distribution but also improves the accuracy and the predictive power of estimated networks.

This article is from PLoS ONE , volume 9 . Abstract The bias-variance dilemma is a well-known and important problem in Machine Learning. It basically relates the generalization capability (goodness of fit) of a learning method to its corresponding complexity. When we have enough data at hand, it is possible to use these data in such a way so as to minimize overfitting (the risk of selecting a complex model that generalizes poorly). Unfortunately, there are many situations where we simply do not have this required amount of data. Thus, we need to find methods capable of efficiently exploiting the available data while avoiding overfitting. Different metrics have been proposed to achieve this goal: the Minimum Description Length principle (MDL), Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC), among others. In this paper, we focus on crude MDL and empirically evaluate its performance in selecting models with a good balance between goodness of fit and complexity: the so-called bias-variance dilemma, decomposition or tradeoff. Although the graphical interaction between these dimensions (bias and variance) is ubiquitous in the Machine Learning literature, few works present experimental evidence to recover such interaction. In our experiments, we argue that the resulting graphs allow us to gain insights that are difficult to unveil otherwise: that crude MDL naturally selects balanced models in terms of bias-variance, which not necessarily need be the gold-standard ones. We carry out these experiments using a specific model: a Bayesian network. In spite of these motivating results, we also should not overlook three other components that may significantly affect the final model selection: the search procedure, the noise rate and the sample size.

This article is from PLoS ONE , volume 9 . Abstract The bias-variance dilemma is a well-known and important problem in Machine Learning. It basically relates the generalization capability (goodness of fit) of a learning method to its corresponding complexity. When we have enough data at hand, it is possible to use these data in such a way so as to minimize overfitting (the risk of selecting a complex model that generalizes poorly). Unfortunately, there are many situations where we simply do not have this required amount of data. Thus, we need to find methods capable of efficiently exploiting the available data while avoiding overfitting. Different metrics have been proposed to achieve this goal: the Minimum Description Length principle (MDL), Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC), among others. In this paper, we focus on crude MDL and empirically evaluate its performance in selecting models with a good balance between goodness of fit and complexity: the so-called bias-variance dilemma, decomposition or tradeoff. Although the graphical interaction between these dimensions (bias and variance) is ubiquitous in the Machine Learning literature, few works present experimental evidence to recover such interaction. In our experiments, we argue that the resulting graphs allow us to gain insights that are difficult to unveil otherwise: that crude MDL naturally selects balanced models in terms of bias-variance, which not necessarily need be the gold-standard ones. We carry out these experiments using a specific model: a Bayesian network. In spite of these motivating results, we also should not overlook three other components that may significantly affect the final model selection: the search procedure, the noise rate and the sample size.

When analysing gene expression time series data an often overlooked but crucial aspect of the model is that the regulatory network structure may change over time. Whilst some approaches have addressed this problem previously in the literature, many are not well suited to the sequential nature of the data. Here we present a method that allows us to infer regulatory network structures that may vary between time points, utilising a set of hidden states that describe the network structure at a given time point. To model the distribution of the hidden states we have applied the Hierarchical Dirichlet Process Hideen Markov Model, a nonparametric extension of the traditional Hidden Markov Model, that does not require us to fix the number of hidden states in advance. We apply our method to exisiting microarray expression data as well as demonstrating is efficacy on simulated test data.

Judea Pearl is a professor at UCLA and a winner of the Turing Award, that's generally recognized as the Nobel Prize of computing. He is one of the seminal figures in the field of artificial intelligence, computer science, and statistics. He has developed and championed probabilistic approaches to AI, including Bayesian Networks and profound ideas in causality in general. These ideas are important not just for AI, but to our understanding and practice of science. But in the field of AI, the idea of causality, cause and effect, to many, lies at the core of what is currently missing and

Local computation for updating Bayesian belief networks proceeds in the context of a join tree, consisting of subsets of interrelated variables (cliques) joined by their intersection sets in a singly-connected graphical structure. When multiple independent and identically-distributed (IID) observations of a variable can be made, identically structured cliques corresponding to each potential observation appear as terminal nodes in the join tree. This note shows how it is possible to absorb information from an indefinite number of observations of this type without preconstructing and manipulating cliques for all potential observations. An update & replace strategy carries the necessary information with only two nodes for a family of IID observations of a variable at any point in time. Bayesian inference networks, Causal probability networks, Expert systems, Influence diagrams, Intelligent tutoring systems, Local computation

We evaluate the uncertainty quality in neural networks using anomaly detection. We extract uncertainty measures (e.g. entropy) from the predictions of candidate models, use those measures as features for an anomaly detector, and gauge how well the detector differentiates known from unknown classes. We assign higher uncertainty quality to candidate models that lead to better detectors. We also propose a novel method for sampling a variational approximation of a Bayesian neural network, called One-Sample Bayesian Approximation (OSBA). We experiment on two datasets, MNIST and CIFAR10. We compare the following candidate neural network models: Maximum Likelihood, Bayesian Dropout, OSBA, and --- for MNIST --- the standard variational approximation. We show that Bayesian Dropout and OSBA provide better uncertainty information than Maximum Likelihood, and are essentially equivalent to the standard variational approximation, but much faster.

We present a method for learning treewidth-bounded Bayesian networks from data sets containing thousands of variables. Bounding the treewidth of a Bayesian greatly reduces the complexity of inferences. Yet, being a global property of the graph, it considerably increases the difficulty of the learning process. We propose a novel algorithm for this task, able to scale to large domains and large treewidths. Our novel approach consistently outperforms the state of the art on data sets with up to ten thousand variables.

Bayesian network structures are usually built using only the data and starting from an empty network or from a naive Bayes structure. Very often, in some domains, like medicine, a prior structure knowledge is already known. This structure can be automatically or manually refined in search for better performance models. In this work, we take Bayesian networks built by specialists and show that minor perturbations to this original network can yield better classifiers with a very small computational cost, while maintaining most of the intended meaning of the original model.

We study the Bayesian model averaging approach to learning Bayesian network structures (DAGs) from data. We develop new algorithms including the first algorithm that is able to efficiently sample DAGs according to the exact structure posterior. The DAG samples can then be used to construct estimators for the posterior of any feature. We theoretically prove good properties of our estimators and empirically show that our estimators considerably outperform the estimators from the previous state-of-the-art methods.