Downloads & Free Reading Options - Results

This article is from Journal of Pathology Informatics , volume 2 . Abstract Increased type I error resulting from multiple statistical comparisons remains a common problem in the scientific literature. This may result in the reporting and promulgation of spurious findings. One approach to this problem is to correct groups of P-values for “family-wide significance” using a Bonferroni correction or the less conservative Bonferroni-Holm correction or to correct for the “false discovery rate” with a Benjamini-Hochberg correction. Although several solutions are available for performing this correction through commercially available software there are no widely available easy to use open source programs to perform these calculations. In this paper we present an open source program written in Python 3.2 that performs calculations for standard Bonferroni, Bonferroni-Holm and Benjamini-Hochberg corrections.

The aim of this scoping review is to identify and describe how multiple treatment comparisons are combined and evaluated in infectious diseases research.

Voxel-based lesion-symptom mapping (VLSM) is an important method for basic and translational human neuroscience research. VLSM leverages modern neuroimaging analysis techniques to build on the classic approach of examining the relationship between location of brain damage and cognitive deficits. Testing an association between deficit severity and lesion status in each voxel involves very many individual tests and requires statistical correction for multiple comparisons. Several strategies have been adapted from analysis of functional neuroimaging data, though VLSM faces a more difficult trade-off between avoiding false positives and statistical power (missing true effects). Non-parametric, permutation-based methods are generally preferable because they do not make assumptions that are likely to be violated by skewed distributions of behavioral deficit (symptom) scores and by the necessary spatial contiguity of stroke lesions. We used simulated and real deficit scores from a sample of approximately 100 individuals with left hemisphere stroke to evaluate two such permutation-based approaches. Using permutation to set a minimum cluster size identified a region that systematically extended well beyond the true region, even under the most conservative settings tested here, making it ill-suited to identifying brain-behavior relationships. In contrast, generalizing the standard permutation-based family-wise error correction approach provided a principled way to balance false positives and false negatives. An implementation of this continuous permutation-based FWER correction method is available at https://gist.github.com/dmirman/05a92e0e9e0027f6fe6e528c648143d7

Voxel-based lesion-symptom mapping (VLSM) is an important method for basic and translational human neuroscience research. VLSM leverages modern neuroimaging analysis techniques to build on the classic approach of examining the relationship between location of brain damage and cognitive deficits. Testing an association between deficit severity and lesion status in each voxel involves very many individual tests and requires statistical correction for multiple comparisons. Several strategies have been adapted from analysis of functional neuroimaging data, though VLSM faces a more difficult trade-off between avoiding false positives and statistical power (missing true effects). Non-parametric, permutation-based methods are generally preferable because they do not make assumptions that are likely to be violated by skewed distributions of behavioral deficit (symptom) scores and by the necessary spatial contiguity of stroke lesions. We used simulated and real deficit scores from a sample of approximately 100 individuals with left hemisphere stroke to evaluate two such permutation-based approaches. Using permutation to set a minimum cluster size identified a region that systematically extended well beyond the true region, even under the most conservative settings tested here, making it ill-suited to identifying brain-behavior relationships. In contrast, generalizing the standard permutation-based family-wise error correction approach provided a principled way to balance false positives and false negatives. An implementation of this continuous permutation-based FWER correction method is available at https://gist.github.com/dmirman/05a92e0e9e0027f6fe6e528c648143d7

A sensitivity analysis in an observational study assesses the robustness of significant findings to unmeasured confounding. While sensitivity analyses in matched observational studies have been well addressed when there is a single outcome variable, accounting for multiple comparisons through the existing methods yields overly conservative results when there are multiple outcome variables of interest. This stems from the fact that unmeasured confounding cannot affect the probability of assignment to treatment differently depending on the outcome being analyzed. Existing methods allow this to occur by combining the results of individual sensitivity analyses to assess whether at least one hypothesis is significant, which in turn results in an overly pessimistic assessment of a study's sensitivity to unobserved biases. By solving a quadratically constrained linear program, we are able to perform a sensitivity analysis while enforcing that unmeasured confounding must have the same impact on the treatment assignment probabilities across outcomes for each individual in the study. We show that this allows for uniform improvements in the power of a sensitivity analysis not only for testing the overall null of no effect, but also for null hypotheses on \textit{specific} outcome variables while strongly controlling the familywise error rate. We illustrate our method through an observational study on the effect of smoking on naphthalene exposure.

A sensitivity analysis in an observational study assesses the robustness of significant findings to unmeasured confounding. While sensitivity analyses in matched observational studies have been well addressed when there is a single outcome variable, accounting for multiple comparisons through the existing methods yields overly conservative results when there are multiple outcome variables of interest. This stems from the fact that unmeasured confounding cannot affect the probability of assignment to treatment differently depending on the outcome being analyzed. Existing methods allow this to occur by combining the results of individual sensitivity analyses to assess whether at least one hypothesis is significant, which in turn results in an overly pessimistic assessment of a study's sensitivity to unobserved biases. By solving a quadratically constrained linear program, we are able to perform a sensitivity analysis while enforcing that unmeasured confounding must have the same impact on the treatment assignment probabilities across outcomes for each individual in the study. We show that this allows for uniform improvements in the power of a sensitivity analysis not only for testing the overall null of no effect, but also for null hypotheses on \textit{specific} outcome variables while strongly controlling the familywise error rate. We illustrate our method through an observational study on the effect of smoking on naphthalene exposure.

Using computer simulated data, the Type I error rate and statistical power were empirically estimated for several pairwise multiple comparison strategies for situations where population variances differ. Focus was on comparing modified Bonferroni procedures with Dunnett's solutions, and determining whether or not J. P. Shaffer's suggestion of using the omnibus test would work when population variances differed. Three factors were manipulated: sample size, variance heterogeneity, and pattern of population mean differences. Twenty-four different combinations of sample sizes and variance patterns were examined for the single factor four group design. The results indicate that all eight contrast procedures considered controlled the familywise Type I error rate under the nominal 0.05 level. In terms of statistical power, the Games-Howell procedure generally provided the greater power in identifying at least one significant difference. However, the magnitude of the any-pair power difference was very small. J. P. Shaffer's (1979) enhancements to the Bonferroni approach provided greater average power per contrast as well as the greatest power in identifying all significant pairwise differences. The results of the present study indicate that previous recommendations concerning the selection of a multiple comparison procedure when population variances differ should be reconsidered, and the adoption of the new strategies for multiple comparisons is recommended. Twelve data tables and a 22-item list of references are included. (Author/RLC)

Contains data and code underlying results presented in Gutknecht, Aaron J.; Wibral, M. (2021): "Significant Subgraph Mining for Neural Network Inference with Multiple Comparisons Correction". CC0 Waiver

This article is from BMC Bioinformatics , volume 14 . Abstract Background: As high-throughput genomic technologies become accurate and affordable, an increasing number of data sets have been accumulated in the public domain and genomic information integration and meta-analysis have become routine in biomedical research. In this paper, we focus on microarray meta-analysis, where multiple microarray studies with relevant biological hypotheses are combined in order to improve candidate marker detection. Many methods have been developed and applied in the literature, but their performance and properties have only been minimally investigated. There is currently no clear conclusion or guideline as to the proper choice of a meta-analysis method given an application; the decision essentially requires both statistical and biological considerations. Results: We performed 12 microarray meta-analysis methods for combining multiple simulated expression profiles, and such methods can be categorized for different hypothesis setting purposes: (1) HSA: DE genes with non-zero effect sizes in all studies, (2) HSB: DE genes with non-zero effect sizes in one or more studies and (3) HSr: DE gene with non-zero effect in "majority" of studies. We then performed a comprehensive comparative analysis through six large-scale real applications using four quantitative statistical evaluation criteria: detection capability, biological association, stability and robustness. We elucidated hypothesis settings behind the methods and further apply multi-dimensional scaling (MDS) and an entropy measure to characterize the meta-analysis methods and data structure, respectively. Conclusions: The aggregated results from the simulation study categorized the 12 methods into three hypothesis settings (HSA, HSB, and HSr). Evaluation in real data and results from MDS and entropy analyses provided an insightful and practical guideline to the choice of the most suitable method in a given application. All source files for simulation and real data are available on the author’s publication website.

RS-HCB is the simultaneous computer implementation of Ranking and Selection (RS) and Multiple Comparisons with the Best (MCB) procedures. This is made possible by recent developments in statistics which showed that Ranking and Selection (both Subset Selection and Indifference Zone) can be executed simultaneously with Multiple Comparisons with the Best without increasing the error rate of any component inference, for equal as well as unequal sample sizes. These developments are described, and the use of RS-MCB is illustrated with sample computer sessions.

The representation of upper tropospheric/lower stratospheric (UTLS) jet and tropopause characteristics is compared in five modern high-resolution reanalyses for 1980 through 2014. Climatologies of upper tropospheric jet, subvortex jet (the lowermost part of the stratospheric vortex), and multiple tropopause frequency distributions in MERRA (Modern Era Retrospective Analysis for Research and Applications), ERA-I (the ECMWF interim reanalysis), JRA-55 (the Japanese 55-year Reanalysis), and CFSR (the Climate Forecast System Reanalysis) are compared with those in MERRA-2. Differences between alternate products from individual reanalysis systems are assessed; in particular, a comparison of CFSR data on model and pressure levels highlights the importance of vertical grid spacing. Most of the differences in distributions of UTLS jets and multiple tropopauses are consistent with the differences in assimilation model grids and resolution: For example, ERA-I (with coarsest native horizontal resolution) typically shows a significant low bias in upper tropospheric jets with respect to MERRA-2, and JRA-55 a more modest one, while CFSR (with finest native horizontal resolution) shows a high bias with respect to MERRA-2 in both upper tropospheric jets and multiple tropopauses. Vertical temperature structure and grid spacing are especially important for multiple tropopause characterization. Substantial differences between MERRA and MERRA-2 are seen in mid- to high-latitude southern hemisphere winter upper tropospheric jets and multiple tropopauses, and in the upper tropospheric jets associated with tropical circulations during the solstice seasons; some of the largest differences from the other reanalyses are seen in the same times and places. Very good qualitative agreement among the reanalyses is seen between the large scale climatological features in UTLS jet and multiple tropopause distributions. Quantitative differences may, however, have important consequences for transport and variability studies. Our results highlight the importance of considering reanalyses differences in UTLS studies, especially in relation to resolution and model grids; this is particularly critical when using high-resolution reanalyses as an observational reference for evaluating global chemistry climate models.

We propose a novel Bayesian model selection technique on linear mixed-effects models to compare multiple treatments with a control. A fully Bayesian approach is implemented to estimate the marginal inclusion probabilities that provide a direct measure of the difference between treatments and the control, along with the model-averaged posterior distributions. Default priors are proposed for model selection incorporating domain knowledge and a component-wise Gibbs sampler is developed for efficient posterior computation. We demonstrate the proposed method based on simulated data and an experimental dataset from a longitudinal study of mouse lifespan and weight trajectories.

The 4STAR (Spectrometer for Sky-Scanning, Sun-Tracking Atmospheric Research) instrument combines airborne sun tracking capabilities of the Ames Airborne Tracking Sun Photometer (AATS-14) with AERONET-like sky-scanning capability and adds state-of-the-art fiber-coupled grating spectrometry to yield hyper spectral measurements of direct solar irradiance and angularly resolved sky radiance. The combination of sun-tracking and sky-scanning capability enables retrievals of wavelength-dependent aerosol optical depth (AOD), mode-resolved aerosol size distribution (SD), asphericity, and complex refractive index, and thus also the scattering phase function, asymmetry parameter, single-scattering albedo (SSA), and absorption aerosol optical thickness (AAOT).From 2012 to 2014 4STAR participated in four major field campaigns: the U.S. Dept. of Energy TCAP I II campaigns, and NASAs SEAC4RS and ARISE campaigns. Establishing a strong performance record, 4STAR operated successfully on all flights conducted during each of these campaigns. Sky radiance spectra from scans in either constant azimuth (principal plane) or constant zenith angle (almucantar) were interspersed with direct beam measurements during level legs. During SEAC4RS and ARISE, 4STAR airborne measurements were augmented with flight-level albedo from the collocated Shortwave Spectral Flux Radiometer (SSFR) providing improved specification of below-aircraft radiative conditions for the retrieval. Calibrated radiances and retrieved products will be presented with particular emphasis on detailed comparisons of ambient SSA retrievals and measurements during SEAC4RS from 4STAR, AERONET, HSRL2, and from in situ measurements.

The 4STAR (Spectrometer for Sky-Scanning, Sun-Tracking Atmospheric Research) instrument combines airborne sun tracking capabilities of the Ames Airborne Tracking Sun Photometer (AATS-14) with AERONET (Aerosol Robotic Network)-like sky-scanning capability and adds state-of-the-art fiber-coupled grating spectrometry to yield hyperspectral measurements of direct solar irradiance and angularly resolved sky radiance. The combination of sun-tracking and sky-scanning capability enables retrievals of wavelength-dependent aerosol optical depth (AOD), mode-resolved aerosol size distribution (SD), asphericity, and complex refractive index, and thus also the scattering phase function, asymmetry parameter, single-scattering albedo (SSA), and absorption aerosol optical thickness (AAOT). From 2012 to 2014 4STAR participated in four major field campaigns: the U.S. Dept. of Energy's TCAP (Two-Column Aerosol Project) I & II campaigns, and NASA's SEAC4RS (Studies of Emissions, Atmospheric Composition, Clouds and Climate Coupling by Regional Surveys) and ARISE (Arctic Radiation - IceBridge Sea & Ice Experiment) campaigns. Establishing a strong performance record, 4STAR operated successfully on all flights conducted during each of these campaigns. Sky radiance spectra from scans in either constant azimuth (principal plane) or constant zenith angle (almucantar) were interspersed with direct beam measurements during level legs. During SEAC4RS and ARISE, 4STAR airborne measurements were augmented with flight-level albedo from the collocated Shortwave Spectral Flux Radiometer (SSFR) providing improved specification of below-aircraft radiative conditions for the retrieval. Calibrated radiances and retrieved products will be presented with particular emphasis on detailed comparisons of ambient SSA retrievals and measurements during SEAC4RS from 4STAR, AERONET, HSRL2 (High Spectral Resolution Lidar), and from in situ measurements.

The 4STAR (Spectrometer for Sky-Scanning, Sun-Tracking Atmospheric Research) instrument combines airborne sun tracking capabilities of the Ames Airborne Tracking Sun Photometer (AATS-14) with AERONET (Aerosol Robotic Network)-like sky-scanning capability and adds state-of-the-art fiber-coupled grating spectrometry to yield hyperspectral measurements of direct solar irradiance and angularly resolved sky radiance. The combination of sun-tracking and sky-scanning capability enables retrievals of wavelength-dependent aerosol optical depth (AOD), mode-resolved aerosol size distribution (SD), asphericity, and complex refractive index, and thus also the scattering phase function, asymmetry parameter, single-scattering albedo (SSA), and absorption aerosol optical thickness (AAOT). From 2012 to 2014 4STAR participated in four major field campaigns: the U.S. Dept. of Energy's TCAP (Two-Column Aerosol Project) I & II campaigns, and NASA's SEAC4RS (Studies of Emissions, Atmospheric Composition, Clouds and Climate Coupling by Regional Surveys) and ARISE (Arctic Radiation - IceBridge Sea & Ice Experiment) campaigns. Establishing a strong performance record, 4STAR operated successfully on all flights conducted during each of these campaigns. Sky radiance spectra from scans in either constant azimuth (principal plane) or constant zenith angle (almucantar) were interspersed with direct beam measurements during level legs. During SEAC4RS and ARISE, 4STAR airborne measurements were augmented with flight-level albedo from the collocated Shortwave Spectral Flux Radiometer (SSFR) providing improved specification of below-aircraft radiative conditions for the retrieval. Calibrated radiances and retrieved products will be presented with particular emphasis on detailed comparisons of ambient SSA retrievals and measurements during SEAC4RS from 4STAR, AERONET, HSRL2 (High Spectral Resolution Lidar), and from in situ measurements.

Using Excel for Dynamic Analysis of Variance and Unplanned Multiple Comparisons Procedures

We study the problem of multiple hypothesis testing for multidimensional data when inter-correlations are present. The problem of multiple comparisons is common in many applications. When the data is multivariate and correlated, existing multiple comparisons procedures based on maximum likelihood estimation could be prohibitively computationally intensive. We propose to construct multiple comparisons procedures based on composite likelihood statistics. We focus on data arising in three ubiquitous cases: multivariate Gaussian, probit, and quadratic exponential models. To help practitioners assess the quality of our proposed methods, we assess their empirical performance via Monte Carlo simulations. It is shown that composite likelihood based procedures maintain good control of the familywise type I error rate in the presence of intra-cluster correlation, whereas ignoring the correlation leads to erratic performance. Using data arising from a diabetic nephropathy study, we show how our composite likelihood approach makes an otherwise intractable analysis possible.

This article is from Environmental Health , volume 12 . Abstract Background: As public awareness of consequences of environmental exposures has grown, estimating the adverse health effects due to simultaneous exposure to multiple pollutants is an important topic to explore. The challenges of evaluating the health impacts of environmental factors in a multipollutant model include, but are not limited to: identification of the most critical components of the pollutant mixture, examination of potential interaction effects, and attribution of health effects to individual pollutants in the presence of multicollinearity. Methods: In this paper, we reviewed five methods available in the statistical literature that are potentially helpful for constructing multipollutant models. We conducted a simulation study and presented two data examples to assess the performance of these methods on feature selection, effect estimation and interaction identification using both cross-sectional and time-series designs. We also proposed and evaluated a two-step strategy employing an initial screening by a tree-based method followed by further dimension reduction/variable selection by the aforementioned five approaches at the second step. Results: Among the five methods, least absolute shrinkage and selection operator regression performs well in general for identifying important exposures, but will yield biased estimates and slightly larger model dimension given many correlated candidate exposures and modest sample size. Bayesian model averaging, and supervised principal component analysis are also useful in variable selection when there is a moderately strong exposure-response association. Substantial improvements on reducing model dimension and identifying important variables have been observed for all the five statistical methods using the two-step modeling strategy when the number of candidate variables is large. Conclusions: There is no uniform dominance of one method across all simulation scenarios and all criteria. The performances differ according to the nature of the response variable, the sample size, the number of pollutants involved, and the strength of exposure-response association/interaction. However, the two-step modeling strategy proposed here is potentially applicable under a multipollutant framework with many covariates by taking advantage of both the screening feature of an initial tree-based method and dimension reduction/variable selection property of the subsequent method. The choice of the method should also depend on the goal of the study: risk prediction, effect estimation or screening for important predictors and their interactions.

Similarity between objects is multi-faceted and it can be easier for human annotators to measure it when the focus is on a specific aspect. We consider the problem of mapping objects into view-specific embeddings where the distance between them is consistent with the similarity comparisons of the form "from the t-th view, object A is more similar to B than to C". Our framework jointly learns view-specific embeddings exploiting correlations between views. Experiments on a number of datasets, including one of multi-view crowdsourced comparison on bird images, show the proposed method achieves lower triplet generalization error when compared to both learning embeddings independently for each view and all views pooled into one view. Our method can also be used to learn multiple measures of similarity over input features taking class labels into account and compares favorably to existing approaches for multi-task metric learning on the ISOLET dataset.

Testing for the significance of a subset of regression coefficients in a linear model, a staple of statistical analysis, goes back at least to the work of Fisher who introduced the analysis of variance (ANOVA). We study this problem under the assumption that the coefficient vector is sparse, a common situation in modern high-dimensional settings. Suppose we have $p$ covariates and that under the alternative, the response only depends upon the order of $p^{1-\alpha}$ of those, $0\le\alpha\le1$. Under moderate sparsity levels, that is, $0\le\alpha\le1/2$, we show that ANOVA is essentially optimal under some conditions on the design. This is no longer the case under strong sparsity constraints, that is, $\alpha>1/2$. In such settings, a multiple comparison procedure is often preferred and we establish its optimality when $\alpha\geq3/4$. However, these two very popular methods are suboptimal, and sometimes powerless, under moderately strong sparsity where $1/2

Testing for the significance of a subset of regression coefficients in a linear model, a staple of statistical analysis, goes back at least to the work of Fisher who introduced the analysis of variance (ANOVA). We study this problem under the assumption that the coefficient vector is sparse, a common situation in modern high-dimensional settings. Suppose we have $p$ covariates and that under the alternative, the response only depends upon the order of $p^{1-\alpha}$ of those, $0\le\alpha\le1$. Under moderate sparsity levels, that is, $0\le\alpha\le1/2$, we show that ANOVA is essentially optimal under some conditions on the design. This is no longer the case under strong sparsity constraints, that is, $\alpha>1/2$. In such settings, a multiple comparison procedure is often preferred and we establish its optimality when $\alpha\geq3/4$. However, these two very popular methods are suboptimal, and sometimes powerless, under moderately strong sparsity where $1/2

This paper reports two experiments which examined processes involved in children's ability to solve class inclusion problems of the form, "Are there more A' (subclass) or more A (class)?" In the first experiment a total of 216 children in age groups 5, 7 and 9 years were placed in three conditions which were designed to distinguish performance on single and double class inclusion problems (involving implicit and explicit partitioning) and to examine subsequent transfer of experience with the double class display procedure when the child returned to the traditional one-class setting. Findings support the position that certain perceptual contexts are more likely to induce the child to perceive the class rather than merely the subclasses. Overall transfer results also showed improvement but gave no significant effects for explicit compared with implicit partioning of the double class display. Experiment 2 placed into two conditions a total of 120 children in age groups 5, 7 and 9 years, each condition comprising three consecutive tests designed to test the effect of intervening double class problems on Test 2 performance and on Test 3 transfer effects. Anticipated improvements were demonstrated. Comparison of data from each experiment suggests that class inclusion is not what is at issue in part-whole comparisons. A general process description of what occurs in class inclusion problems is offered. (GO)

This paper reports two experiments which examined processes involved in children's ability to solve class inclusion problems of the form, "Are there more A' (subclass) or more A (class)?" In the first experiment a total of 216 children in age groups 5, 7 and 9 years were placed in three conditions which were designed to distinguish performance on single and double class inclusion problems (involving implicit and explicit partitioning) and to examine subsequent transfer of experience with the double class display procedure when the child returned to the traditional one-class setting. Findings support the position that certain perceptual contexts are more likely to induce the child to perceive the class rather than merely the subclasses. Overall transfer results also showed improvement but gave no significant effects for explicit compared with implicit partioning of the double class display. Experiment 2 placed into two conditions a total of 120 children in age groups 5, 7 and 9 years, each condition comprising three consecutive tests designed to test the effect of intervening double class problems on Test 2 performance and on Test 3 transfer effects. Anticipated improvements were demonstrated. Comparison of data from each experiment suggests that class inclusion is not what is at issue in part-whole comparisons. A general process description of what occurs in class inclusion problems is offered. (GO)

This paper reports two experiments which examined processes involved in children's ability to solve class inclusion problems of the form, "Are there more A' (subclass) or more A (class)?" In the first experiment a total of 216 children in age groups 5, 7 and 9 years were placed in three conditions which were designed to distinguish performance on single and double class inclusion problems (involving implicit and explicit partitioning) and to examine subsequent transfer of experience with the double class display procedure when the child returned to the traditional one-class setting. Findings support the position that certain perceptual contexts are more likely to induce the child to perceive the class rather than merely the subclasses. Overall transfer results also showed improvement but gave no significant effects for explicit compared with implicit partioning of the double class display. Experiment 2 placed into two conditions a total of 120 children in age groups 5, 7 and 9 years, each condition comprising three consecutive tests designed to test the effect of intervening double class problems on Test 2 performance and on Test 3 transfer effects. Anticipated improvements were demonstrated. Comparison of data from each experiment suggests that class inclusion is not what is at issue in part-whole comparisons. A general process description of what occurs in class inclusion problems is offered. (GO)

Ranking a set of objects involves establishing an order allowing for comparisons between any pair of objects in the set. Oftentimes, due to the unavailability of a ground truth of ranked orders, researchers resort to obtaining judgments from multiple annotators followed by inferring the ground truth based on the collective knowledge of the crowd. However, the aggregation is often ad-hoc and involves imposing stringent assumptions in inferring the ground truth (e.g. majority vote). In this work, we propose Expectation-Maximization (EM) based algorithms that rely on the judgments from multiple annotators and the object attributes for inferring the latent ground truth. The algorithm learns the relation between the latent ground truth and object attributes as well as annotator specific probabilities of flipping, a metric to assess annotator quality. We further extend the EM algorithm to allow for a variable probability of flipping based on the pair of objects at hand. We test our algorithms on two data sets with synthetic annotations and investigate the impact of annotator quality and quantity on the inferred ground truth. We also obtain the results on two other data sets with annotations from machine/human annotators and interpret the output trends based on the data characteristics.

Ranking a set of objects involves establishing an order allowing for comparisons between any pair of objects in the set. Oftentimes, due to the unavailability of a ground truth of ranked orders, researchers resort to obtaining judgments from multiple annotators followed by inferring the ground truth based on the collective knowledge of the crowd. However, the aggregation is often ad-hoc and involves imposing stringent assumptions in inferring the ground truth (e.g. majority vote). In this work, we propose Expectation-Maximization (EM) based algorithms that rely on the judgments from multiple annotators and the object attributes for inferring the latent ground truth. The algorithm learns the relation between the latent ground truth and object attributes as well as annotator specific probabilities of flipping, a metric to assess annotator quality. We further extend the EM algorithm to allow for a variable probability of flipping based on the pair of objects at hand. We test our algorithms on two data sets with synthetic annotations and investigate the impact of annotator quality and quantity on the inferred ground truth. We also obtain the results on two other data sets with annotations from machine/human annotators and interpret the output trends based on the data characteristics.

This article is from Malaria Journal , volume 11 . Abstract Background: Artemisinin-based combination therapies (ACT) are widely used in African countries, including Cameroon. Between 2005 and 2007, five randomized studies comparing different treatment arms among artesunate-amodiaquine and other ACT were conducted in Cameroonian children aged two to 60 months who had uncomplicated Plasmodium falciparum malaria. In these studies, the categorical criterion proposed by the World Health Organization (WHO) to assess the relative effectiveness of anti-malarial drugs was repeatedly evaluated on Days 14, 21 and 28 after treatment initiation. The aim of the present study was to compare the effects of different treatments on this repeated ordinal outcome, hence using the fully available information. Methods: The quantitative synthesis was based on individual patient data. Due to the incomplete block design concerning treatment arms between different trials, a mixed treatment comparison (MTC) meta-analysis approach was adopted. The repeated ordinal outcome was modelled through a latent variable, as a proportional odds mixed model with trial, period and treatment arms as covariates. The model was further complexified to account for the variance heterogeneity, and the individual log-residual variance was modelled as a linear mixed model, as well. The effects of individual covariates at inclusion, such as parasitaemia, fever, gender and weight, were also tested. Model parameters were estimated using a Bayesian approach via the WinBUGS software. After selecting the best model using Deviance Information Criterion (DIC), mixed treatment comparisons were based on the estimated treatment effects. Results: Modeling the residual variance improved the model ability to adjust the data. The results showed that, compared to artesunate-amodiaquine (ASAQ), dihydroartemisinin-piperaquine (DHPP) was significantly more efficacious. Artesunate-chlorproguanil-dapsone (ASCD) was less efficacious than artesunate-sulphadoxine-pyrimethamine (ASSP), artemether-lumefantrine (AMLM) and DHPP, the difference with the latter being significant. No difference in efficacy was found between ASAQ and AMLM. Conclusions: Bayesian mixed treatment comparisons of a network of connected randomized trials with repeated measurements of the primary categorical outcome allowed to take into account both the individual- and between- studies sources of heterogeneity. The results of the present study complete the previous quantitative review based on a binary outcome at a fixed time point, suggesting that DHPP represents an alternative for the treatment of uncomplicated P. falciparum malaria in Cameroonian children.

This article is from PLoS ONE , volume 9 . Abstract Background: Edoxaban recently proved non-inferior to warfarin for prevention of thromboembolism in patients with non-valvular atrial fibrillation (AF). We conducted an imputed-placebo analysis with estimates of the proportion of warfarin effect preserved by each non vitamin K antagonist oral anticoagulant (NOAC) and indirect comparisons between edoxaban and different NOACs. Methods and Findings: We performed a literature search (up to January 2014), clinical trials registers, conference proceedings, and websites of regulatory agencies. We selected non-inferiority randomised controlled phase III trials of dabigatran, rivaroxaban, apixaban and edoxaban compared with adjusted-dose warfarin in non-valvular AF. Compared to imputed placebo, all NOACs reduced the risk of stroke (ORs between 0.24 and 0.42, all p

Gene expression levels in a population vary extensively across tissues. Such heterogeneity is caused by genetic variability and environmental factors, and is expected to be linked to disease development. The abundance of experimental data now enables the identification of features of gene expression profiles that are shared across tissues, and those that are tissue-specific. While most current research is concerned with characterising differential expression by comparing mean expression profiles across tissues, it is also believed that a significant difference in a gene expression's variance across tissues may also be associated to molecular mechanisms that are important for tissue development and function. We propose a sparse multi-view matrix factorisation (sMVMF) algorithm to jointly analyse gene expression measurements in multiple tissues, where each tissue provides a different "view" of the underlying organism. The proposed methodology can be interpreted as an extension of principal component analysis in that it provides the means to decompose the total sample variance in each tissue into the sum of two components: one capturing the variance that is shared across tissues, and one isolating the tissue-specific variances. sMVMF has been used to jointly model mRNA expression profiles in three tissues - adipose, skin and LCL - which are available for a large and well-phenotyped twins cohort, TwinsUK. Using sMVMF, we are able to prioritise genes based on whether their variation patterns are specific to each tissue. Furthermore, using DNA methylation profiles available, we provide supporting evidence that adipose-specific gene expression patterns may be driven by epigenetic effects.

The normalized maximum likelihood (NML) is a recent penalized likelihood that has properties that justify defining the amount of discrimination information (DI) in the data supporting an alternative hypothesis over a null hypothesis as the logarithm of an NML ratio, namely, the alternative hypothesis NML divided by the null hypothesis NML. The resulting DI, like the Bayes factor but unlike the p-value, measures the strength of evidence for an alternative hypothesis over a null hypothesis such that the probability of misleading evidence vanishes asymptotically under weak regularity conditions and such that evidence can support a simple null hypothesis. Unlike the Bayes factor, the DI does not require a prior distribution and is minimax optimal in a sense that does not involve averaging over outcomes that did not occur. Replacing a (possibly pseudo-) likelihood function with its weighted counterpart extends the scope of the DI to models for which the unweighted NML is undefined. The likelihood weights leverage side information, either in data associated with comparisons other than the comparison at hand or in the parameter value of a simple null hypothesis. Two case studies, one involving multiple populations and the other involving multiple biological features, indicate that the DI is robust to the type of side information used when that information is assigned the weight of a single observation. Such robustness suggests that very little adjustment for multiple comparisons is warranted if the sample size is at least moderate.

An important issue raised by Efron in the context of large-scale multiple comparisons is that in many applications the usual assumption that the null distribution is known is incorrect, and seemingly negligible differences in the null may result in large differences in subsequent studies. This suggests that a careful study of estimation of the null is indispensable. In this paper, we consider the problem of estimating a null normal distribution, and a closely related problem, estimation of the proportion of non-null effects. We develop an approach based on the empirical characteristic function and Fourier analysis. The estimators are shown to be uniformly consistent over a wide class of parameters. Numerical performance of the estimators is investigated using both simulated and real data. In particular, we apply our procedure to the analysis of breast cancer and HIV microarray data sets. The estimators perform favorably in comparison to existing methods.

Bibliography: leaf 23

For more details, see the University of Florida George A. Smathers library catalog: http://uf.catalog.fcla.edu/permalink.jsp?20UF021867360

For more details, see the University of Florida George A. Smathers library catalog: http://uf.catalog.fcla.edu/permalink.jsp?20UF021867360

Research was conducted to provide educational researchers with a choice of pairwise multiple comparison procedures (P-MCPs) to use with single group repeated measures designs. The following were studied through two Monte Carlo (MC) simulations: (1) The T procedure of J. W. Tukey (1953); (2) a modification of Tukey's T (G. Keppel, 1973); (3) the Dunn-Bonferroni procedure (DB); (4) the sequentially rejective Bonferroni procedure (J. P. Shaffer, 1986) (SB); (5) Hayter's modification of Fisher's Least Significant Difference Test (A. J. Hayter, 1986) (FH); (6) a modified range procedure combining others (SRW); (7) a multiple range procedure based on Ryan-Welsch critical values (MRW) (T. A. Ryan and R. E. Welsch); (8) E. Peritz's (1970) procedure (P); and (9) Welsch's step-up procedure (W). The first MC study was exploratory and was based on variance-covariance matrices that were created so as to conform to different sphericity values. Power in this study was examined for a fixed set of mean differences. The second MC study, based on the results of the first, used variance-covariance matrices found in 100 real repeated measures data sets. Based on study results, the stepwise tests SB, FH, SRW, MRW, and P and the T and W are not recommended, but the DB procedure is recommended for use with single group repeated measures data. (Contains 5 tables, 10 figures, and 34 references.) (SLD)

Major results have been obtained in the areas of nonparametric estimation of quantiles and of density functions under censoring, discrete failure models, and multiple comparisons. In particular, smooth nonparametric estimators of quantile functions from censored data were developed which give better estimates of percentiles of the lifetime distribution than the usual product-limit quantile function. Also, smooth density estimators from censored data were investigated using maximum penalized likelihood procedures. Several parametric models were proposed for the case of discrete failure data. These models provide a better fit to such data than some previously used discrete models. Finally, new methods of constructing simultaneous confidence intervals for pairwise differences of means of normal populations were developed, and the problem of selecting an asymptotically optimal design for comparing several new treatments with a control was solved. Work is continuing on the study of properties of kernel type quantile function estimators and development of goodness-of-fit tests for the model assumptions in accelerated life testing. Keywords: Nonparametric quantile estimation; Density estimation; Right-censored data; Discrete failure models; Multiple comparisons; Accelerated life testing.

Major results have been obtained in the areas of nonparametric estimation of quantiles and of density functions under censoring, discrete failure models, and multiple comparisons. In particular, smooth nonparametric estimators of quantile functions from censored data were developed which give better estimates of percentiles of the lifetime distribution than the usual product-limit quantile function. Also, smooth density estimators from censored data were investigated using maximum penalized likelihood procedures. Several parametric models were proposed for the case of discrete failure data. These models provide a better fit to such data than some previously used discrete models. Finally, new methods of constructing simultaneous confidence intervals for pairwise differences of means of normal populations were developed, and the problem of selecting an asymptotically optimal design for comparing several new treatments with a control was solved. Work is continuing on the study of properties of kernel type quantile function estimators and development of goodness-of-fit tests for the model assumptions in accelerated life testing. Keywords: Nonparametric quantile estimation; Density estimation; Right-censored data; Discrete failure models; Multiple comparisons; Accelerated life testing.

A recent paper by Einot and Gabriel restates a basic result for multiple range tests, which had earlier been given by Tukey and Spjotvoll. It is pointed out that this result requires for its validity an assumption which is not explicitly provided by any of these earlier versions. There is also some discussion of a related result.

Dunnett (1955) developed a procedure comparing m treatments to one control with an exact overall type I error of alpha when all sampling distributions as normal. Sometimes it is desirable to compare m treatments to K or = 2 controls. In particular, it is often desired to compare m treatments with two controls. For instance, several new treatments (e.g., pain relievers) could be compared to two standard treatments (e.g., Aspirin and Tylenol). Alternatively, a standard treatment could be very expensive, difficult to apply and/or have bad side effects, making it useful to compare each new treatment to both standard treatment and not treatment (Placebo). Dunnett's method is expanded here to give comparisons of mean values for m treatments to mean values for K or = 2 controls at an exact overall type I error of a alpha when all sampling distributions are normal. Tabled values needed to make exact simultaneous comparisons at alpha =.05 are given for K=2. An application is made to an illustrative example for the literature.

Dunnett has prepared tables for making multiple comparisons with a control for single-factor (or multi-factor) experiments involving means of normal populations. In the present paper the author uses the multiplicative model for variances which he introduced in an earlier paper, as the basis for making multiple comparisons with a control for multi-factor experiments involving variances of normal populations. A limited set of tables for use with this procedure is included.

Multiple comparison procedures for controlling familywise Type I error and the false discovery rate are described and compared, including the traditional Bonferroni correction, a sequential (step-up) Bonferroni procedure (Hochberg, 1988), and a sequential false discovery rate procedure proposed by Benjamini and Hochberg (1995). Motivation for formally considering the false discovery rate is discussed. A simulation study demonstrates that the Benjamini and Hochberg (BH) technique results in greater power than either of the Bonferroni procedures, and the power advantage increases with the number of inferences in the family. Another important advantage of the BH procedure is its relative consistency about the statistical significance of comparisons over alternative family sizes. It is concluded that, in situations where there is a great number of hypotheses to be tested and strong control of familywise error is unnecessary, it is reasonable to apply the BH technique as a statistical approach to error control. (Contains 12 references, 2 tables, 2 figures, and an appendix of Statistical Analysis System code for implementing the Bonferroni and BH procedures.) (Author)

In many experimental situations the pertinent inferences are made on the basis of orthogonal contrasts among the treatment means (as in 2 to the nth power factorial experiments). In this setting a particularly useful form of inference is one involving multiple comparisons. The present paper describes situations in which such inferences are meaningful, gives examples of their use, and provides an extensive set of tables of constants needed to implement such multiple comparison procedures. The procedures can also be used for statistically legitimate data snooping to help decide which contrasts within a specified set warrant further study.

A large number of pairwise multiple comparisons (P-MCPs) have been introduced recently to the educational research community. The use of these P-MCPs with single group repeated measures data was studied through an exploratory Monte Carlo study of P-MCPs that have been shown to control different types of Type 2 error and Type 1 familywise error under both no violations and violations of assumptions in other designs. A second purpose of the study was to recommend the P-MCPs based on ease of use. The stringent level of robustness developed by J. V. Bradley (1978) was used to examine the P-MCPs empirical rate of Type 1 error, and the range of sphericity was expanded to cover the values found in practice more realistically. Pairwise power among the P-MCPs was also compared. Nine P-MCPs were studied. Results indicate that all the new methods can not be recommended with single group repeated measures designs because their omnibus tests failed to control Type I error adequately. A familiar and easy-to-calculate method, the Dunn-Bonferroni procedure, successfully controlled familywise Type I error and may be recommended for use as a followup procedure with single group repeated measures designs. Further research with single group repeated measures designs through the Studentized maximum modulus statistic is recommended. (Contains 3 tables and 27 references.) (SLD)

A parallel implementation of an efficient method for comparison of multiple DNA sequences is presented. The method is described in terms of a conceptual tree data structure for the sequences begin compared. The parallel algorithm shows efficient utilization of processors on an Encore Multimax computer in a sample comparison of eleven sequences totaling over 4000 bases. Timing data show the strong influence of computer system details on this parallel program. Also presented is a graphics program for displaying multiple sequence comparison output data. The display is capable of representing large volumes of multiple sequence comparison data in a single plot. The program has several additional features that allow closer examination of subsets of sequences. A display of matches from the sample comparison reflects the known structure of these sequences.

Research studies in Educational Data Mining (EDM) often involve several variables related to student learning activities. As such, it may be necessary to run multiple statistical tests simultaneously, thereby leading to the problem of multiple comparisons. The Benjamini-Hochberg (BH) procedure is commonly used in EDM research to address this issue, and it has proven to be a useful method. However, the main limitation of the procedure is that it requires the statistical tests to either be independent or satisfy certain dependency conditions. The Benjamini-Yekutieli (BY) procedure is an alternative that can be applied under arbitrary dependence assumptions, but this extra flexibility comes with a loss of statistical power; hence, the BH procedure is preferred whenever it can be properly applied. Based on these considerations, in this work we employ simulation studies to assess the validity of the BH procedure in two scenarios common to EDM research. The first scenario considers the evaluation and comparison of different classification models--such an analysis might occur, for instance, during the model tuning and validation stage of a study. Then, in the second scenario we look at experiments involving the study of state transitions in sequential data, examples of which occur in affect dynamics research. We find that the BH procedure performs as expected when used with simulated classification model predictions; however, when applied to simulated sequential data, it does not perform at the expected level. Based on these results, as well as previous studies evaluating the BH and BY methods, we discuss the appropriate usage of these procedures for the scenarios under examination. [For the full proceedings, see ED615472.]