Downloads & Free Reading Options - Results

We introduce a new method to detect solar-like oscillations in frequency power spectra of stellar observations, under conditions of very low signal to noise. The Moving-Windowed-Power-Search, or MWPS, searches the power spectrum for signatures of excess power, over and above slowly varying (in frequency) background contributions from stellar granulation and shot or instrumental noise. We adopt a false-alarm approach (Chaplin et al. 2011) to ascertain whether flagged excess power, which is consistent with the excess expected from solar-like oscillations, is hard to explain by chance alone (and hence a candidate detection). We apply the method to solar photometry data, whose quality was systematically degraded to test the performance of the MWPS at low signal-to-noise ratios. We also compare the performance of the MWPS against the frequently applied power-spectrum-of-power-spectrum (PSxPS) detection method. The MWPS is found to outperform the PSxPS method.

An association rule is statistically significant, if it has a small probability to occur by chance. It is well-known that the traditional frequency-confidence framework does not produce statistically significant rules. It can both accept spurious rules (type 1 error) and reject significant rules (type 2 error). The same problem concerns other commonly used interestingness measures and pruning heuristics. In this paper, we inspect the most common measure functions - frequency, confidence, degree of dependence, $\chi^2$, correlation coefficient, and $J$-measure - and redundancy reduction techniques. For each technique, we analyze whether it can make type 1 or type 2 error and the conditions under which the error occurs. In addition, we give new theoretical results which can be use to guide the search for statistically significant association rules.

It is a false, but common, belief that statistical significance testing evaluates result replicability. In truth, statistical significance testing reveals nothing about results replicability. Since science is based on replication of results, methods that assess replicability are important. This is particularly true when multivariate methods, which capitalize on sampling error, are used. This paper explores three methods that can give an idea of the replicability of results in multivariate analysis without having to repeat the study. The first method is cross validation, a replication technique in which the entire sample is first run through the planned analysis and then the sample is randomly split into two unequal parts so that separate analyses are done on each half. The jackknife is a second method of replicability that relies on partitioning out the impact or effect of a particular subset of the data on an estimate derived from the total sample. The bootstrap, a third method of studying replicability, involves copying the data set into an infinitely large "mega" data set. Many different samples are then drawn from the file and results are computed separately for each sample and then averaged. The main drawback of all these internal replicability procedures is that their results are all based on the data from the one sample being analyzed. However, internal replication techniques are better than not addressing the issue at all. (Contains 18 references.) (SLD)

It is a false, but common, belief that statistical significance testing evaluates result replicability. In truth, statistical significance testing reveals nothing about results replicability. Since science is based on replication of results, methods that assess replicability are important. This is particularly true when multivariate methods, which capitalize on sampling error, are used. This paper explores three methods that can give an idea of the replicability of results in multivariate analysis without having to repeat the study. The first method is cross validation, a replication technique in which the entire sample is first run through the planned analysis and then the sample is randomly split into two unequal parts so that separate analyses are done on each half. The jackknife is a second method of replicability that relies on partitioning out the impact or effect of a particular subset of the data on an estimate derived from the total sample. The bootstrap, a third method of studying replicability, involves copying the data set into an infinitely large "mega" data set. Many different samples are then drawn from the file and results are computed separately for each sample and then averaged. The main drawback of all these internal replicability procedures is that their results are all based on the data from the one sample being analyzed. However, internal replication techniques are better than not addressing the issue at all. (Contains 18 references.) (SLD)

We discuss a statistical analysis of Aharonov-Bohm conductance oscillations measured in a two-dimensional ring, in the presence of Rashba spin-orbit interaction. Measurements performed at different values of gate voltage are used to calculate the ensemble-averaged modulus of the Fourier spectrum and, at each frequency, the standard deviation associated to the average. This allows us to prove the statistical significance of a splitting that we observe in the h/e peak of the averaged spectrum. Our work illustrates in detail the role of sample specific effects on the frequency spectrum of Aharonov-Bohm conductance oscillations and it demonstrates how fine structures of a different physical origin can be discriminated from sample specific features.

The main aim of this study is to point out the difficulties found when trying to assess the statistical significance of the North-South asymmetry (hereafter SSNSA) of the most usually considered time series of solar activity. First of all, we distinguish between solar activity time series composed by integer or non-integer and dimensionless data, or composed by non-integer and dimensional data. For each of these cases, we discuss the most suitable statistical tests which can be applied and highlight the difficulties to obtain valid information about the statistical significance of solar activity time series. Our results suggest that, apart from the need to apply the suitable statistical tests, other effects such as the data binning, the considered units and the need, in some tests, to consider groups of data, affect substantially the determination of the statistical significance of the asymmetry. Our main conclusion is that the assessment of the statistical significance of the N-S asymmetry of solar activity is a difficult matter and that an absolute answer cannot be given, since many different effects influence the results given by the statistical tests. In summary, the quantitative results about the statistical significance of the N-S asymmetry of solar activity provided by different authors, as well as the studies about its behaviour, must be considered with care because they depend from the chosen values of different parameters or from the considered units.

A recent analysis of data collected by the Planck satellite detected a net dipole at the location of X-ray selected galaxy clusters, corresponding to a large-scale bulk flow extending at least to $z\sim 0.18$, the median redshift of the cluster sample. The amplitude of this flow, as measured with Planck, is consistent with earlier findings based on data from the Wilkinson Microwave Anisotropy Probe (WMAP). However, the uncertainty assigned to the dipole by the Planck team is much larger than that found in the WMAP studies, leading the authors of the Planck study to conclude that the observed bulk flow is not statistically significant. We here show that two of the three implementations of random sampling used in the error analysis of the Planck study lead to systematic overestimates in the uncertainty of the measured dipole. Random simulations of the sky do not take into account that the actual realization of the sky leads to filtered data that have a 12% lower root-mean-square dispersion than the average simulation. Using rotations around the Galactic pole (the Z axis), increases the uncertainty of the X and Y components of the dipole and artificially reduces the significance of the dipole detection from 98-99% to less than 90% confidence. When either effect is taken into account, the corrected errors agree with those obtained using random distributions of clusters on Planck data, and the resulting statistical significance of the dipole measured by Planck is consistent with that of the WMAP results.

A recent analysis of data collected by the Planck satellite detected a net dipole at the location of X-ray selected galaxy clusters, corresponding to a large-scale bulk flow extending at least to $z\sim 0.18$, the median redshift of the cluster sample. The amplitude of this flow, as measured with Planck, is consistent with earlier findings based on data from the Wilkinson Microwave Anisotropy Probe (WMAP). However, the uncertainty assigned to the dipole by the Planck team is much larger than that found in the WMAP studies, leading the authors of the Planck study to conclude that the observed bulk flow is not statistically significant. We here show that two of the three implementations of random sampling used in the error analysis of the Planck study lead to systematic overestimates in the uncertainty of the measured dipole. Random simulations of the sky do not take into account that the actual realization of the sky leads to filtered data that have a 12% lower root-mean-square dispersion than the average simulation. Using rotations around the Galactic pole (the Z axis), increases the uncertainty of the X and Y components of the dipole and artificially reduces the significance of the dipole detection from 98-99% to less than 90% confidence. When either effect is taken into account, the corrected errors agree with those obtained using random distributions of clusters on Planck data, and the resulting statistical significance of the dipole measured by Planck is consistent with that of the WMAP results.

A recent analysis of data collected by the Planck satellite detected a net dipole at the location of X-ray selected galaxy clusters, corresponding to a large-scale bulk flow extending at least to $z\sim 0.18$, the median redshift of the cluster sample. The amplitude of this flow, as measured with Planck, is consistent with earlier findings based on data from the Wilkinson Microwave Anisotropy Probe (WMAP). However, the uncertainty assigned to the dipole by the Planck team is much larger than that found in the WMAP studies, leading the authors of the Planck study to conclude that the observed bulk flow is not statistically significant. We here show that two of the three implementations of random sampling used in the error analysis of the Planck study lead to systematic overestimates in the uncertainty of the measured dipole. Random simulations of the sky do not take into account that the actual realization of the sky leads to filtered data that have a 12% lower root-mean-square dispersion than the average simulation. Using rotations around the Galactic pole (the Z axis), increases the uncertainty of the X and Y components of the dipole and artificially reduces the significance of the dipole detection from 98-99% to less than 90% confidence. When either effect is taken into account, the corrected errors agree with those obtained using random distributions of clusters on Planck data, and the resulting statistical significance of the dipole measured by Planck is consistent with that of the WMAP results.

This article is from Trials , volume 14 . Abstract None

An explanation of the misuse of statistical significance testing and the true meaning of "significance" is offered. Literature about the criticism of current practices of researchers and publications is reviewed in the context of tests of significance. The problem under consideration occurs when researchers attempt to do more than just establish that a relationship has been observed. More often than not, too many researchers assume that the difference, and even the size of the difference, proves or at least confirms the research hypothesis. Statistical significance is not a measure of "substantive' significance or what might be called scientific importance. Significance testing was designed to yield yes/no decisions. It is suggested that authors or research projects should not try to interpret the magnitudes of their significance findings. Significance testing must be returned to its proper place in the scientific process. (SLD)

Philip Morris Records; bibliography; publication, scientific; extr, extra

The purposes of the present paper are to address the historical development of statistical significance testing and to briefly examine contemporary practices regarding such testing in the light of these historical origins. Precursors leading to the advent of statistical significance testing are examined as are more recent controversies surrounding the issue. As the etiology of current practice is explored, it will become more apparent whether current practices evolved from deliberative judgment or merely developed from happenstance that has become reified in routine. Examination of the history of analysis suggests that the development of statistical significance testing has indeed involved a degree of deliberative judgment. It may be that the time for significance testing came and went, but there is no doubt that significance testing served as an important catalyst for the growth of science in the 20th century. (Contains 39 references.) (Author/SLD)

Too few researchers understand what statistical significance testing does and does not do, and consequently their results are misinterpreted. This Digest explains the concept of statistical significance testing and discusses the meaning of probabilities, the concept of statistical significance, arguments against significance testing, misinterpretation, and alternatives. Statistical significance testing requires subjective judgment in setting a predetermined acceptable probability of making an inferential error caused by the sampling error. Sampling error can only be eliminated by gathering data from the entire population. Statistical significance addresses the question of whether, assuming the sample data came from a population in which the null hypothesis is (exactly) true, the calculated probability of the sample results is less than the acceptable limit imposed regarding a Type I error. Reasons not to use statistical significance testing and questions of misinterpretation are reviewed. Two analyses that should be emphasized over statistical significance testing are effect sizes and the empirical replicability of results. (Contains 6 references.) (SLD)

A better grasp of the physical foundations of life is necessary before we can understand the processes occurring inside a living cell. In his physical theory of the cell, American physiologist Gilbert Ling introduced an important notion of the resting state of the cell. He describes this state as an independent stable thermodynamic state of a living substance in which it has stored all the energy it needs to perform all kinds of biological work. This state is characterised by lower entropy of the system than in an active state. However, Ling's approach is primarily qualitative in terms of thermodynamics and it needs to be characterised more specifically. To this end, we propose a new thermodynamic approach to studying Ling's model of the living cell (Ling's cell), the center piece of which is the non-ergodicity property which has recently been proved for a wide range of systems in statistical mechanics [7]. These approach allowed us to develop general thermodynamic approaches to explaining some of the well-known physiological phenomena, which can be used for further physical analysis of these phenomena using specific physical models.

The controversy about the use or misuse of statistical significance testing has become the major methodological issue in educational research. This special issue contains three articles that explore the controversy, three commentaries on these articles, an overall response, and three rejoinders by the first three authors. They are: (1) "Introduction to the Special Issue on Statistical Significance Testing" (Alan S. Kaufman); (2) "The Data Analysis Dilemma: Ban or Abandon. A Review of Null Hypothesis Significance Testing" (Thomas W. Nix and J. Jackson Barnette); (3) "The Role of Statistical Significance Testing in Educational Research" (James E. McClean and James M. Ernest); (4) "Statistical Significance Testing: A Historical Overview of Misuses and Misinterpretation with Implications for the Editorial Policies of Educational Journals" (Larry G. Daniel); (5) "Statistical Significance and Effect Size Reporting: Portrait of a Possible Future" (Bruce Thompson); (6) "Comments on the Statistical Significance Testing Articles" (Thomas R. Knapp); (7) "What If There Were No More Bickering about Statistical Significance Tests?" (Joel R. Levin); (8) "A Review of Hypothesis Testing Revisited: Rejoinder to Thompson, Knapp, and Levin" (Thomas W. Nix and J. Jackson Barnette); (9) "Fight the Good Fight: A Response to Thompson, Knapp, and Levin" (James M. Ernest and James E. McLean); and (10) "The Statistical Significance Controversy Is Definitely Not Over: A Rejoinder to Responses by Thompson, Knapp, and Levin" (Larry G. Daniel). Each paper contains references. (SLD)

This study looks at the impact of reducing the nominal P-value threshold for statistical significance from 0.05 to 0.005 in previously published Anesthesiology RCTs. This study also looks at the minimum false positive risk (FPR) associated with primary outcomes in these articles

The present paper discusses criticisms of statistical significance testing from both historical and contemporary perspectives. Statistical significance testing is greatly influenced by sample size and often results in meaningless information being over-reported. Variance-accounted-for-effect sizes are presented as an alternative to statistical significance testing. A review of the "Journal of Clinical Psychology" (1993) reveals a continued reliance on statistical significance testing on the part of researchers. Finally, scatterplots and correlation coefficients are presented to illustrate the lack of linear relationship between sample size and effect size. Two figures are included. (Contains 24 references.) (Author)

The use of tests of statistical significance was explored, first by reviewing some criticisms of contemporary practice in the use of statistical tests as reflected in a series of articles in the "American Psychologist" and in the appointment of a "Task Force on Statistical Inference" by the American Psychological Association (APA) to consider recommendations leading to improved practice. Related practices were reviewed in seven volumes of the "School Psychology Quarterly," an APA journal. This review found that some contemporary authors continue to use and interpret statistical significance tests inappropriately. The 35 articles reviewed reported a total of 321 statistical tests for which sufficient information was provided for effect sizes to be computed, but authors of only 19 articles did report various magnitudes of effect indices. Suggestions for improved practice are explored, beginning with the need to interpret statistical significance tests correctly, using more accurate language, and the need to report and interpret magnitude of effect indices. Editorial policies must continue to evolve to require authors to meet these expectations. (Contains 50 references.) (SLD)

This paper begins with a general discussion of statistical significance, effect size, and power analysis; and concludes by extending the discussion to the multivariate case (MANOVA). Historically, traditional statistical significance testing has guided researchers' thinking about the meaningfulness of their data. The use of significance testing alone in making these decisions has proved problematic. It is likely that less reliance on statistical significance testing, and an increased use of power analysis and effect size estimates in combination could contribute to an overall improvement in the quality of new research produced. The more informed researchers are about the benefits and limitations of statistical significance, effect size, and power analysis, the more likely it is that they will be able to make more sophisticated and useful interpretations about the meaningfulness of research results. One table illustrates the discussion. (Contains 37 references.) (SLD)

Researchers increasingly recognize that significance tests are limited in their ability to inform scientific practice. Common errors in interpreting significance tests and three strategies for augmenting the interpretation of significance test results are illustrated. The first strategy for augmenting the interpretation of significance tests involves evaluating significance test results in a sample size context. A second strategy involves interpretation of effect size estimates; several estimates and corrections are discussed. A third strategy emphasizes interpretation based on estimated likelihood that results will replicate. The bootstrap method of B. Efron and others and cross-validation strategies are illustrated. A 28-item list of references and four data tables are included. (Author/SLD)

A test of statistical significance is a procedure for determining how likely a result is assuming a null hypothesis to be true with randomization and a sample of size n (the given size in the study). Randomization, which refers to random sampling and random assignment, is important because it ensures the independence of observations, but it does not guarantee independence beyond the initial sample selection. A test of statistical significance provides a statement of probability of occurrence in the long run, with repeated random sampling under the null hypothesis, but provides no basis for a conclusion about the probability that a particular result is attributable to chance. A test of statistical significance also does not indicate the probability that the null hypothesis is true or false and does not indicate whether a treatment being studied had an effect. Statistical significance indicates neither the magnitude nor the importance of a result, and is no indication of the probability that a result would be obtained on study replication. Although tests of statistical significance yield little valid information for questions of interest in most educational research, use and misuse of such tests remain common for a variety of reasons. Researchers should be encouraged to minimize statistical significance tests and to state expectations for quantitative results as critical effect sizes. There is a 58-item list of references. (SLD)

This article is from BMC Bioinformatics , volume 13 . Abstract Background: Genetic mutation, selective pressure for translational efficiency and accuracy, level of gene expression, and protein function through natural selection are all believed to lead to codon usage bias (CUB). Therefore, informative measurement of CUB is of fundamental importance to making inferences regarding gene function and genome evolution. However, extant measures of CUB have not fully accounted for the quantitative effect of background nucleotide composition and have not statistically evaluated the significance of CUB in sequence analysis. Results: Here we propose a novel measure--Codon Deviation Coefficient (CDC)--that provides an informative measurement of CUB and its statistical significance without requiring any prior knowledge. Unlike previous measures, CDC estimates CUB by accounting for background nucleotide compositions tailored to codon positions and adopts the bootstrapping to assess the statistical significance of CUB for any given sequence. We evaluate CDC by examining its effectiveness on simulated sequences and empirical data and show that CDC outperforms extant measures by achieving a more informative estimation of CUB and its statistical significance. Conclusions: As validated by both simulated and empirical data, CDC provides a highly informative quantification of CUB and its statistical significance, useful for determining comparative magnitudes and patterns of biased codon usage for genes or genomes with diverse sequence compositions.

The mission of the "Journal of Counseling and Development" (JCD) includes the attempt to serve as a "scholarly record of the counseling profession" and as part of the "conscience of the profession." This responsibility requires the willingness to engage in self-study. This study investigated two aspects of research practice in 25 quantitative studies reported in 1996 JCD issues, the use and interpretation of statistical significance tests, and the meaning of and ways of evaluating the score reliabilities of measures used in substantive research inquiry. Too many researchers have persisted in equating result improbability with result value, and too many have persisted in believing that statistical significance evaluates result replicability. In addition, too many researchers have persisted in believing that result improbability equals the magnitude of study effects. Authors must consistently begin to report and interpret effect sizes to aid the interpretations they make and those made by their readers. With respect to score reliability evaluation, more authors need to recognize that reliability inures to specific sets of scores and not to the test itself. Thirteen of the JCD articles involved reports of score reliability in previous studies and eight reported reliability coefficients for both previous scores and those in hand. These findings suggest some potential for improved practice in the quantitative research reported in JCD and improved editorial policies to support these changes. (Contains 39 references.) (SLD)

Statistical significance tests (SSTs) have been the object of much controversy among social scientists. Proponents have hailed SSTs as an objective means for minimizing the likelihood that chance factors have contributed to research results. Critics have both questioned the logic underlying SSTs and bemoaned the widespread misapplication and misinterpretation of the results of these tests. This paper offers a framework for remedying some of the common problems associated with SSTs via modification of journal editorial policies. The controversy surrounding SSTs is reviewed, with attention given to both historical and more contemporary criticisms of bad practices associated with misuse of SSTs. Examples from the editorial policies of "Educational and Psychological Measurement" and several other journals that have established guidelines for reporting results of SSTs are discussed, and suggestions are provided regarding additional ways that educational journals may address the problem. These guidelines focus on selecting qualified editors and reviewers, defining policies about use of SSTs that are in line with those of the American Psychological Association, and stressing effect size reporting. An appendix presents a manuscript review form. (Contains 61 references.) (Author/SLD)

This article is from Annals of Ibadan Postgraduate Medicine , volume 6 . Abstract While it’s not the intention of the founders of significance testing and hypothesis testing to have the two ideas intertwined as if they are complementary, the inconvenient marriage of the two practices into one coherent, convenient, incontrovertible and misinterpreted practice has dotted our standard statistics textbooks and medical journals. This paper examine factors contributing to this practice, traced the historical evolution of the Fisherian and Neyman-Pearsonian schools of hypothesis testing, exposed the fallacies and the uncommon ground and common grounds approach to the problem. Finally, it offers recommendations on what is to be done to remedy the situation.

Assessing variability according to distinct factors in data is a fundamental technique of statistics. The method commonly regarded to as analysis of variance (ANOVA) is, however, typically confined to the case where all levels of a factor are present in the data (i.e. the population of factor levels has been exhausted). Random and mixed effects models are used for more elaborate cases, but require distinct nomenclature, concepts and theory, as well as distinct inferential procedures. Following a hierarchical Bayesian approach, a comprehensive ANOVA framework is shown, which unifies the above statistical models, emphasizes practical rather than statistical significance, addresses issues of parameter identifiability for random effects, and provides straightforward computational procedures for inferential steps. Although this is done in a rigorous manner the contents herein can be seen as ideological in supporting a shift in the approach taken towards analysis of variance.

A national survey of a stratified random sample of members of the American Educational Research Association was undertaken to explore perceptions of contemporary statistical issues, and especially of statistical significance tests. The 225 actual respondents were found to be reasonably representative of the population from which the sample was drawn. The respondents had sophisticated understanding of some statistical issues (e.g., that nonsignificant results may still be important), but other features of the perceptions (e.g., perceptions of stepwise analysis) were not as encouraging. (Contains 1 table, 9 figures, and 28 references.) (Author/SLD)

This paper suggests that statistical significance testing and effect size are two sides of the same coin; they complement each other, but do not substitute for one another. Good research practice requires that both should be taken into consideration to make sound quantitative decisions. A Monte Carlo simulation experiment was conducted, and a three-factor crossed design, with 500 replications within each cell, was implemented in the simulation. The sampling variability of two popular effect sizes ("d" and "R squared") was empirically obtained under different data conditions. It is shown empirically that there is considerable variability of sample effect size measure, and the extent of sampling variability of effect size measures is strongly influenced by sample size. Although that which is statistically significant may not be practically meaningful, that which appears to be a practically meaningful effect size could occur by chance (i.e., sampling error), thus not trustworthy. It is pointed out that statistical significance testing and effect size measurement serve different purposes, and the sole reliance on either may be misleading. Some practical guidelines are recommended for combining statistical significance testing and effect size measure for making decisions in quantitative analysis. (Contains 2 tables, 3 figures, and 20 references.) (Author/SLD)

Achieving the Dream (ATD) is one of 12 higher education and digital learning organizations that make up the Every Learner Everywhere (Every Learner) Network, whose mission is to help higher education institutions improve and ensure more equitable student outcomes through advances in digital learning, particularly among poverty-impacted, racially minoritized, and first-generation students. Every Learner partners are addressing high failure rates in foundational courses through the provision of scalable, high-quality support to colleges and universities seeking to implement adaptive courseware on their campuses. In this case study, faculty in several disciplines at Lorain County Community College (LCCC) in Ohio explored the use of adaptive courseware, ultimately implementing it in a gateway statistics course and several business courses. This case study is part of a series of studies conducted by ATD examining how adaptive courseware is implemented at those institutions as well as how courseware is used in particular disciplines to better serve students. Case studies are based on a series of interviews with college leaders, faculty, instructional designers, developers, technology specialists and students who were enrolled in classes using the courseware.

The purposes of this paper are to: (1) describe some of the serious shortcomings in the current use of tests of statistical significance, (2) discuss how misuses are perpetuated in some widely used references, and (3) present an alternative significance testing model that overcomes some, but not all, of the shortcomings of the currently used method. For the purposes of this paper, the discussion is restricted to fixed effects analysis of variance (ANOVA) (including t-tests), which is perhaps the most pervasive of the data analyses used by educational researchers. (Author)

Scientists developing environmental monitoring programs must consider the ultimate question: Has a significant impact occurred? This question represents a primary concern of environmentalists and regulatory agencies alike. Therefore, the investigator must design baseline and trend assessment studies in such a way as to allow detection of environmental impacts. However, to properly address this question the investigator should have a basic definition of a 'significant' impact. Typically, it is approached as two components: 1) What is a statistical impact? and 2) What is an ecological impact? In order for an impact to be considered ecologically significant, it really should be statistically significant. However, the converse is not necessarily true. In fact, it would be desirable to design a monitoring program which would allow the statistical detection of ecological changes before they become critical. This document offers strategies for defining statistical impacts for an environmental monitoring program. Specifically, a series of statistical techniques have been developed to estimate 'minimum detectable impacts' (MDIs) for variables examined during the baseline phase of a monitoring program at an open ocean dredged material disposal site. The MDIs are dependent upon natural spatiotemporal variability of baseline data and the intensity of the monitoring effort.

Scientists developing environmental monitoring programs must consider the ultimate question: Has a significant impact occurred? This question represents a primary concern of environmentalists and regulatory agencies alike. Therefore, the investigator must design baseline and trend assessment studies in such a way as to allow detection of environmental impacts. However, to properly address this question the investigator should have a basic definition of a 'significant' impact. Typically, it is approached as two components: 1) What is a statistical impact? and 2) What is an ecological impact? In order for an impact to be considered ecologically significant, it really should be statistically significant. However, the converse is not necessarily true. In fact, it would be desirable to design a monitoring program which would allow the statistical detection of ecological changes before they become critical. This document offers strategies for defining statistical impacts for an environmental monitoring program. Specifically, a series of statistical techniques have been developed to estimate 'minimum detectable impacts' (MDIs) for variables examined during the baseline phase of a monitoring program at an open ocean dredged material disposal site. The MDIs are dependent upon natural spatiotemporal variability of baseline data and the intensity of the monitoring effort.

The incorporation of uncertainties to calculations of signal significance in planned experiments is an actual task. Several approaches to this problem are discussed. We present a procedure for taking into account the systematic uncertainty related to nonexact knowledge of signal and background cross sections. A method of a treatment of statistical errors of the expected signal and background rates is proposed. The interrelation between Gamma- and Poisson distributions is demonstrated.

This paper evaluates the logic underlying various criticisms of statistical significance testing and makes specific recommendations for scientific and editorial practice that might better increase the knowledge base. Reliance on the traditional hypothesis testing model has led to a major bias against nonsignificant results and to misinterpretation of significant results. A finding of statistical significance does not mean that the null hypothesis is false, since there are many factors affecting statistical significance such as sample size and the measurement reliability of the data. Furthermore, statistical significance alone does not permit evaluation of the importance of a finding. An effect size statistic, such as eta-squared, is more appropriate for this purpose, and editors of scholarly publications should encourage routine reporting of effect sizes. Greater reporting of nonsignificant results should also be encouraged, accompanied by power analyses to estimate Type II error. Nonsignificant results can be meaningful if the study's power to detect an effect was high. Finally, the paper emphasizes the crucial role of replication in separating true effects from Type I errors. The paper integrates analyses and criticisms of statistical practice from a variety of sources--77 references are included. (LPG)

A research design is described for the situation in which a program, particularly a compensatory education program funded by Chapter 1 of the Hawkins Stafford Act of 1988, can be evaluated when there is no available comparison group and no pretest data. The design requires content specialists to identify which objectives on the posttest were included in the compensatory curriculum (C objectives) and which were included only in the regular curriculum (R objectives). Compensatory students should perform better on the C objectives to which they were exposed in both regular and compensatory curricula than on the R objectives. The analysis would be a simple t-test of the differences between two groups, the C items and the R items. The design is valuable because: (1) students serve as their own control group; (2) it is not necessary to identify a test that can measure pretest and posttest knowledge; and (3) it allows for identification of successful components of the Chapter 1 program. Two exhibits illustrate sample designs. Five figures and two tables present the analysis method and results from a 20-item test for 47 students in grades 1, 2, and 3 in 1987-88 and 34 students in 1988-89. An eight-item list of references is included. (SLD)

Perma.cc archive of https://hbr.org/2016/02/a-refresher-on-statistical-significance created on 2021-12-26 19:55:41+00:00.

Next-generation sequencing technology enables the identification of thousands of gene regulatory sequences in many cell types and organisms. We consider the problem of testing if two such sequences differ in their number of binding site motifs for a given transcription factor (TF) protein. Binding site motifs impart regulatory function by providing TFs the opportunity to bind to genomic elements and thereby affect the expression of nearby genes. Evolutionary changes to such functional DNA are hypothesized to be major contributors to phenotypic diversity within and between species; but despite the importance of TF motifs for gene expression, no method exists to test for motif loss or gain. Assuming that motif counts are Binomially distributed, and allowing for dependencies between motif instances in evolutionarily related sequences, we derive the probability mass function of the difference in motif counts between two nucleotide sequences. We provide a method to numerically estimate this distribution from genomic data and show through simulations that our estimator is accurate. Finally, we introduce the R package {\tt motifDiverge} that implements our methodology and illustrate its application to gene regulatory enhancers identified by a mouse developmental time course experiment. While this study was motivated by analysis of regulatory motifs, our results can be applied to any problem involving two correlated Bernoulli trials.

Next-generation sequencing technology enables the identification of thousands of gene regulatory sequences in many cell types and organisms. We consider the problem of testing if two such sequences differ in their number of binding site motifs for a given transcription factor (TF) protein. Binding site motifs impart regulatory function by providing TFs the opportunity to bind to genomic elements and thereby affect the expression of nearby genes. Evolutionary changes to such functional DNA are hypothesized to be major contributors to phenotypic diversity within and between species; but despite the importance of TF motifs for gene expression, no method exists to test for motif loss or gain. Assuming that motif counts are Binomially distributed, and allowing for dependencies between motif instances in evolutionarily related sequences, we derive the probability mass function of the difference in motif counts between two nucleotide sequences. We provide a method to numerically estimate this distribution from genomic data and show through simulations that our estimator is accurate. Finally, we introduce the R package {\tt motifDiverge} that implements our methodology and illustrate its application to gene regulatory enhancers identified by a mouse developmental time course experiment. While this study was motivated by analysis of regulatory motifs, our results can be applied to any problem involving two correlated Bernoulli trials.

In this paper we propose a technique for spectral envelope estimation using maximum values in the sub-bands of Fourier magnitude spectrum (MSASB). Most other methods in the literature parametrize spectral envelope in cepstral domain such as Mel-generalized cepstrum etc. Such cepstral domain representations, although compact, are not readily interpretable. This difficulty is overcome by our method which parametrizes in the spectral domain itself. In our experiments, spectral envelope estimated using MSASB method was incorporated in the STRAIGHT vocoder. Both objective and subjective results of analysis-by-synthesis indicate that the proposed method is comparable to STRAIGHT. We also evaluate the effectiveness of the proposed parametrization in a statistical parametric speech synthesis framework using deep neural networks.

Structural equation modeling (SEM) has been deemed as a proper method when variables contain measurement errors. In contrast, path analysis with composite-scores is preferred for prediction and diagnosis of individuals. While path analysis with composite-scores has been criticized for yielding biased parameter estimates, recent literature pointed out that the population values of parameters in a latent-variable model depend on artificially assigned scales. Consequently, bias in parameter estimates is not a well-grounded concept for models involving latent constructs. This article compares path analysis with composite-scores against SEM with respect to effect size and statistical power in testing the significance of the path coefficients, via the z- or t-statistics. The data come from many sources with various models that are substantively determined. Results show that SEM is not as powerful as path analysis even with equally-weighted-composites. But path analysis with Bartlett-factor- scores and the partial-least-squares approach to SEM perform the best with respect to effect size and power. [This paper will be published in "Behavior Research Methods." Discrepancy between the title of the article and authored paper.]

This paper presents a testing procedure that incorporates three key elements. The first element is the use of non-nil null hypotheses. The second element is the determination of a practically significant level that is incorporated into the corresponding non-nil null hypothesis. The third element is the use of a randomization test to statistically test each non-nil null hypothesis. This procedure stresses two philosophical positions. First, the concepts of practical and statistical significance are both essential components in the evaluation process. Second, the use of this procedure encourages the researchers to consider the process of establishing the level of practical significance, not only as a statistical one, but also, as one in which the researchers would consider societal concerns and cost versus benefit comparisons. An appendix contains the computer program for the randomization test. (Contains 38 references.) (Author/SLD)

We use a volume-limited sample of quasars in the Sloan Digital Sky Survey (SDSS) DR7 quasar catalog to identify quasar groups and address their statistical significance. This quasar sample has a uniform selection function on the sky and nearly a maximum possible contiguous volume that can be drawn from the DR7 catalog. Quasar groups are identified by using the Friend-of-Friend algorithm with a set of fixed comoving linking lengths. We find that the richness distribution of the richest 100 quasar groups or the size distribution of the largest 100 groups are statistically equivalent with those of randomly-distributed points with the same number density and sky coverage when groups are identified with the linking length of 70 h-1Mpc. It is shown that the large-scale structures like the huge Large Quasar Group (U1.27) reported by Clowes et al. (2013) can be found with high probability even if quasars have no physical clustering, and does not challenge the initially homogeneous cosmological models. Our results are statistically more reliable than those of Nadathur (2013), where the test was made only for the largest quasar group. It is shown that the linking length should be smaller than 50 h-1Mpc in order for the quasar groups identified in the DR7 catalog not to be dominated by associations of quasars grouped by chance. We present 20 richest quasar groups identified with the linking length of 70 h-1Mpc for further analyses.

Controversy over the merits of Null Hypothesis Statistical Significance Testing (NHST) as a tool for advancing knowledge in the social sciences has intensified in recent years. Literature for and against the use of statistical significant tests is reviewed and three major limitations of these tests are summarized. The first is that "p" values themselves cannot be used as indices of effect size. A second limitation is the recognition that unlikely results are not necessarily interesting or important. A third limitation is that "p" values do not bear on the important issue of result replicability because statistical tests do not test the possibility that sample results occur in the population. A summary is also presented of what NHST can and cannot do. (Contains 1 table and 48 references.) (SLD)

Experimental research on behavior and cognition frequently rests on stimulus or subject selection where not all characteristics can be fully controlled, even when attempting strict matching. For example, when contrasting patients to controls, variables such as intelligence or socioeconomic status are often correlated with patient status. Similarly, when presenting word stimuli, variables such as word frequency are often correlated with primary variables of interest. One procedure very commonly employed to control for such nuisance effects is conducting inferential tests on confounding stimulus or subject characteristics. For example, if word length is not significantly different for two stimulus sets, they are considered as matched for word length. Such a test has high error rates and is conceptually misguided. It reflects a common misunderstanding of statistical tests: interpreting significance not to refer to inference about a particular population parameter, but about 1. the sample in question, 2. the practical relevance of a sample difference (so that a nonsignificant test is taken to indicate evidence for the absence of relevant differences). We show inferential testing for assessing nuisance effects to be inappropriate both pragmatically and philosophically, present a survey showing its high prevalence, and briefly discuss an alternative in the form of regression including nuisance variables.

As the 1994 publication manual of the American Psychological Association emphasized, "p" values are affected by sample size. As a result, it can be helpful to interpret the results of statistical significant tests in a sample size context by conducting so-called "what if" analyses. However, these methods can be inaccurate unless "corrected" effect sizes are used. This paper proposes a new method by which "what if" analyses can be conducted using estimated true population effects. Two appendixes contain EXCEL spreadsheet commands for previous "what if" methods and the present method. (Contains 4 tables and 48 references.) (Author/SLD)

The aim of the present work is to assess the extent to which comparisons of interventions for active treatment versus no treatment from Cochrane meta-analyses in various fields of medicine that report statistically significant results would have been interpreted differently (“suggestive” rather than “statistically significant”) if the p-value threshold was shifted from the 0.05 routine threshold for claiming success to .005.

Three criticisms of overreliance on results from statistical significance tests are noted. It is suggested that: (1) statistical significance tests are often tautological; (2) some uses can involve comparisons that are not completely sensible; and (3) using statistical significance tests to evaluate both methodological assumptions (e.g., the homogeneity of variance or of regression assumptions) and substantive hypotheses creates inescapable dilemmas. Three strategies for augmenting statistical significance testing are elaborated. First, a review of effect sizes is presented. Second, a method for evaluating statistical significance in a sample size context is discussed. Finally, strategies for empirically evaluating whether results will replicate are reviewed, with an emphasis on explaining one computer-intensive resampling strategy (the bootstrap method). It is inconsistent to use sample results to estimate population values, but to be unwilling to consult the sample to estimate the variability and shape of samples drawn from the population. Three tables and one figure illustrate the discussion, and a 79-item list of references is included. (Author/SLD)

Perma.cc archive of https://www.iwh.on.ca/what-researchers-mean-by/statistical-significance created on 2022-02-25 04:47:08.746176+00:00.

We discuss problems the null hypothesis significance testing (NHST) paradigm poses for replication and more broadly in the biomedical and social sciences as well as how these problems remain unresolved by proposals involving modified p-value thresholds, confidence intervals, and Bayes factors. We then discuss our own proposal, which is to abandon statistical significance. We recommend dropping the NHST paradigm--and the p-value thresholds intrinsic to it--as the default statistical paradigm for research, publication, and discovery in the biomedical and social sciences. Specifically, we propose that the p-value be demoted from its threshold screening role and instead, treated continuously, be considered along with currently subordinate factors (e.g., related prior evidence, plausibility of mechanism, study design and data quality, real world costs and benefits, novelty of finding, and other factors that vary by research domain) as just one among many pieces of evidence. We have no desire to "ban" p-values or other purely statistical measures. Rather, we believe that such measures should not be thresholded and that, thresholded or not, they should not take priority over the currently subordinate factors. We also argue that it seldom makes sense to calibrate evidence as a function of p-values or other purely statistical measures. We offer recommendations for how our proposal can be implemented in the scientific publication process as well as in statistical decision making more broadly.