SECOND-GENERATION P-VALUES: A NEW PRACTICAL APPROACH FOR EDUCATIONAL RESEARCH

Authors

  • Sally A. Lesik Central Connecticut State University, USA
  • Anna M. Duffin Central Connecticut State University, USA

DOI:

https://doi.org/10.46827/ejes.v11i12.5712

Keywords:

statistical inference, p-value, second-generation p-value, interval null hypothesis, effect size, practical significance, statistical significance

Abstract

The limitations, misconceptions, and misuse of traditional p-values remain a concern for researchers in education. Second-generation p-values provide a descriptive measure that can be used to decide between the null and alternative hypotheses. They are intuitive and easy to understand and provide a new approach to addressing some of the limitations of traditional p-values, mostly by considering the practical implication of an estimated effect. This article describes the general idea and principles behind second-generation p-values and illustrates their application in educational research and data science. Limitations and directions for further study are discussed.

 

Article visualizations:

Hit counter

Author Biographies

  • Sally A. Lesik, Central Connecticut State University, USA

    Sally A. Lesik is a Professor of Mathematical Sciences at Central Connecticut State University where she teaches undergraduate and graduate courses in mathematics, statistics, and data science. Dr. Lesik’s research agenda is focused on applied statistics, mathematics and statistics education, and policy evaluation in higher education.

  • Anna M. Duffin, Central Connecticut State University, USA

    Anna M. Duffin is an Assistant Professor of Mathematical Sciences at Central Connecticut State University, where she teaches undergraduate courses in mathematics, mathematics education, and statistics. Her research interests focus on mathematics education, statistics education, curriculum development, and strategies for supporting multilingual learners in the mathematics classroom.

References

Armstrong, J.S. and Hubbard, R. (2008). Why we don’t really know what “statistical significance” means: A major educational failure, SSRN Electronic Journal [Preprint]. doi:10.2139/ssrn.1154386.

Blume, J.D., Greevy, R.A., Welty, V.F., Smith, J.R. and Dupont, W.D. (2019). An Introduction to Second-Generation p-Values. The American Statistician, 73(sup1), pp.157–167. doi: https://doi.org/10.1080/00031305.2018.1537893.

Blume, J.D, D’Agostino McGowan, L, Dupont, W.D, Greevy, R.A., (2018) Second- generation p-values: Improved rigor, reproducibility, & transparency in statistical analyses. PLoS ONE 13(3): e0188299. https://doi.org/10.1371/journal.pone.0188299.

Bormann, S.-K. (2022). ‘A Stata implementation of second-generation P-values’, The Stata Journal: Promoting communications on statistics and Stata, 22(3), pp. 496–520. doi:10.1177/1536867x221124466.

Campbell, H. and Gustafson, P. (2019). ‘The World of Research Has Gone Berserk: Modeling the Consequences of Requiring “Greater Statistical Stringency” for Scientific Publication’, The American Statistician, 73(sup1), pp. 358–373. doi: 10.1080/00031305.2018.1555101.

Cohen, M.P. (2021). ‘Why not an interval null hypothesis?’, Journal of Data Science, 17(2), pp. 383–390. doi:10.6339/jds.201904_17(2).0008.

Colquhoun, D. (2017). The reproducibility of research and the misinterpretation of p -values. Royal Society Open Science, 4(12), p.171085. doi: https://doi.org/10.1098/rsos.171085.

Das D, Das T. (2023). The “P”‑value: The primary alphabet of research revisited. International Journal Preventive Medicine, 14(41).

Diaz-Quijano, F.A., Calixto, F.M. and Da Silva, J.M.N. (2020). 'How feasible is it to abandon statistical significance? A reflection based on a short survey,' BMC Medical Research Methodology, 20(1). https://doi.org/10.1186/s12874-020-01030-x.

Di Leo, G. and Sardanelli, F. (2020). Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach. European Radiology Experimental, [online] 4(1). doi: https://doi.org/10.1186/s41747-020-0145-y.

Edgeworth, F.Y. (1885). Methods of Statistics, Jubilee volume of the Statist. Soc., E.

Gao, J. (2020). P-values – a chronic conundrum. BMC Medical Research Methodology, 20(1). doi: https://doi.org/10.1186/s12874-020-01051-6.

Goodman, S. (2008). A Dirty Dozen: Twelve P-Value Misconceptions. Seminars in Hematology, 45(3), pp.135–140. doi: https://doi.org/10.1053/j.seminhematol.2008.04.003.

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical test, p value, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31(4), 337-350. DOI: 10.1007/s10654-016-0149-3.

Hurlbert, S. H., Levine, R. A. and Utts, J. (2019). Coup de Grâce for a Tough Old Bull: “Statistically Significant” Expires’, The American Statistician, 73(sup1), pp. 352–357. doi: 10.1080/00031305.2018.1543616.

Kmetz, J. L. (2019). Correcting Corrupt Research: Recommendations for the Profession to Stop Misuse of p-Values. The American Statistician, 73(sup1), 36–45. https://doi.org/10.1080/00031305.2018.1518271.

Lin, M., Lucas, H.C. and Shmueli, G. (2013). ‘research commentary—too big to fail: Large samples and the p-value problem’, Information Systems Research, 24(4), pp. 906–917. doi:10.1287/isre.2013.0480.

McShane, B.B., Gal, D., Gelman, A., Robert, C. and Tackett, J.L. (2019). Abandon Statistical Significance. The American Statistician, [online] 73(sup1), pp.235–245. doi: https://doi.org/10.1080/00031305.2018.1527253.

Package ‘SGPV’. (2022). Available at: https://cran.r-project.org/web/packages/sgpv/sgpv.pdf [Accessed 2 May 2024].

Pericchi, L., Pereira, C., and Perez, M. (2014). Adaptive revised standards for statistical evidence. Proceedings of the National Academy of Sciences of the United States of America. 111. 10.1073/pnas.1322191111.

Robert,D.(2020, June26).Demystifying the p-value. The Startup. https://www.tandfonline.com/doi/full/10.1080/00031305.2019.1583913.

Stahel, W.A., (2021). New relevance and significance measures to replace p-values. PLoS ONE 16(6): e0252991. https://doi.org/10.1371/journal.pone.0252991.

Stewart, T.G. and Blume, J.D. (2019). Second-Generation P-Values, Shrinkage, and Regularized Models. Frontiers in Ecology and Evolution, 7. doi: https://doi.org/10.3389/fevo.2019.00486.

Vidgen, B. and Yasseri, T. (2016). ‘P-values: Misunderstood and misused’, Frontiers in Physics, 4. doi:10.3389/fphy.2016.00006.

Wasserstein, R.L., Schirm, A. L. and Lazar, N. A. (2019). ‘Moving to a World Beyond “p < 0.05”’, The American Statistician, 73(sup1), pp. 1–19. doi: 10.1080/00031305.2019.1583913.

Wasserstein, R.L. and Lazar, N.A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, [online] 70(2), pp.129–133. doi: https://doi.org/10.1080/00031305.2016.1154108 .

Zuo, Y., Stewart, T. G., & Blume, J. D. (2022). ProSGPV: An R package for variable selection with second-generation P-values. F1000Research, 11, 58. https://doi.org/10.12688/f1000research.74401.1.

Downloads

Published

2024-12-10

Issue

Vol. 11 No. 12 (2024)

Section

Articles

License

Authors who publish with this journal agree to the following terms:

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).