Estimating Effect Sizes, Heterogeneity Parameters and Weighted Standard Deviation (WSD) of Postgraduate Theses using Meta-Analytic and Systematic Review Methods

Simon Ntumi^1,

¹Department of Educational Foundations, University of Education, Winneba (UEW), West Africa, Ghana

Cite this paper:
Simon Ntumi. Estimating Effect Sizes, Heterogeneity Parameters and Weighted Standard Deviation (WSD) of Postgraduate Theses using Meta-Analytic and Systematic Review Methods. American Journal of Applied Mathematics and Statistics. 2024; 12(3):41-54. doi: 10.12691/ajams-12-3-2

Abstract

The purpose of the study was to conduct a meta-analytic from an institutional repository by exploring the statistical analysis and reporting practices of postgraduate students’ theses in the University of Cape Coast. To achieve this, the study was nested into the quantitative approach where archival data were retrieved from University of Cape Coast institutional repository (UCCIR). The 2020 PRISMA chart flow was used to extract 778 researched studies from the UCCIR. The study found overall medium resultant significant effect size indicating the extracted studies may have some limited practical applications (Min. ES=.378; p=.000; Hg=1.72, z=12.20; Std Err=.238; n=752; Max. ES=.430; p=.012; Hg=.812, z=14.12, Std Err=.623; Overall=.591, p=.000**). Again, the results from the heterogeneity analysis also showed that there was some degree of probability sampling errors in the extracted studies. Also, the study found that most of the extracted studies were likely to produce similar results and conclusions as a result of similar statistical methods employed by the students (T²=.627, p=.023**, df=6, Z=16.12; CI=95%, n=752; Pq=.934, p=.002**, df=6, Z=12.01; CI=95%; Tau=.723, p=.004**, df=6, Z=30.01; CI=95%; n=752; WSD=.075, p=.013**, df=6, Z=17.23; CI=95%; n=752). From the ensued findings, it was concluded that there were some statistical misapplications and misinterpretations of studies results leading to different conclusions and recommendations from the studies. The study recommended that to improve upon the robustness and rigorousness of postgraduate studies in Ghana, research mechanisms such as series and timely training workshops should be put in place by the university to re-orient and expose postgraduate students to modern statistical methods.

Keywords:
statistical analysis postgraduate thesis meta-analysis reporting practices institutional repository

This work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Figures

References:

[1]	Baird, M. D., & Pane, J. F. (2019). Translating standardised effects of education programmes into more interpretable metrics. Educational Researcher, 4(8), 217-228.
[2]	Matthew, A. S. & Sunday, O. M. (2014). The Role of Statistical Software in Data Analysis. International Journal of Applied Research and Studies, 3(8), 1-15.
[3]	Petrişor, A. I. (2019). The use and misuse of statistics in research: Theory and practice. Romanian Statistical Review, 2(2), 59-70.
[4]	Žerovnik, J. (2015). About use and misuse of statistics in education: On mathematics exams at general matura in Slovenia. Education Practice and Innovation, 2(1), 1-7.
[5]	Khusainova, R. M., Shilova, Z. V., & Curteva, O. V. (2016). Selection of appropriate statistical methods for research results processing. International Electronic Journal of Mathematics Education, 11(1), 303-315.
[6]	Gardenier, J., & Resnik, D. (2002). The misuse of statistics: concepts, tools, and a research agenda. Accountability in Research: Policies and Quality Assurance, 9(2), 65-74.
[7]	Pigott, T. D. (2020). Missing data in meta-analysis. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (3rd ed.). New York, NY: Russell Sage Foundation.
[8]	Calzon, B. (2021). Misleading statistics examples discover the potential for misuse of statistics & data in the digital age. News, Insights and Advice for Getting your Data in Shape.
[9]	Ntumi, S. (2021). Reporting and Interpreting Multivariate Analysis of Variance (MANOVA): Adopting the Best Practices in Educational Research. Journal of Research in Educational Sciences, 12(14), 48-57.
[10]	Sohn, S. Y., Rees, P., Wildridge, B., Kalk, N. J., & Carter, B. (2019). Prevalence of problematic smartphone usage and associated mental health outcomes amongst children and young people: A systematic review, meta-analysis and GRADE of the evidence. BMC Psychiatry, 19(1), 1-10.
[11]	Sabharwal, M. (2018). The use of soft computing technique of decision tree in selection of appropriate statistical test for hypothesis testing. In Soft computing: Theories and applications (pp. 161-169). Springer, Singapore.
[12]	Male, F., & Jensen, J. L. (2022). Three common statistical Lyu missteps we make in reservoir characterisation. AAPG Bulletin, 106(11), 2149-2161.
[13]	Mishra, P., Pandey, C. M., Singh, U., & Gupta, A. (2018). Scales of measurement and presentation of statistical data. Annals of Cardiac Anaesthesia, 21(4), 419-434.
[14]	Mandelboum, S., Manber, Z., Elroy-Stein, O., & Elkon, R. (2019). Recurrent functional misinterpretation of RNA-SEQ data caused by sample-specific gene length bias. PLoS Biology, 17(11), 370-381.
[15]	Ntumi, S., & Twum Antwi-Agyakwa, K. (2022). A Systematic Review of Reporting of Psychometric Properties in Educational Research. Mediterranean Journal of Social & Behavioural Research, 6(2), 53-59.
[16]	Bland, J. M. & Altman, D. G. (2017). Misleading statistics: Errors in textbooks, software and manuals. International Journal of Epidemiology, 17(2), 45-79.
[17]	Ghasemi, A., & Zahediasl, S. (2012). Normality tests for statistical analysis: a guide for non-statisticians. International Journal of Endocrinology and Metabolism, 10(2), 486-489.
[18]	Bettany‐Saltikov, J., & Whittaker, V. J. (2014). Selecting the most appropriate inferential statistical test for your quantitative research study. Journal of Clinical Nursing, 23(11-12), 1520-1531.
[19]	Matthew, H., Thiese, G., Zachary, G., Arnold, D., Skyler, S. & Walker, F. (2015). Statistics and ethics in medical research. Misuse of statistics is unethical. British Medical Journal, 19(2), 281-291.
[20]	Bacci, S., & Chiandotto, B. (2019). Introduction to statistical decision theory: Utility theory and causal analysis. Chapman and Hall/CRC.
[21]	Berger, J. O. (2013). Statistical decision theory and Bayesian analysis. Springer Science & Business Media.
[22]	Tawfik, G. M., Dila, K. A. S., Mohamed, M. Y. F., Tam, D. N. H., Kien, N. D., Ahmed, A. M., & Huy, N. T. (2019). A step by step guide for conducting a systematic review and meta-analysis with simulation data. Tropical Medicine and Health, 47(1), 1-9.
[23]	Kelley, J. M., Kraft-Todd, G., Schapira, L., Kossowsky, J., & Riess, H. (2014). The influence of the patient-clinician relationship on healthcare outcomes: A systematic review and meta-analysis of randomised controlled trials. Plos One, 9(4), 194-207.
[24]	Moreno-Peral, P., Conejo-Ceron, S., Rubio-Valera, M., Fernandez, A., Navas-Campaña, D., Rodriguez-Morejon, A., & Bellón, J. Á. (2017). Effectiveness of psychological and/or educational interventions in the prevention of anxiety: A systematic review, meta-analysis, and meta-regression. JAMA Psychiatry, 74(10), 1021-1029.
[25]	Fry, D., Fang, X., Elliott, S., Casey, T., Zheng, X., Li, J. & McCluskey, G. (2018). The relationships between violence in childhood and educational outcomes: A global systematic review and meta-analysis. Child Abuse & Neglect, 7(5), 16-28.
[26]	Garzón, J., Pavón, J., & Baldiris, S. (2019). Systematic review and meta-analysis of augmented reality in educational settings. Virtual Reality, 23(4), 447-459.
[27]	Rigabert, A., Motrico, E., Moreno-Peral, P., Resurrección, D. M., Conejo-Ceron, S., Cuijpers, P., & Bellón, J. Á. (2020). Effectiveness of online psychological and psychoeducational interventions to prevent depression: Systematic review and meta-analysis of randomised controlled trials. Clinical Psychology Review, 82(10), 19-31.
[28]	Ntumi, S. (2021). Reporting and Interpreting One-Way Analysis of Variance (ANOVA) Using a Data-Driven Example: A Practical Guide for Social Science Researchers. Journal of Research in Educational Sciences, 12(14), 38-47.
[29]	Wald, L. J. (1950). The theory of statistical decision. Journal of the American Statistical Association, 46(3), 55-67.
[30]	Williams, P. J., & Hooten, M. B. (2016). Combining statistical inference and decisions in ecology. Ecological Applications, 26(6), 1930-1942.
[31]	Insua, D. R., González-Ortega, J., Banks, D., & Ríos, J. (2018). Concept uncertainty in adversarial statistical decision theory. In The mathematics of the uncertain (pp. 527-542). Springer, Cham.
[32]	Huang, J., Yuan, Q., Zhang, B., Xu, K., Tankam, P., Clarkson, E., & Rolland, J. P. (2014). Measurement of a multi-layered tear film phantom using optical coherence tomography and statistical decision theory. Biomedical Optics Express, 5(12), 4374-4386.
[33]	Sackett, C. A., Kielpinski, D., King, B. E., Langer, C., Meyer, V., Myatt, C. J., & Monroe, C. (2017). Experimental entanglement of four particles. Nature, 404(6775), 256-259.
[34]	Akhter, S., Pauyo, T., & Khan, M. (2019). What is the difference between a systematic review and a meta-analysis? Basic Methods Handbook for Clinical Orthopaedic Research, 7(8), 331-342.
[35]	Mengist, W., Soromessa, T., & Legese, G. (2020). Method for conducting systematic literature review and meta-analysis for environmental science research. Methods, 7(3), 100-109.
[36]	Masuri, M. G., Othman, N., Wicaksono, G., & Isa, K. A. M. (2021). Translation and Validation of the Indonesian Version of SaringSikap Assessment. Environment-Behaviour Proceedings Journal, 6(16), 283-289.
[37]	Bolland, M. J., Grey, A., & Avenell, A. (2018). Effects of vitamin D supplementation on musculoskeletal health: a systematic review, meta-analysis, and trial sequential analysis. The Lancet Diabetes & Endocrinology, 6(11), 847-858.
[38]	Giang, H. T. N., Ahmed, A. M., Fala, R. Y., Khattab, M. M., Othman, M. H. A., Abdelrahman, S. A. M., & Huy, N. T. (2019). Methodological steps used by authors of systematic reviews and meta-analysis of clinical trials: A cross-sectional.
[39]	Zhang, W., & Creswell, J. (2013). The use of “mixing” procedure of mixed methods in health services research. Medical Care, 51(8), e51-e57.
[40]	Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2011). Introduction to meta-analysis. Chichester, England: Wiley.
[41]	Iyengar, S., & Greenhouse, J. B. (2019). Sensitivity analysis and diagnostics. Handbook of Research Synthesis and Meta-Analysis, 8(7), 417-433.
[42]	Aloe, A. M., & Garside, R. (2021). Types of methods research papers in the journal Campbell systematic reviews. Campbell Systematic Reviews, 17(2), 1-9.
[43]	Brinckmann, J., Grichnik, D., & Kapsa, D. (2010). Should entrepreneurs plan or just storm the castle? A meta-analysis on contextual factors impacting the business planning performance relationship in small firms. Journal of Business Venturing, 25(1), 24-40.
[44]	Dalton, B., Bartholdy, S., Robinson, L., Solmi, M., Ibrahim, M. A., Breen, G., & Himmerich, H. (2018). A meta-analysis of cytokine concentrations in eating disorders. Journal of Psychiatric Research, 10(3), 252-264.
[45]	Schäfer, T., & Schwarz, M. A. (2019). The meaningfulness of effect sizes in psychological research: Differences between sub-disciplines and the impact of potential biases. Frontiers in Psychology, 10(2), 8-13.
[46]	Chatterjee, K. (2017). Statistical fallacies in orthopaedics research. Indian Journal of Orthopaedics, 41(3), 37-46.
[47]	Hamrick, L. R., Haney, A. M., Kelleher, B. L., & Lane, S. P. (2020). Using generalizability theory to evaluate the comparative reliability of developmental measures in neurogenetic syndrome and low-risk populations. Journal of Neurodevelopmental Disorders, 1(2), 1-15.
[48]	Frey, B. (2018). The SAGE encyclopaedia of educational research, measurement, and evaluation (Vols. 1-4). Thousand Oaks, CA: SAGE Publications, Inc.
[49]	Goh, J. X., Hall, J. A., & Rosenthal, R. (2016). Mini meta‐analysis of your own studies: Some arguments on why and a primer on how. Social and Personality Psychology Compass, 10(10), 535-549.
[50]	Colquhoun, D. (2017). The reproducibility of research and the misinterpretation of p-values. Royal Society Open Science, 4(12), 171-185.
[51]	Benjamin, D. J., & Berger, J. O. (2019). Three recommendations for improving the use of p-values. The American Statistician, 73(1), 186-191.
[52]	Heo, M., Kim, N., & Faith, M. S. (2015). Statistical power as a function of Cronbach alpha of instrument questionnaire items. BMC Medical Research Methodology, 15(1), 1-9.
[53]	Shrestha, N. (2020). Detecting multicollinearity in regression analysis. American Journal of Applied Mathematics and Statistics, 8(2), 39-42.
[54]	Apfelbaum, E. P., Phillips, K. W., & Richeson, J. A. (2014). Rethinking the baseline in diversity research: Should we be explaining the effects of homogeneity?. Perspectives on Psychological Science, 9(3), 235-244.
[55]	Omrani, H., Shamsi, M., & Emrouznejad, A. (2022). Evaluating sustainable efficiency of decision-making units considering undesirable outputs: an application to airline using integrated multi-objective DEA-TOPSIS. Environment, Development and Sustainability, 1(3), 1-32.
[56]	Bahar, B., Pambuccian, S. E., Barkan, G. A., & Akdaş, Y. (2019). The use and misuse of statistical methods in cytopathology studies: review of 6 journals. Laboratory Medicine, 50(1), 8-15.
[57]	Lucena, C., Lopez, J. M., Pulgar, R., Abalos, C., & Valderrama, M. J. (2013). Potential errors and misuse of statistics in studies on leakage in endodontics. International Endodontic Journal, 46(4), 323-331.
[58]	Worthy, G. (2015). Statistical analysis and reporting: common errors found during peer review and how to avoid them. Swiss Medical Weekly, 145(5), 1-7.