Direct Comparison of Online Tests Using Single-choice Items or Multiple-select Items in Pharmacology over One Year

Joachim Neumann¹, Stephanie Simmrodt¹, Beatrice Bader², Bertram Opitz² and Ulrich Gergs^1,

¹Institute for Pharmacology and Toxicology, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany

²Center for multimedia-enhanced teaching and learning (@LLZ), Martin Luther University Halle-Wittenberg, Halle (Saale), Germany

Cite this paper:
Joachim Neumann, Stephanie Simmrodt, Beatrice Bader, Bertram Opitz and Ulrich Gergs. Direct Comparison of Online Tests Using Single-choice Items or Multiple-select Items in Pharmacology over One Year. American Journal of Educational Research. 2023; 11(3):125-132. doi: 10.12691/education-11-3-4

Abstract

One aim of testing medical students is to assess their level of knowledge. Another aim of examinations is that they can help to enhance the storage in memory for a longer time, at best, till they are in clinical practice. There is some debate what format of online examinations can be more useful to attain these aims. Here, in a formative online test, we compared single-answer items (SC) with multiple-select items (MS) for the very same learning objectives in a study of pharmacology in medical students. Medical students were randomly divided into two groups: group A was first given 15 SC (#1-15) followed by 15 different MS (#16-30). The opposite design was used for group B. One year later, four groups were formed from the previous two groups and were again given the same online test, but in a different order of questions. The main result was that all students fared better in the second test than in the initial test. But in the second test, students fared better when in the first test the same 15 items were asked in the MS mode than in the SC mode. In summary, we provide evidence that MS is better for knowledge retention over one year than SC. We speculate that MS is useful in formative tests to prepare students for any type of examination.

Keywords:
single answer multiple-select items pharmacology medical students

This work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Figures

References:

[1]	Desjardins, I., Touchie, C., Pugh, D., Wood, T. J., and Humphrey-Murto, S., “The impact of cueing on written examinations of clinical decision making: a case study,” Medical education, 48 (3). 255-261. 2014.
[2]	Verbic, S., “Information value of multiple response questions,” Psihologija, 45 (4). 467-485. 2012.
[3]	Elstein, A. S., “Beyond multiple-choice questions and essays: the need for a new way to assess clinical competence,” Academic medicine : journal of the Association of American Medical Colleges, 68 (4). 244-249. 1993.
[4]	Veloski, J. J., Rabinowitz, H. K., Robeson, M. R., and Young, P. R., “Patients don't present with five choices: an alternative to multiple-choice tests in assessing physicians' competence,” Academic Medicine [online], 74 (5). 539-546. 1999.
[5]	Schuwirth, L. W., van der Vleuten, C. P., and Donkers, H. H., “A closer look at cueing effects in multiple-choice questions,” Medical education, 30 (1). 44-49. 1996.
[6]	Krebs, R., “The Swiss Way to Score Multiple True-False Items: Theoretical and Empirical Evidence,” Advances in Medical Education, edited by A. J. J. A. Scherpbier, et al., Vol. 12, Springer Netherlands, Dordrecht, 1997, 158-161.
[7]	Krebs, R., “Anleitung zur Herstellung von MC-Fragen und MC-Prüfungen für die ärztliche Ausbildung.,” Bern, 2004.
[8]	Neumann, J., Gergs, U., Simmrodt, S., Aslan, A., and Teichert, H., “Direct comparison of very short answer versus single best answer questions for medical students in a pharmacology course.,” Association for Medical Education in Europe, Vienna, Abstractbook; Vol. 1068, 2019.
[9]	Sam, A. H., Hameed, S., Harris, J., and Meeran, K., “Validity of very short answer versus single best answer questions for undergraduate assessment,” BMC medical education, 16 (1). 266. Oct. 2016.
[10]	Sam, A. H., Field, S. M., Collares, C. F., van der Vleuten, C. P. M., Wass, V. J., Melville, C., Harris, J., and Meeran, K., “Very-short-answer questions: reliability, discrimination and acceptability,” Medical education, 52 (4). 447-455. Feb. 2018.
[11]	Sam, A. H., Fung, C. Y., Wilson, R. K., Peleva, E., Kluth, D. C., Lupton, M., Owen, D. R., Melville, C. R., and Meeran, K., “Using prescribing very short answer questions to identify sources of medication errors: a prospective study in two UK medical schools,” BMJ open, 9 (7). e028863. Jul. 2019.
[12]	Wood, T., “Assessment not only drives learning, it may also help learning,” Medical education, 43 (1). 5-6. 2009.
[13]	McConnell, M. M., St-Onge, C., and Young, M. E., “The benefits of testing for learning on later performance,” Advances in health sciences education : theory and practice, 20 (2). 305-320. 2015.
[14]	Melzer, A., Gergs, U., Lukas, J., and Neumann, J., “Rating Scale Measures in Multiple-Choice Exams: Pilot Studies in Pharmacology,” Education Research International, 2018 (1). 1-12. 2018.
[15]	Neumann, J., Simmrodt, S., Teichert, H., and Gergs, U., “Poster Abstracts, 21st Annual Meeting of the International Association of Medical Science Educators, Burlington, VT, USA, June 10-13, 2017: Problems when using online mock examination in pharmacology.,” Medical Science Educator, 27 (S1). 19-94. 2017.
[16]	Roediger, H. L., and Karpicke, J. D., “The Power of Testing Memory: Basic Research and Implications for Educational Practice,” Perspectives on psychological science : a journal of the Association for Psychological Science 1 (3). 181-210. 2006.
[17]	Roediger, H. L., and Karpicke, J. D., “Test-enhanced learning: taking memory tests improves long-term retention,” Psychological science, 17 (3). 249-255. 2006.
[18]	Feldman, M., Fernando, O., Wan, M., Martimianakis, M. A., and Kulasegaram, K., “Testing Test-Enhanced Continuing Medical Education: A Randomized Controlled Trial,” Academic medicine : journal of the Association of American Medical Colleges, 93 (11S). S30-S36. 2018.
[19]	Newble, D. I., Baxter, A., and Elmslie, R. G., “A comparison of multiple-choice tests and free-response tests in examinations of clinical competence,” Medical education, 13 (4). 263-268. 1979.
[20]	Frisbie, D. A., “The Multiple True-False Item Format: A Status Review,” Educational Measurement: Issues and Practice, 11 (4). 21-26. 1992.
[21]	Tarasowa, D., and Auer, S., “Balanced Scoring Method for Multiple-mark Questions,” Proceedings of the 5th International Conference on Computer Supported Education, SciTePress - Science and and Technology Publications, 06.05.2013 - 08.05.2013, 411-416.
[22]	Albanese, M. A., and Sabers, D. L., “Multiple True-False Items: A Study of Interitem Correlations, Scoring Alternatives, and Reliability Estimation,” Journal of Educational Measurement, 25 (2). 111-123. 1988.
[23]	Haladyna, T. M., “The Effectiveness of Several Multiple-Choice Formats,” Applied Measurement in Education, 5 (1). 73-88. 1992.
[24]	Tsai, F.-J., and Suen, H. K., “A Brief Report on a Comparison of Six Scoring Methods for Multiple True-False Items,” Educational and Psychological Measurement, 53 (2). 399-404. 1993.
[25]	Wu, B. C., Scoring Multiple True False Items: A Comparison of Summed Scores and Response Pattern Scores at Item and Test Levels., Taiwan, 2003.
[26]	Bauer, D., Holzer, M., Kopp, V., and Fischer, M. R., “Pick-N multiple choice-exams: a comparison of scoring algorithms,” Advances in health sciences education : theory and practice, 16 (2). 211-221. 2011.
[27]	Siddiqui, N. I., Bhavsar, V. H., Bhavsar, A. V., and Bose, S., “Contemplation on marking scheme for Type X multiple choice questions, and an illustration of a practically applicable scheme,” Indian journal of pharmacology, 48 (2). 114-121. 2016.
[28]	Durning, S. J., Dong, T., Ratcliffe, T., Schuwirth, L., Artino, A. R., Boulet, J. R., and Eva, K., “Comparing Open-Book and Closed-Book Examinations: A Systematic Review,” Academic medicine: journal of the Association of American Medical Colleges, 91 (4). 583-599. 2016.