Staff View: How to grade items for a question bank and rank tests based on student performance

How to grade items for a question bank and rank tests based on student performance

Introduction: Not utilising post-test psychometric analyses of questions, and not maintaining a question bank seemed to adversely affect the quality of tests and increase the workload of academicians, as they are required to write fresh questions for all examinations. The literature review did not r...

Full description

Saved in:

Bibliographic Details
Main Authors:	Thomas, Puthiaparampil, Md Mizanur, Rahman, Henry Rantai, Gudum, Brohi, Imam Bux, Fong, Isabel Lim, Rosalia, Saimon
Format:	Article
Language:	English
Published:	MedEdPublish 2020
Subjects:	RZ Other systems of medicine
Online Access:	http://ir.unimas.my/id/eprint/33026/1/Thomas.pdf http://ir.unimas.my/id/eprint/33026/ https://www.mededpublish.org/manuscripts/3605
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my.unimas.ir.33026
record_format	eprints
spelling	my.unimas.ir.330262021-03-30T08:12:43Z http://ir.unimas.my/id/eprint/33026/ How to grade items for a question bank and rank tests based on student performance Thomas, Puthiaparampil Md Mizanur, Rahman Henry Rantai, Gudum Brohi, Imam Bux Fong, Isabel Lim Rosalia, Saimon RZ Other systems of medicine Introduction: Not utilising post-test psychometric analyses of questions, and not maintaining a question bank seemed to adversely affect the quality of tests and increase the workload of academicians, as they are required to write fresh questions for all examinations. The literature review did not reveal any gold standard method for creating a question bank. This study has formulated criteria for recruiting multiple choice questions for a question bank and introduced a formula to rank whole tests. Methods: We collected used question papers of multiple true-false questions (MCQ) and one best answer questions (BAQ) and got two experienced academicians to scrutinize them and identify items with flaws. The flawless items were counted in each test and their test performance index (TPi) determined. The psychometric item analysis reports of the tests were analysed to enlist bankable items. The TPi of the tests were also calculated by this method. The TPi derived by expert opinions were compared with those obtained by objective criteria using the Pearson Correlation Coefficient and the Spearman’s rho. Results: Judgements by two experts showed a positive correlation, so also expert judgements against the objective formulae. Omission rates in MTF items showed a highly significant negative correlation with difficulty index, falling short of a perfect -1, which supported including omission index in their triple formula. The mean number of functioning distractors (FD) per BAQ item was1.87 (SD 1.14), which supported ≥2 FD per item in their triple formula. Conclusion: Expert judgments in question vetting is essential. However, objective post-test scrutiny of items using difficulty and discrimination indexes enhanced with omission index and distractor efficiency to recruit items for question banks are required. Test Performance Index will be a useful metric to rank the tests. MedEdPublish 2020-11-19 Article PeerReviewed text en http://ir.unimas.my/id/eprint/33026/1/Thomas.pdf Thomas, Puthiaparampil and Md Mizanur, Rahman and Henry Rantai, Gudum and Brohi, Imam Bux and Fong, Isabel Lim and Rosalia, Saimon (2020) How to grade items for a question bank and rank tests based on student performance. MedEdPublish, 2020, Volume 9, Issue 1, Volume (Issue1). pp. 1-7. ISSN ISBN 2312_7996 https://www.mededpublish.org/manuscripts/3605 10.15694/mep.2020.000260.1
institution	Universiti Malaysia Sarawak
building	Centre for Academic Information Services (CAIS)
collection	Institutional Repository
continent	Asia
country	Malaysia
content_provider	Universiti Malaysia Sarawak
content_source	UNIMAS Institutional Repository
url_provider	http://ir.unimas.my/
language	English
topic	RZ Other systems of medicine
spellingShingle	RZ Other systems of medicine Thomas, Puthiaparampil Md Mizanur, Rahman Henry Rantai, Gudum Brohi, Imam Bux Fong, Isabel Lim Rosalia, Saimon How to grade items for a question bank and rank tests based on student performance
description	Introduction: Not utilising post-test psychometric analyses of questions, and not maintaining a question bank seemed to adversely affect the quality of tests and increase the workload of academicians, as they are required to write fresh questions for all examinations. The literature review did not reveal any gold standard method for creating a question bank. This study has formulated criteria for recruiting multiple choice questions for a question bank and introduced a formula to rank whole tests. Methods: We collected used question papers of multiple true-false questions (MCQ) and one best answer questions (BAQ) and got two experienced academicians to scrutinize them and identify items with flaws. The flawless items were counted in each test and their test performance index (TPi) determined. The psychometric item analysis reports of the tests were analysed to enlist bankable items. The TPi of the tests were also calculated by this method. The TPi derived by expert opinions were compared with those obtained by objective criteria using the Pearson Correlation Coefficient and the Spearman’s rho. Results: Judgements by two experts showed a positive correlation, so also expert judgements against the objective formulae. Omission rates in MTF items showed a highly significant negative correlation with difficulty index, falling short of a perfect -1, which supported including omission index in their triple formula. The mean number of functioning distractors (FD) per BAQ item was1.87 (SD 1.14), which supported ≥2 FD per item in their triple formula. Conclusion: Expert judgments in question vetting is essential. However, objective post-test scrutiny of items using difficulty and discrimination indexes enhanced with omission index and distractor efficiency to recruit items for question banks are required. Test Performance Index will be a useful metric to rank the tests.
format	Article
author	Thomas, Puthiaparampil Md Mizanur, Rahman Henry Rantai, Gudum Brohi, Imam Bux Fong, Isabel Lim Rosalia, Saimon
author_facet	Thomas, Puthiaparampil Md Mizanur, Rahman Henry Rantai, Gudum Brohi, Imam Bux Fong, Isabel Lim Rosalia, Saimon
author_sort	Thomas, Puthiaparampil
title	How to grade items for a question bank and rank tests based on student performance
title_short	How to grade items for a question bank and rank tests based on student performance
title_full	How to grade items for a question bank and rank tests based on student performance
title_fullStr	How to grade items for a question bank and rank tests based on student performance
title_full_unstemmed	How to grade items for a question bank and rank tests based on student performance
title_sort	how to grade items for a question bank and rank tests based on student performance
publisher	MedEdPublish
publishDate	2020
url	http://ir.unimas.my/id/eprint/33026/1/Thomas.pdf http://ir.unimas.my/id/eprint/33026/ https://www.mededpublish.org/manuscripts/3605
_version_	1696979533147668480
score	13.211869

How to grade items for a question bank and rank tests based on student performance

Similar Items