Multiple choice questions

Multiple choice questions (MCQs) are possibly the most frequently used (written) means of testing students' knowledge, and one of the most common of the teacher-constructed tests. Despite their widespread use, there is not much written about their "format, structure, validity and reliability ... and most of the current literature in this area is based on opinion or consensus".

Typical multiple choice questions will consist of a stem question and some options/choices including:
 * The correct answer, and
 * a number of distractors:
 * an answer that is plausible but not quite correct
 * an answer that is actually half right but is also half wrong
 * an answer that is wrong and perhaps even ridiculous.

Pros and cons
The advantages of MCQs are that they require less time for the students to answer and are more easily marked and graded than other types of tests.

On the downside, they are actually very difficult and time-consuming to prepare well in that they require time to draw up effective stem questions and corresponding choices. Teachers may actually have no formal training in setting tests/exams and/or marking them correctly, and a relatively typical mistake made by teachers making up their own multiple choice questions is that of asking ambiguous stem questions or questions that do not always have a precise answer. It might be a good idea to draw up the questions (and the corresponding answers) in advance and then return to them some days later to check whether they stand the test of time, i.e., are "good" questions with "good" answers.

Other criticisms include the possibility that MCQs don't actually test relevant knowledge and may even "lead to assessment of trivial knowledge".

Probability
Although the fact is that, in the case shown above, there's a minimum 25% probability of correctly answering each question, any student with more than an inkling of the subject matter will greatly improve that probability (by eliminating that ridiculous answer). This leads to criticism that students may be rewarded not for knowing the correct answer, but for "knowing that an answer is incorrect". Obviously a 5-choice question will lower that probability. To further reduce some of the statistically relevant probability of getting a correct answer without actually knowing anything about the subject and/or to discourage mere guesswork, universities have, for some years now, applied negative marking or penalty scoring, that is, awarding, for instance, one point for a correct answer, no point for a question left unanswered, and a minus point for a wrong answer, or variations on that theme. This editor has it on good authority that if the penalty is lower than the probability of guessing correctly, it makes sense to guess. In other words, if the penalty for a wrong answer is -0.25 for questions with five choices, it's "better" to guess. One survey, carried out for the Royal College of Anaesthetists, showed that while all testees clearly benefited from making educated guesses, only 3 out of 27 actually lost marks from making wild guesses.

On the other hand, recent research suggests that MCQs offering only two distractors are just as good at evaluating knowledge, "require less time to develop and administer and additional options provide no psychometric advantage," and encourages teachers to adopt three-option items, i.e., two distractors, for multiple-choice questions.

In praise of MCQs
This editor makes use of MCQs not as a testing tool but as an ongoing revision activity, in which the group as a whole has to agree on an answer and then justify that answer, and explain why the other options are not acceptable.