Avoid using complex questions or answers
The traditional MCQ is a single question with a single correct answer and several wrong answers (or distractors). Some question writers make this more complex by providing answers such as ‘none of the above’ or ‘option A and B but not C’. Research suggests that these questions are more difficult to write well and that they can be particularly susceptible to learners guessing, or engaging with cognitive processes that are focused on test-taking strategies rather than the content the question is concerned with.
Simpler questions mean that different learners are more likely to be engaging in the same cognitive processes when answering them. Butler argues that for both assessment and learning, you want learners to be using the cognitive processes you intend them to.
Create questions that require engaging with specific cognitive processes
It is important to consider how a question addresses both specific content and specific ways of thinking about that content. One way to achieve this is by using resources that provide template questions, or shells, based around different cognitive processes. For example, questions such as ‘Which best defines X?’ or ‘Which distinguishes X from Y?’ Frameworks such as Bloom’s taxonomy can also be used to consider how questions can address different types of thinking.
Use three plausible possible answers
The literature finds that a question with three answers is the format that strikes the best balance between beneficial engagement with the question and the amount of time it takes a student to answer. However, the more important aspect of this is that all the possible answers must be plausible — and if this means only having two options, this is preferable. Plausible answers are often common errors or misconceptions. Some MCQ platforms allow educators to analyse incorrect answers, which can help them to identify the misconceptions or errors learners are making. When using MCQs for learning, it can help to use correct answers to later questions as distractors in earlier questions, but only if they could plausibly be possible answers.
Sets of questions should be challenging, but not too difficult
For assessment, questions should be able to determine whether a student has understood a concept or not. Educators should therefore pitch MCQs at a level at which students get some answers right and some wrong. This allows the educator to assess the level across the group. The same applies in a learning situation, as questions can be used to identify which learners may need additional support in learning the material.
Across all of these key considerations, Butler emphasises that it is important to provide feedback to help learners understand where they are at and continue to learn. This feedback can be effective whether it is immediate or after a delay.
Whether you are using multiple-choice questions to assess students or support their learning, considering these key points will help you to make the most of this powerful tool.
