Writing test questions is the easiest thing in the world.

  1. The statement is true
  2. The statement is false

Whichever answer you thought was correct, you are right. Writing test questions is easy; writing good test questions is considerably harder. Even today, the most widespread way of checking acquired knowledge in adult education is still the self-check and examination test.

This practice became widespread for a number of reasons:

  • it is easy to score
  • it is easy to turn into summary statistics
  • putting together (a bad) test set requires no particular expertise — neither methodological nor technical

Examining how effective the method really is would deserve a post of its own, so for now let us settle on this: if this format is so widespread, we should put as much thought as possible into assembling the test set, so that the learner’s score and their actual knowledge overlap as closely as possible.

With that in mind, we have collected a few ground rules worth following:

  • Do not ask learners to recall data they have memorised — this is not a memory game. The test result can give the learner a false picture of their own level of knowledge if recognising the right answer among the options is enough to answer the question. To avoid this, try to ask questions where the learner has to interpret, evaluate and use the data and facts. It can be useful, for example, to apply the new knowledge in a specific situation.
  • For practice test sets, an adaptive approach is very useful. By this we mean that the difficulty of the questions increases steadily with the number of correct answers. Unfortunately, very few authoring tools support this.
  • Take the learning objectives into account when assembling the questions; ask about what you set out to teach.
  • The wording of the question should be unambiguous and to the point. It should not contain parts that play no real role in the solution.
  • The wrong answers should be plausible too. Their wording and language should match the correct answer. We often come across tests where the wrong answers are humorous or obviously wrong. That may even be useful in a warm-up phase, but such answers should never be used in question sets that serve as assessment.
  • The wrong answers should be the same length as the correct ones. It is not very practical to give one or two words as the wrong answers while the correct one is a whole paragraph explaining the topic.
  • While we are at it… do not use answers several sentences long, unless the task is about reading comprehension.
  • Avoid attaching a parenthetical clarification or explanation to the correct answer, because it gives away what to choose.
  • Do not use the question type where a word is missing from the question text and has to be picked from the answers. This question type places far more cognitive load on the learner.
  • The order of the answers should be randomised, so that “b” or “c” is not the correct one most of the time. If that is not possible, review the finished test set and change the order of the answers where necessary.
  • Avoid answers of the “the statement is true”, “the statement is false”, “none of these”, “all of these” type. Be more creative! It is still better to have fewer questions than a test set full of pointless ones.
  • Do not write trick questions. It may seem like a good idea at first, but it is incompatible with the goal of measuring the learner’s actual knowledge rather than how well they cope with trick questions.
  • Following on from the previous point: the wrong answers should not differ from the correct one by a hair’s breadth in their wording — this is not a spot-the-difference game.
  • Avoid negative questions, but if you do use one, highlight the negation, for example Which of the following is NOT a dog breed?
  • The same topic should not recur across questions. What appears in one question may be the answer to another.

Sources:

Designing Multiple-Choice Questions. (2017, June 27). Centre for Teaching Excellence. https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/developing-assignments/assignment-design/designing-multiple-choice-questions

Malamed, C. (2019, September 16). 10 Rules For Writing Multiple Choice Questions. The ELearning Coach. https://theelearningcoach.com/elearning_design/rules-for-multiple-choice-questions/

Mcdaniel, R. (2021, December 7). Writing Good Multiple Choice Test Questions. Vanderbilt University. https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/

Author: Mariann Tóth