When writing multiple choice questions always consider white space on the page. Cluttered pages will cause even the most diligent learner to feel overwhelmed and possibly shut down. If you have staff with vision problems, or who battle to deal with the written word, white space will help them to be more successful in the test.
Use a one-inch margin at least, with one-and-a-half or double line spacing, when formatting the question-and-answer choices. If there are staff members who struggle with handwriting or have dyslexia, put the answer choices in capital letters: A, B, C and D. These letters are more accurately assessed than lowercase letters: a, b, c and d.
The question itself is referred to as a ‘stem’. The stem should be a complete question, it should be meaningful by itself and it should present a definite problem. For example:
Money laundering is:
- a South African phenomenon
- an African phenomenon
- A global phenomenon
- a Third World phenomenon
Avoid too many ‘which of the following’ items in the stem. These questions require participants to read every option and can penalise slow readers in a time-test situation.
Don’t ask questions that can be answered from common knowledge.
Also, watch out for ‘interlocking’ items. These are items in which a learner can discern the answer to one question from the content of another. Review carefully all items that share similar options.
In a similar vein, don’t ask learners to use their answer to one question to answer another. If they get the first question wrong, they will automatically get the other question wrong as well, even if they understand the concept tested in the second question.
State the question in a positive form whenever possible. For example:
Financial services documentation for all customers who have a relationship with an FSP must be stored for:
- three years from the date of the start of the relationship
- three years from termination of the relationship
- five years from the date of the start of the relationship
- five years from termination of the relationship
This is an example of a negative question:
Which of the options concerning financial services in relation to the storage of documentation for customers who have a relationship with an FSP would not be correct:
- during the relationship
- three years from termination of the relationship
- five years from the date of the conclusion of the relationship
- five years from termination of the relationship.
Use multi-level thinking with questions that include wording such as ‘the most appropriate’ or ‘the most important’. Such questions serve to test the learners’ judgement skills or understanding of an in-depth subject. For example:
Describing human rights as a ‘contract’ suggests that human rights:
- cannot be cancelled
- have a moral but not a legal force
- have a moral force above the law
- are an agreement that has the force of law.
Distractors are there to lead the learner away from the correct answer. Keep the distractors the same length as the answer. For example:
The DMAIC model stands for:
- define, measure, analyse, improve and control
- design, measure, anticipate, improvise and control
- detect, measure, anticipate, innovation and control
- determine, measure, anticipate, innovation and control.
Avoid using distractors with minute differences. For example:
The most correct statement to describe a step in the money-laundering process is:
- layering, to conceal the criminal origin of the proceeds of a crime
- layering, to change the criminal origin of the proceeds of crime
- layering, to conceal the criminal origin of the proceeds of crime
- layering, to conceal the origin from the proceeds of crime.
Provide a minimum of three answer choices, but no more than five. Take care to vary the position of the correct answer. In addition, use multiple-choice responses that force learners to analyse and synthesise the information from the text and the non-text features to answer the question. Check that the answer choices are believable or could make sense. This ensures that the learner has to apply knowledge to select the correct answer. If answers start with similar terms or phrases, move them to the stem.
Your company has had an oil spill. What is done first?
- The company will need to clean up the spill.
- The company will need to inform the relevant authorities.
- The company will need to cordon off the area.
- The company will need to secure the water supply.
The question should be rephrased as follows:
Your company has had an oil spill. What should the company do first?
- clean up the spill
- inform the relevant authorities
- cordon off the area
- secure the water supply.
Avoid using answers that state ‘all of the above’, ‘none of the above’, or ‘A’ and ‘B’. These types of answers can provide clues to the answer. Organise the answers in a logical order, ie alphabetical, numerical or shortest to longest. In addition, make the answers grammatically parallel with the stem. So, if one choice starts with a verb, they all should start with a verb. In this example, Option B is grammatically incorrect because it does not start with a verb:
The major purpose of Monte Carlo analysis is to:
- determine the probability of risk
- validity patterns of risk responses is determined
- check the risk-testing reliability
- evaluate the overall risk process.
Avoid including a word(s) in the stem that furnishes the learner with a verbal association with the correct option, such as using the articles ‘a’ or ‘an’, which gives a clue.
Monitoring a job function is an:
In a ‘fill in the blank’ question, do not put the blank at the beginning of the question. For example: ‘________________ is an example of money laundering.’
This is preferable: ‘An example of money laundering is ________________.’
Never use qualifiers unless you bring them to the learners’ attention. For example:
Which of the following is the most correct definition:
- Money laundering is a form of criminal activity.
- Money-laundering legislation criminalises the crime of money laundering.
- Money laundering may be any act that involves the proceeds of criminal activity.
- Money laundering is criminalised by the act to which it relates.
Answer sheet options for online systems
When the GRC professional consults with an online developer, the GRC professional may need to explain how they wish the answers to be formulated. The following explanations of certain terms may help:
Radio buttons: A radio button or option button is a type of graphical user interface (GUI) that allows the user to choose only one of a predefined set of options.
Check boxes: A check box or tick box is a GUI element (widget) that permits the user to make multiple selections from a number of options.
Drop downs: A drop-down list is a user interface control GUI element, similar to a list box, which allows the user to choose one value from a list. When a drop-down list is inactive, it displays a single value. When activated, it drops down a list of values, from which the user may select one.
Select lists: A list box is a GUI widget that allows the user to select one or more items from a list contained within a static, multiple-line text box. The user clicks inside the box on an item to select it, sometimes in combination with the shift key or control key, in order to make multiple selections.