Multiple-choice items can be used to measure knowledge outcomes and various types of learning outcomes.
They are most widely used for measuring knowledge, comprehension, and application outcomes. The multiple-choice item provides the most useful format for measuring achievement at various levels of learning. When selection-type items are to be used (multiple-choice, true-false, matching, check all that apply) an effective procedure is to start each item as a multiple-choice item and switch to another item type only when the learning outcome and content make it desirable to do so.
1. Learning outcomes from simple to complex can be measured.
2. Highly structured and clear tasks are provided.
3. A broad sample of achievement can be measured.
4. Incorrect alternatives provide diagnostic information.
5. Scores are less influenced by guessing than true-false items.
6. Scores are more reliable than subjectively scored items (e.g., essays).
7. Scoring is easy, objective, and reliable.
8. Item analysis can reveal how difficult each item was and how well it discriminated between the strong and weaker students in the class
9. Performance can be compared from class to class and year to year
10. Can cover a lot of material very efficiently (about one item per minute of testing time).
11. Items can be written so that students must discriminate among options that vary in degree of correctness.
12. Avoids the absolute judgments found in True-False tests.
1. Constructing good items is time consuming.
2. It is frequently difficult to find plausible distractors.
3. This item is ineffective for measuring some types of problem solving and the ability to organize and express ideas.
4. Real-world problem solving differs – a different process is involved in proposing a solution versus selecting a solution from a set of alternatives.
5. Scores can be influenced by reading ability.
6. There is a lack of feedback on individual thought processes – it is difficult to determine why individual students selected incorrect responses.
7. Students can sometimes read more into the question than was intended.
8. Often focus on testing factual information and fails to test higher levels of cognitive thinking.
9. Sometimes there is more than one defensible “correct” answer.
10. They place a high degree of dependence on the student’s reading ability and the instructor’s writing ability.
11. Does not provide a measure of writing ability.
12. May encourage guessing.
Multiple-Choice Item Writing Guidelines
Multiple-choice questions typically have 3 parts:
a stem, the correct answer – called the key, and several wrong answers, called distractors.
• Use either the best answer or the correct answer format.
• Best answer format refers to a list of options that can all be correct in the sense that each has an advantage, but one of them is the best.
• Correct answer format refers to one and only one right answer.
• Format the items vertically, not horizontally (i.e., list the choices vertically)
• Allow time for editing and other types of item revisions.
• Use good grammar, punctuation, and spelling consistently.
• Minimize the time required to read each item.
• Avoid trick items. • Use the active voice.
• The ideal question will be answered by 60-65% of the tested population.
• Have your questions peer-reviewed.
• Avoid giving unintended cues – such as making the correct answer longer in length than the distractors. Content-related Rules: • Base each item on an educational or instructional objective of the course, not trivial information.
• Test for important or significant information.
• Focus on a single problem or idea for each test item.
• Keep the vocabulary consistent with the examinees’ level of understanding.
• Avoid cueing one item with another; keep items independent of one another.
• Use the author’s examples as a basis for developing your items. • Avoid overly specific knowledge when developing items.
• Avoid textbook, verbatim phrasing when developing the items.
• Avoid items based on opinions.
• Use multiple-choice to measure higher level thinking.
• Be sensitive to cultural and gender issues.
• Use case-based questions that use a common text to which a set of questions refers. Stem Construction Rules:
• State the stem in either question form or completion form.
• When using a completion form, don’t leave a blank for completion in the beginning or middle of the stem.
• Ensure that the directions in the stem are clear, and that wording lets the examinee know exactly what is being asked.
• Avoid window dressing (excessive verbiage) in the stem.
• Word the stem positively; avoid negative phrasing such as “not” or “except.” If this cannot be avoided, the negative words should always be highlighted by underlining or capitalization: Which of the following is NOT an example ……
• Include the central idea and most of the phrasing in the stem.
• Avoid giving clues such as linking the stem to the answer (…. Is an example of an: test-wise students will know the correct answer should start with a vowel)
General Option Development Rules:
• Place options in logical or numerical order.
• Use letters in front of options rather than numbers; numerical answers in numbered items may be confusing to students.
• Keep options independent; options should not be overlapping. • Keep all options homogeneous in content.
• Keep the length of options fairly consistent.
• Avoid, or use sparingly, the phrase all of the above.
• Avoid, or use sparingly, the phrase none of the above.
• Avoid the use of the phrase I don’t know.
• Phrase options positively, not negatively.
• Avoid distractors that can clue test-wise examinees; for example, absurd options, formal prompts, or semantic (overly specific or overly general) clues.
• Avoid giving clues through the use of faulty grammatical construction. • Avoid specific determinates, such as never and always.
• Position the correct option so that it appears about the same number of times in each possible position for a set of items.
• Make sure that there is one and only one correct option.
Distractor (incorrect options) Development Rules:
• Use plausible distractors.
• Incorporate common errors of students in distractors.
• Avoid technically phrased distractors.
• Use familiar yet incorrect phrases as distractors.
• Use true statements that do not correctly answer the item.
• Avoid the use of humor when developing options.
• Distractors that are not chosen by any examinees should be replaced.
sourcce: Dawn M. Zimmaro, Ph.D. , 2010 (http://www6.cityu.edu.hk/edge/workshop/seminarseries/2010-11/Seminar03-WritingGoodMultipleChoiceExams.pdf)