One type of objective question is multiple choice.  We all know what it is but let’s look in detail at its description anyway.

Source: https://www.msu.edu/dept/soweb/writitem.html

Description of a multiple choice item:

Presents a problem or question in the stem of the item and requires the selection of the best answer or option. The options consist of a most-correct answer and one or more distractors or foils.

The major purpose of a multiple choice item is to identify examinees who do not have complete command of the concept or principle involved.

Properties:
• State the problem in the stem
• Include one correct or most defensible answer
• Select diagnostic foils or distractors such as:

o Clichés
o Common misinformation
o Logical interpretations
o Partial answers
o Technical terms or textbook jargon

The distractors must appear reasonable as the correct answer to the students who have not mastered the material.

So the structure of a multiple choice question is a stem followed by options.  The options contain one correct answer and a set of distractors.

The Stem

Some advice for constructing a good stem is

Source: http://www.iub.edu/~best/pdf_docs/better_tests.pdf

• Write questions that test a significant concept, that are unambiguous, and that don’t give test-wise students an advantage
• The stem should fully state the problem and all qualifications. Always include a verb in the statement
• Items should measure students’ ability to comprehend, apply, analyze, and evaluate as well as recall
• Include words in the stem that would otherwise be repeated in each option
• Eliminate excessive wording and irrelevant information in the stem

Here are some examples of good and bad stem design:

Source:  http://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/#stem

A stem that does not present a clear problem, however, may test students’ ability to draw inferences from vague descriptions rather serving as a more direct test of students’ achievement of the learning outcome.

If a significant learning outcome requires negative phrasing, such as identification of dangerous laboratory or clinical practices, the negative element should be emphasized with italics or capitalization.

A question stem is preferable because it allows the student to focus on answering the question rather than holding the partial sentence in working memory and sequentially completing it with each alternative

The best thought about the stem I have seen on the Internet:

Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf

Way to judge a good stem: students who know the content should be able to answer before reading the alternatives.

The Options

Sometimes known as “the alternatives”, they are composed of one right answer and a group of “foils” or distractors.

One point that is emphasized regularly in the resources is that the distractors should all be plausible and attractive answers.

Source:  http://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/

Common student errors provide the best source of distractors.

Alternatives should be stated clearly and concisely. Items that are excessively wordy assess students’ reading ability rather than their attainment of the learning objective

Alternatives should be mutually exclusive. Alternatives with overlapping content may be considered “trick” items by test-takers, excessive use of which can erode trust and respect for the testing process.

[Ed. Note:  I have some issues with this particular example but I get the point of their suggestion.]

Alternatives should be homogenous in content. Alternatives that are heterogeneous in content can provide cues to student about the correct answer.

The alternatives should be presented in a logical order (e.g., alphabetical or numerical) to avoid a bias toward certain positions.

Avoid complex multiple choice items, in which some or all of the alternatives consist of different combinations of options. As with “all of the above” answers, a sophisticated test-taker can use partial knowledge to achieve a correct answer.

Other suggestions from this source:

Alternatives should be free from clues about which response is correct. Sophisticated test-takers are alert to inadvertent clues to the correct answer, such differences in grammar, length, formatting, and language choice in the alternatives. It’s therefore important that alternatives

• have grammar consistent with the stem.
• are parallel in form.
• are similar in length.
• use similar language (e.g., all unlike textbook language or all like textbook language).

The alternatives “all of the above” and “none of the above” should not be used. When “all of the above” is used as an answer, test-takers who can identify more than one alternative as correct can select the correct answer even if unsure about other alternative(s). When “none of the above” is used as an alternative, test-takers who can eliminate a single option can thereby eliminate a second option. In either case, students can use partial knowledge to arrive at a correct answer.

The number of alternatives can vary among items as long as all alternatives are plausible. Plausible alternatives serve as functional distractors, which are those chosen by students that have not achieved the objective but ignored by students that have achieved the objective. There is little difference in difficulty, discrimination, and test score reliability among items containing two, three, and four distractors.

Keep the specific content of items independent of one another. Savvy test-takers can use information in one question to answer another question, reducing the validity of the test.

 

There is more to think about for multiple choice questions, which we will examine in the next post.