We are at the point of our investigation where we need to start looking in more detail at test construction.  Here is a brief summary that puts together the pieces of what we have learned so far.

Our challenges

Write an accurate measure of student achievement that

• motivates students and reinforces learning,
• enables us to assess student mastery of course objectives,
• and allows us to recognize what material was or was not communicated clearly.

Some things we can do to accomplish this

In general, when designing a test we need to

• consider the length of the test,
• write clear, concise instructions,
• use a variety of test item formats,
• test early and/or frequently,
• proofread and check for accuracy,
• consider the needs of our disabled or non-native speaker students,
• and use humor.

More specifically, our test goals are to

• Assess achievement of instructional objectives,
• measure important aspects of the subject,
• accurately reflect the emphasis placed on important aspects of instruction,
• measure an appropriate level of student knowledge,
• and have the questions vary in levels of difficulty.

We should consider the technical quality of our tests

Quality means “conformance to requirements” and “fitness for use.”  The criteria are

• offering cognitive complexity,
• reviewing content quality,
• writing meaningful questions,
• using appropriate language,
• being able to generalize about student learning from their test performance,
• and writing a fair test with answers that represent what students know.

A useful tool is Bloom’s Taxonomy

It lists learning levels in increasing order of complexity:

1. Remembering
2. Understanding
3. Applying
4. Analyzing
5. Evaluating
6. Creating

To apply Bloom’s directly, we looked at

• Lists of verbs associated with each level (some were discipline-specific),
• question frames — nearly complete questions we can modify for our topics,
• and knowledge domains — the kinds of knowledge that can be tested:
  • factual,
  • conceptual,
  • procedural,
  • and metacognitive.

Next up:  Learning what question types to use to achieve our goals.