General Tips About Test Design

Test. Keyboard

In the previous post (below) we asked, “Do you think about what you are testing and how you are assessing that information?” when it comes to test design.

This site,

Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf,

provides some general tips:

General tips about testing

  • Length of test

The more items it has, the more reliable it is.  However, if a test is too long, the students may get tired and not respond accurately.  If a test needs to be lengthy, divide it into sections with different kinds of tasks.

  • Clear, concise instructions

It is useful to provide an example of a worked problem, which helps the student understand exactly what is necessary.

  • Mix it up!

It is often advantageous to mix types of items (multiple choice, true-false, essay) on a written exam.  Weaknesses connected with one kind of item or component or in students’ test taking skills will be minimized.

  • Test Early

Consider discounting the first test if the results are poor.  Students often need a practice test to understand the format each instructor uses and anticipate the best way to prepare and take particular tests.

  • Test frequently

Frequent testing helps students to avoid getting behind, provides instructors with multiple sources of information to use in computing the final course grade, and gives students regular feedback.

  • Check for accuracy

Instructors should be cautious about using tests written by others.  They should be checked for accuracy and appropriateness in the given course.

  • Proofread exams

Check them carefully for misspellings, misnumbering responses, and page collation.

  • One wrong answer

It is wise to avoid having separate items or tasks depend upon answers or skills required in previous items or tasks.

  • Special considerations

Anticipate special considerations that learning disabled students or non-native speakers may need.

  • A little humor

Using a little humor or placing less difficult items or tasks at the beginning of an exam can help reduce test anxiety and thus promote a more accurate demonstration of their progress.

My reaction to their advice is mixed.  I’m not sure I could provide good examples of worked problems on the test itself because I teach mathematics — working problems for the students defeats the purpose of the test.  However I can have the students get that knowledge before the exam by having them complete homework problems and emphasize that many of the test problems will utilize those skills and strategies.

I am able to “mix it up” sometimes, depending on the course and the material being covered.  When testing vocabulary in statistics, for example, sometimes I use multiple choice and sometimes I use fill-in-the-blank.

I am not fond of the idea of discounting the first test if it is poor.  I get around that by offering my students short quizzes on a regular basis — I write the problems and grade them so students get a feel for my writing style and notation expectations before the longer, high-stakes exams.  My goal for the quizzes is to have the cumulative points be similar to an exam but then that total is weighted less than an exam towards the overall grade.

In math it is difficult to completely avoid having separate items or tasks depend upon previous answers.  The dilemma is this:  Do I write a complicated problem and have the students recall all the steps I want?  Or do I walk them through the steps knowing the answer to one may be dependent on the answer of another?

I think humor is a wonderful addition to tests.  Whenever I can (i.e., there is room), I include a math-related cartoon on the last page.

Also, the phrase I have heard about placing less difficult items at the beginning of test is “establishing a pattern of success.”  Give the student who has prepared a chance to start off with a victory thus building confidence for the rest of the questions.

What do you think of the list?  Would you add to it?  Is there anything with which you disagree?

The Challenges to Writing a Good Test

Paying attention to the details

What sorts of challenges do we, as professors, face when writing an exam?  That was one question in my mind when I started reading the resources.  This site made a statement that really struck a chord with me:

Source: https://www.psychologytoday.com/blog/thinking-about-kids/201212/how-write-final-exam

In reality, most professors develop exams as best they can.

Few have any formal training in assessment (the field that focuses on how to accurately measure performance).

Although many professors spend most of their time teaching, most of us have no formal training in education whatsoever.

So we tend to write questions that sound good and make sense to us.

We try to minimize cheating by writing new exams every semester so we never have a chance to weed out bad questions and develop really good measurement instruments.

We often use the same types of tools used by our own professors to assess the skills and learning of our students instead of thinking about what would work best.

We often don’t think clearly enough about our course goals to accurately measure them.

And sometimes our questions are not clear enough so different students interpret them differently and we only recognize interpretations that match our own.

And all this happens despite our best efforts and all our hard work.

For better or for worse.

Some of us have an education background but many of us do not.  We mimic the test styles we liked in our experience and perhaps avoid the ones we disliked.  Certainly we formed opinions about our teachers and decided which we wanted to emulate when we taught.

I don’t see anything wrong with that but if we want to improve, we need to explore new ideas.

We can start by acknowledging the basic features of a good exam.

A test should be an accurate measure of student achievement.

Source: http://www.iub.edu/~best/pdf_docs/better_tests.pdf :

Problems that keep tests from being accurate measures of students’ achievement:

  • Too many questions measuring only knowledge of facts.
  • Too little feedback.
  • The questions are often ambiguous and unclear.
  • The tests are too short to provide an adequate sample of the body of content to be covered.
  • The number of exams is insufficient to provide a good sample of the students’ attainment of the knowledge and skills the course is trying to develop.

Source: http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf :

Well-constructed tests:

  1. Motivate students and reinforce learning
  2. Enable teachers to assess the students’ mastery of course objectives
  3. Provide feedback on teaching, often showing what was or was not communicated clearly

What makes a test good or bad?  The most basic and obvious answer is that good tests measure what you want to measure and bad tests do not.

The whole point of testing is to encourage learning.  A good test is designed with items that are not easily guessed without proper studying.

Have you ever spent time studying your tests?  When designing them, do you think about what you are testing and how you are assessing that information?

Have you analyzed the responses students put on the test to see if they understood what you were asking?  Do you think about how the wording or design could be improved on future tests?

We will explore these in more detail in the next posts.

List of Resources

Here are the various web links that we will reference for this blog.  The list may be edited over time.

Feel free to post more resources in the Comments section.

  1. http://www.iub.edu/~best/pdf_docs/better_tests.pdf
  2. http://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/
  3. http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf
  4. https://www.msu.edu/dept/soweb/writitem.html
  5. http://www.helpteaching.com/about/how_to_write_good_test_questions/
  6. http://www.psychologytoday.com/blog/thinking-about-kids/201212/how-write-final-exam
  7. http://www.instructables.com/id/How-to-write-a-test/
  8. http://www.uleth.ca/edu/runte/tests/
  9. http://teachonline.asu.edu/2013/06/quick-reference-guide-for-writing-effective-test-questions/
  10. http://www.lshtm.ac.uk/edu/taughtcourses/writinggoodexamquestions.pdf
  11. http://www.crlt.umich.edu/P8_0
  12. http://www.cmu.edu/teaching/assessment/assesslearning/creatingexams.html
  13. http://www.teaching-learning.utas.edu.au/assessment/authentic-assessment/designing-exams
  14. http://depts.washington.edu/eproject/ExamChecklist.htm
  15. http://www.princeton.edu/mcgraw/library/sat-tipsheets/designing-exam/
  16. http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/TestingTips.htm
  17. http://www.tltc.ttu.edu/teach/TLTC%20Teaching%20Resources/CreateTests.asp
  18. http://www.edutopia.org/better-tests-differentiate
  19. http://teaching.uncc.edu/learning-resources/articles-books/best-practice/assessment-grading/designing-test-questions
  20. http://www.calm.hw.ac.uk/GeneralAuthoring/031112-goodpracticeguide-hw.pdf

What We are About

Welcome!

We are teachers and we want to do our best for our students.  Sometimes we need a chance to see what others are doing to help us “improve our game.”

The goal of this blog is to explore the strategies, philosophies, and various options of test writing.  We’ll take a systematic approach, starting with general tips about tests and test construction and then proceeding through different test item types.

We will look at articles and advice on the Internet and discuss how the ideas may or may not apply to our discipline.  This is not a “one-size-fits-all” topic!  Neither should it be considered a best practices list.  We are the topic experts and the best judges for the information we are assessing.

Feel free to share these posts.

Due to the number of spammers, comments have been disallowed.

Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license

Tracy Johnston
STEM 1 Curriculum and Program Improvement (CPI) Coordinator
Palomar College

This is a sticky post; newer posts appear below.

For a list of the resources on this blog, click here.