Writing test items is a matter of precision, perhaps more akin to computer programming than to writing prose. A test item must focus the attention of the examinee on the principle or construct upon which the item is based. Ideally, students who answer a test item incorrectly will do so because their mastery of the principle or construct in focus was inadequate or incomplete. Any characteristics of a test item which distract the examinee from the major point or focus an item, reduce the effectiveness of that item. Any item answered correctly or incorrectly because of extraneous factors in the item, results in misleading feedback to both examinee and examiner.
Essay questions allow for perhaps the most unrestricted type of written assessment item that we will cover in Tests & Measurement for People Who Hate Tests & Measurement. What you want to know is how well the test taker can organize information and express his or her ideas in writing. That’s why the really, really big exams in one’s academic career are usually of the essay type—these types of items just tap more higher-level and complex skills.
THINGS TO REMEMBER: WHEN TO WRITE
Essay items are the item of choice if you want an unrestricted response and want to access higher-order thinking, such as the relationship between ideas and the pros and cons of a particular argument.
Essay questions come in two basic flavors:
open-ended (also called unrestricted or extended) questions and closed-ended (also called restricted) questions. An open-ended (or unrestricted response) essay question is one where there are no restrictions on the response, including the amount of time allowed to finish, the number of pages written, or material included. Now, it is a bit impractical to allow test takers to have 25 hours to answer one essay question or to write hundreds of pages about anything they like. So, of course there are practical limits. It’s just that the limits do not define the scope of the response.
closed-ended question where the response is restricted. The more restricted closed-ended question places definite limits on the content as well as the format.
HOW TO WRITE ESSAY ITEMS: THE GUIDELINES
Here are just a few guidelines that will be helpful when it comes time to write an essay question.
1. Allow adequate time to answer the question. By their very design, essay questions can take a considerable amount of time to answer.
2. Be sure the question is complete and clear.
3. Essay questions should be used only to evaluate higher-order outcomes, such as when comparisons, evaluations, analyses, and interpretations are required.
4. Have all test takers answer the same questions. This just reduces the burden placed on the developer of the test, but more important, makes the test easier to score—it’s just more practical.
Because essay questions should have one correct answer, and (as you will learn shortly) there should be a model answer against which test takers’ responses should be compared. Allowing for multiple questions requires the test scores to have multiple standards against which to compare them—too much work! Instead, rotate questions from year to year or test to test so there is a nice collection of questions and model answers.
Advantages of Essay Items
• They help find out how ideas are related to one another.
• They increase security
. • They provide increased flexibility in item design.
• They are relatively easy to construct.
Disadvantages of Essay Items
• They emphasize writing.
• They are difficult to write.
• They provide an inadequate sampling of subject matter.
• They are hard to score.
• They emphasize writing skills over content.
THINGS TO REMEMBER: WHY ESSAYS?
- Among the many reasons why essays are particularly useful is that they are very flexible in both form (the size and complexity) and purpose (relate simple ideas or elaborate a complex argument).
- The use of essay questions increases security because it is very difficult to plagiarize during an essay item examination. And, along the same lines, it is almost impossible for test takers to effectively guess the correct answer—so guessing is removed as a legitimate concern.
- The essay item format has unparalleled flexibility. Take the time necessary to create a really good question—one that is exciting to consider and one that gives the test taker a fair chance at success.
- Essay questions can be relatively easy to construct. If you know your material well (and we would sure assume such), you can put together four essay questions in less than an hour that can effectively tap higher-order learning.
Why Essay Items Are Not So Good
1. They emphasize writing. Well, no kidding—that’s what they are supposed to do. But what they don’t do well at all is tapping the test taker who is knowledgeable about ideas and their relationship to one another, but just cannot express it in words. For these kinds of test takers, evaluating them fairly and accurately is always a challenge.
2. They can be tough to write. But didn’t we just say above that they were easy to write? Well, we said they can be. They can be a bear as well! The test designer has to invest a great deal of time in creating each essay question and making sure that it taps the objectives that are to be tested, but does so in a way to encourage the expression of ideas that you can’t find in a true/false or matching test.
3. Precision in sampling counts. Because essay questions take a good deal of time to create and even more time to complete as part of a test, it’s tough for the test to adequately sample the entire universe of what the test taker might have learned. This should always be a consideration in the overall evaluation of knowledge and understanding of any topic.
4. Essay questions are not easy to score. Think about it—in a class of 25 students, each of whom completes five essay questions—that’s more than 120 questions that have to be graded. No teacher will begrudge grading them (well, maybe a few), but all will have some serious problems remaining neutral, staying on task, and being consistent.
5. Writing can become more important than content. Because essays are written, some students can bluff their way through the answer by virtue of their excellent writing ability. Just about the only way to counter this is through the use of model answers, as discussed in the next section.
HOW TO SCORE ESSAY ITEMS
Essay items are absolutely terrific and almost indispensable for sampling higher-order thinking. I’ve mentioned that several times throughout this chapter. But they are a bear to score—time consuming and very demanding of all the scorer’s attention. With that in mind, here are a few tips that might make the scoring process more efficient and result in a fairer assessment. 1. Scorers should provide plenty of time to score an essay item. Each of the items has to be read and then scored, and often the scorer will read the items more than once—the first time for a general overview of the content, and the second time for a more detailed analysis, including an assessment of content (again) and writing skills (such as grammar, transitions, and sentence usage).
source: SAGE Publication, 2015 (http://www.sagepub.com/sites/default/files/upm-binaries/6429_Chapter_7__Salkind_(Tests)_I_Proof_3.pdf)