A well-written test, quiz, or examination will complement the many other forms of formative assessment we use, so it’s important to know the how to’s of valid and equitable test construction.

In the jargon of assessment development, test questions are referred to as “items.” Items fall under two categories: Selected Response (Multiple Choice) items and Constructed Response (Open Response) items. While both categories can be used for assessments administered in written, oral, visual, or other formats, it’s important that all items provide valid information. Here are the pros, the cons and the how to’s selected response of item writing.

Selected Response Items

PROs:

• Selected Response Questions provide students with an opportunity to demonstrate their understanding without writing a response.
• Selected Response Questions provide students with an opportunity to consider and compare a variety of options before responding.
• Selected Response Questions, when written according to guidelines, are faster to score.

CONs:

• Selected Response Questions increase the reading load of the assessment.
• The number of selected response options provided may cause confusion.

A valid and equitable selected response item is written according to the following guidelines:

• The focus of the item reflects information that has been taught.
• The answer must reflect the student’s ability to gather information from the text, not the student’s background knowledge. If a student can answer a question without reading the text, the question is invalid.
• The students understand they are looking for the “best” answer among the options provided (between 3 and 5 options are usually provided)
• The stem is written as a complete statement or question, not an unfinished sentence.
• The stem is written in language familiar to the students.
• The context (but not the content) of the question reflects student experience.
• The stem is concise; extraneous words are removed.
• The stem must not include negatives; for example, “Which of these is not an example of . . .”. NOTE: If a negative in a stem is unavoidable, underline or bold the word and use positive options to avoid a double negative structure.
• The sentence structure of the options should be parallel.
• The options do not include “all of the above”, “none of the above” or “a and c, but not b and d”.
• All options must be plausible – do not include “joke” options.
• The correct answer must not be present in another item.
• To ensure students are not searching for patterns in correct letter response (“option C has been correct 3 times already, so it must be something else”), it can be helpful to organize the options in alphabetical order.

 Constructed Response Items

PROs:

• Constructed Response Questions provide students with an opportunity to demonstrate their understanding in their own words.
• Constructed Response Questions may be easier for the teacher to write.

CONs:

• Constructed Response Questions increase the student writing load.
• Students who have difficulty transferring their thinking to writing may be disadvantaged.
• Constructed response items take longer to score.

A fair (equitable) Constructed Response Item is written according to the following guidelines:

• The focus of the item reflects information that has been taught.
• The answer must reflect the student’s ability to gather information from the text, not from the student’s background knowledge. If a student can answer a question without reading the text, the question is invalid.
• The stem is a complete question or statement.
• The stem is written in language familiar to the students.
• The context (but not the content) of the question reflects student experience.
• The stem is concise; extraneous words are removed.
• The task is described with clear verbs, such as “list”, explain, draw a diagram of, or complete the chart, etc.
• The amount of detail (“give 3 examples”) expected.
• The answer is not embedded in another question.
• The space provided for the answer reflects the length of response expected.

Remember: Share the secrets about assessment items with your students.

• Provide students with information about the purpose of the assessment and how the results will be used.
• Teach students how to approach and respond to the Selected Response Items and Constructed Response Items.
• Teach students the “text features” of an assessment – the icon indicating the value of each question, how and where to record an answer, the timings (when applicable), the icons for “start” and “stop” etc.
• Teach students to learn how to highlight the key points of the question, how to think about what the question is asking, how to formulate a response, and how to best convey that response to others.
• Ensure students are aware of time limits (when applicable).
• Teach students how to check their responses.

Sources Used:

Trends in Internationals Mathematics and Science Study: 2011 Item Writing Guidelinestimssandpirls.bc.edu/methods/pdf/T11_Item_writing_guidelines.pdf

Nova Scotia Dept. of Education: Evaluation Services. Guidelines for Item Writers

For more conversations about education, please visit:Beyond the Apple . . . Reframing Conversations in Education or contact us at Beyondtheapplecontact@gmail.com