Writing Fair (Equitable) Assessments

A well-written test, quiz, or examination will complement the many other forms of classroom assessment we use. A poorly written test, quiz, or examination is invalid; it does little to provide information about student learning.

In the jargon of assessment development, test questions are not always questions, so they are referred to as “items”. Items fall under two categories: Selected Response (Multiple Choice) items and Constructed Response (open response) items. Both categories can be used for assessments administered in written, oral, visual, or other formats.

1) What You Need To Know About Selected Response (Multiple Choice) Assessment Items

A selected response question is comprised of:

  • the stem
  • the options or choices (sometimes referred to as distractors)

A fair (equitable) Selected Response Question is written according to the following guidelines for validity:

  • The focus of the item reflects information that has been taught.
  • The answer must reflect the student’s ability to gather information from the text, not the student’s background knowledge. If a student can answer a question without reading the text, the question is invalid.
  • The students understand they are looking for the “best” answer among the options provided (between 3 and 5 options are usually provided)
  • The stem is written as a complete statement or question, not an unfinished sentence.
  • The stem is written in language familiar to the students.
  • The context (but not the content) of the question reflects student experience.
  • The stem is concise; extraneous words are removed.
  • The stem must not include negatives; for example, “Which of these is not an example of . . .”. NOTE: If a negative in a stem is unavoidable, underline or bold the word and use positive options to avoid a double negative structure.
  • The sentence structure of the options should be parallel.
  • The options do not include “all of the above”, “none of the above” or “a and c, but not b and d”.
  • All options must be plausible – do not include “joke” options.
  • The correct answer must not be present in another item.
  • To ensure students are not searching for patterns in correct letter response (“option C has been correct 3 times already, so it must be something else”), it can be helpful to organize the options in alphabetical order.

Selected Response Question PROs:

  • Selected Response Questions provide students with an opportunity to demonstrate their understanding without writing a response.
  • Selected Response Questions provide students with an opportunity to consider and compare a variety of options before responding.
  • Selected Response Questions, when written according to guidelines, are faster to score.

Selected Response Question CONs:

  • Selected Response Questions increase the reading load of the assessment.
  • The number of selected response options provided may cause confusion.

2) What You Need To Know About Constructed Response (Open Response) Assessment Items

A fair (equitable) Constructed Response Question is comprised of:

  • the stem
  • an answer space. Constructed response answer spaces include one, several or many lines, an empty text box, a partially completed table, a partially completed graph, an empty page, etc.

  A fair (equitable) Constructed Response Question is written according to the following guidelines:

  • The focus of the item reflects information that has been taught.
  • The answer must reflect the student’s ability to gather information from the text, not from the student’s background knowledge. If a student can answer a question without reading the text, the question is invalid.
  • The stem is a complete question or statement.
  • The stem is written in language familiar to the students.
  • The context (but not the content) of the question reflects student experience.
  • The stem is concise; extraneous words are removed.
  • The task is described with clear verbs, such as “list”, explain, draw a diagram of, or complete the chart, etc.
  • The amount of detail (“give 3 examples”) expected.
  • The answer is not embedded in another question.
  • The space provided for the answer reflects the length of response expected.

 Constructed Response Question PROs:

  • Constructed Response Questions provide students with an opportunity to demonstrate their understanding in their own words.
  • Constructed Response Questions may be easier for the teacher to write.

Constructed Response Question CONs:

  • Constructed Response Questions increase the student writing load.
  • Students who have difficulty transferring their thinking to writing may be disadvantaged.
  • Constructed response items take longer to score.

3) Remember: Share the secrets about assessment items with your students.

  • Provide students with information about the purpose of the assessment and how the results will be used.
  • Teach students how to approach and respond to the Selected Response Questions and Constructed Responses.
  • Teach students the “text features” of an assessment – the icon indicating the value of each question, how and where to record an answer, the timings (when applicable), the icons for “start” and “stop” etc.
  • Teach students to learn how to highlight the key points of the question, how to think about what the question is asking, how to formulate a response, and how to best convey that response to others.
  • Ensure students are aware of time limits (when applicable).
  • Teach students how to check their responses.

Sources Used:

Trends in Internationals Mathematics and Science Study: 2011 Item Writing Guidelinestimssandpirls.bc.edu/methods/pdf/T11_Item_writing_guidelines.pdf

Nova Scotia Dept. of Education: Evaluation Services. Guidelines for Item Writers

About Beyond The Apple

Beyond the Apple provides everything a Professional Learning Community needs! Designed to follow Beyond the Apple's Tenets of Adult Education, our videos re-ignite the excitement of professional conversations among educators in the classroom, university, colleges and professional training. Our free teaching and learning resources provide a follow up with more information that is current, research based and practical.
This entry was posted in Assessment and tagged , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s