Archive for the ‘Test Development’ Category

Why would a long test have a low reliability?

Introduction: Recently, we ran a reliability analysis for a client that is worth sharing. The certification exam had about 400 items and the reliability came in under .80 as measured by Cronbach’s Alpha (Cronbach, 1951), an undesirable if not unacceptable reliability for any test let alone a test containing 400 items. (NOTE: Mountain Measurement didn’t […]

Comments (2)

U.S. Supreme Court Rules in Favor of Valid Tests

There have not been too many court cases related to the testing industry that are commonly cited as landmark cases. It is for this reason that the recent decision by the U.S. Supreme Court in The Ricci, et. al. vs. DeStefano, et. al. case may be cited for years to come. In this case, the […]

Comments (1)

The Point-Biserial Correlation Coefficient

One common metric used to assess item quality is the point biserial correlation coefficient (rpb). The “pt bis” as it is sometimes called is the correlation between an item score (1/0) and the total score on a test. Positive values are desirable and indicate that the item is good at differentiating between high ability and […]

Comments (12)

What do you get from a single test item?

If an examinee answers a single test item (aka question) about a specific topic correctly, what can we infer about the examinee’s knowledge of that topic?  Even with the highest quality performance assessment items, the answer is very little. That’s because there are several factors that interact when a test item is presented to an […]

Comments (2)

A Criterion-Referenced Test is NOT a Mastery Test

This week’s entry is inspired by a Figure in a widely publicized book by Sharon A. Shrock and William Coscarelli called Criterion Referenced Test Development: Technical and Legal Guidelines for Corporate Training. I have not completed reading this book, but I find the authors perspective to be an interesting departure from the traditional testing paradigms […]

Comments (4)

© 2009-2017 Mountain Measurement, Inc. All Rights Reserved -- Copyright notice by Blog Copyright