OK, so it’s not April or May anymore.  Yes, it’s a little late to be providing a review of the Spring conferences put on by the International Objective Measurement Workshop and the National Council on Measurement in Education. Better late than never.

IOMW is the biennial Rasch modeling conference.  I attend a few times in the 90s but have not been back in 10 years. I stopped going because the conference contained a great deal of Rasch evangelizing and seemed to lack relevance to practitioners in the licensure and certification arena.

I am pleased to report that the 2010 IOMW was a breath of fresh air.  Here is a quick review of the sessions that caught my attention.

1.) The Rasch testlet model – The term testlet is defined very differently here than it has traditionally been used in the Computerized Adaptive Testing arena.  Here it refers to a subtest which is modeled as a separate dimension in a multidimensional framework. For more information see Wang, W., & Wilson, M. (2005). The Rasch testlet model. Applied Psychological Measurement, 29. I don’t know enough about this model to comment, but I will be curious to see how this differs from the MIRT (multidimensional IRT) model that Mark Reckase has been promoting for years.  The skeptic in me is still doubtful of this model.  I have not seen a multi-dimensional model replicated successfully using REAL world data.  In essence, multi-dimensional modeling is like cold fusion for measurement.

2.) A subdimension model – Steffen Brandt proposed a new model where the multi-dimensional components are modeled as subdimensions rather than as separate dimensions.  This means that the primary ability estimate is no different than it would be in a uni-dimensional framework, but the standard error of measurement (SEM) is.  Again, this is still too new for me to make really good comment, but the theoretical premise of a subdimension model jives with me.

3.) Multiple Test Form Linking Quality – When we link two test forms using common item linking, the quality of the link can very greatly.  It will depend on the number of items in common, the distribution of the difficulty of the common items and the breadth of content that those items cover. Although the quality varies, never have I seen anyone try to quantify the quality. Mary Garner gave an excellent presentation where she used paired comparison matrices to analyze the connectivity of the forms and the impact that connectivity has on parameter estimation. Mountain Measurement will certainly be investigating this further and will definitely be searching for better ways to report linking quality on our technical reports.

4.) Person Fit indices – Sébastien Béland taught me about Snijders correction to Drasgow’s likehood Z (Lz) fit stat. This stat is a good compliment to the infit and outfit stats produced by Winsteps.  Thanks to Richard Smith and his cohort of folks at the DRC, for reminding me that every fit stat has it purpose and place.  No one fit stat can tell us everything about an item, and given the relative nature of fit statistics, absolute thresholds are not a reality. Moreover, the measurement community has only just begun to use the residual matrix effectively. Look for more in the future from Mountain Measurement about this.

5.) Lastly, a big thanks to David Irribarra for his presentation on modified Wright maps. The Rasch community is notorious for data visualizations that suck. David’s work is a stark contrast to this and has some colorful, beautiful visualizations that really convey information. This was a welcome sight and compliments Mountain Measurement’s mission with the TRANSOM project quite nicely. Maybe we can find a way to work these in.

All in all, there are some impressive new folks in the Rasch community these days doing fantastic work. And the old guard including David Andrich, George Engelhard, and Mark Wilson continue to impress me. It has been 10 years since Ben Wright’s stroke and it is apparent that the next generation has stepped up to the plate. This generation presents more information and less rhetoric and doesn’t have an axe to grind. I enjoyed myself. For those Raschees that fled, I encourage you come back. This group is totally different than it was 10 years ago.