Item Response Theory in plain terms
· 2 min read
Item Response Theory, or IRT, is the measurement approach behind many modern assessments, including SubjectPath. The core idea is simple, even if the math isn’t.
The idea
Every question has a difficulty, and every student has an ability, and IRT places both on the same scale. The closer a student’s ability is to a question’s difficulty, the less certain it is whether they’ll answer correctly, and the more that answer tells you.
Where it comes from
- Separate people from items. Rasch set out measurement models that describe each person and each test item separately, so that comparisons between people can be independent of the particular items used.
- Practical testing. Lord’s work set out how item response theory can be applied to practical testing problems, including tailored tests that adapt to each person.
This research is about measurement, not about SubjectPath.
How SubjectPath applies this idea
SubjectPath estimates each student’s ability with Item Response Theory. It uses a Rasch-family model with a guessing floor for multiple-choice questions, which accounts for the chance of picking the right answer by luck. After each answer, the diagnostic updates its estimate and chooses one of the questions expected to tell it the most.
Why one scale helps
In Rasch’s models, ability and difficulty sit on the same scale, so comparisons between people need not depend on the particular questions each one answered. That is the idea behind an adaptive diagnostic: each student gets questions suited to them. It also lets the diagnostic choose questions near a student’s current level, where an answer says the most.
What IRT doesn’t do on its own
IRT estimates how far along a student is. It doesn’t, by itself, say which skill is missing or why. That’s why SubjectPath pairs it with a map of prerequisites and with answer choices tagged to misconceptions: the estimate says where to look, the map says which skill is missing, and the tags name the idea behind each wrong answer.
Sources
- Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research (reprinted 1993, MESA Press).
- Lord, F. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum.