Psych 591 Advanced Psychometrics Spring 2005
Prof. Roger E. Millsap
Office: Rm 286, Psychology
965-2584, millsap@asu.edu
Required Text: Embretson, S.E. & Reise, S.P. (2000). Item response theory for psychologists. Mahwah, New Jersey: Erlbaum.
Topics Covered:
1) Background theory: problems with traditional measurement theory, historical roots of IRT, concepts from probability theory and statistics, latent variable models, latent dimensionality, conditional independence, item response functions, linear factor models, nonlinear IRT models.
2) Models for dichotomous response scales: the Rasch model and its properties, two and three-parameter logistic models, parameter interpretation, the normal ogive model, relations with factor analytic models.
3) Item parameter estimation in dichotomous models: likelihood functions, maximum likelihood, conditional maximum likelihood and the Rasch model, marginal maximum likelihood for 2PL and 3PL, properties of estimates, parameter identification issues.
4) Fit evaluation for dichotomous models: concepts of model fit, Pearson chi-square fit statistics, residuals, testing dimensionality assumptions, likelihood-ratio fit procedures, graphical plots, fit procedures for the Rasch model.
5) Item and test information functions: traditional standard error of measurement vs. item/test information, information functions for logistic test models, test information functions, using information functions.
6) Latent variable score estimation: concepts of score estimates, score estimates in the Rasch model, posterior distributions in marginal maximum likelihood, estimates under marginal maximum likelihood, properties of score estimates, indeterminacy, uses for score estimates.
7) Models for polytomous response scales: Rasch-family models: partial credit and rating scale models, the graded-response model, the nominal response model, expected response functions, interpretation of parameters, relations to linear factor models.
8) Item parameter estimation in polytomous response models: conditional maximum likelihood for Rasch-family models, marginal maximum likelihood estimation, properties of estimates.
9) Fit evaluation for polytomous response models: fit evaluation for Rasch-family models, Pearson fit statistics, residuals, likelihood-ratio approaches, testing dimensionality assumptions, graphical plots, problems in model fit.
10) Applications of IRT: Test development: traditional item analysis, item analysis using IRT, dimensionality assessment, characteristics of good items, limitations of the IRT approach.
11) Applications of IRT: Item and test bias: measurement bias, IRT-based approaches to bias assessment, bias indices, practical problems in bias assessment, additional approaches to bias assessment.
12) Final topics: multidimensional IRT, use of IRT in adaptive testing, test equating issues, use of IRT in assessment of change, future directions in IRT.
Overview
This course will cover a family of measurement models collectively denoted Aitem response theory@, or Alatent trait models.@ These new models are stronger in their assumptions than are models in classical test theory, the theory that forms a basis for traditional measurement practice. We will study aspects of the theory underlying these new models, and we will study applications of this theory as described above. I assume that you have already completed a course in psychometrics, either my course (Psych 534) or one that is equivalent to it. We will be using the computer for analyses of real data using item response theory. We will use the program BILOG for analyses involving dichotomous items, but other programs will be used as well, such as the Mplus program for confirmatory factor analysis and SPSS for some general statistical work. There will be a midterm exam that will be a Atake-home@ exam, and a final project at the end of the semester. The project will be chosen by the student (within guidelines to be discussed in class), and will include the application of the models discussed in class to data. We may also have occasional homework assignments. There will be no final exam.
References
Ackerman, T.A. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67-91.
Agresti, A. (1990). Categorical data analysis. New York: Wiley.
Baker, F.B. (1992). Item response theory: Parameter estimation techniques. New York: Marcel Dekker.
Bartholomew, D.J. (1987). Latent Variable Models and Factor Analysis. London: Charles Griffin & Company.
Fischer, G.H. & Molenaar, I.W. (Eds.) (1995). Rasch Models: Foundations, Recent Developments, and Applications. New York: Springer-Verlag.
Gustafsson, J.E. (1980). Testing and obtaining fit of data to the Rasch model. British Journal of Mathematical and Statistical Psychology, 33, 205-233.
Hambleton, R.K. & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nijhoff.
Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139-164.
Heinen, T. (1996). Latent Class and Disrete Latent Trait Models: Similarities and Differences. Thousand Oaks, CA: Sage.
Holland, P.W. & Thayer, D.T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. Braun (Eds.) Test Validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum Associates.
Lord, F.M. (1980). Applications of Item Response Theory to Practical Testing Problems. Hillsdale, NJ: Lawrence Erlbaum Associates.
Lord, F.M. & Novick, M.E. (1968). Statistical Theories of Mental Test Scores. Reading: Addison-Wesley.
Mellenbergh, G.J. (1989). Item bias and item response theory. International Journal of Educational Research, 13, 127-143.
Rasch, G. (1960). Probabilistic Models for some Intelligence and Attainment Tests. Copenhagen, Denmark: Danish Institute for Educational Research.
Reckase, M.D. (1997). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 25-36.
Reise, S.P., Widaman, K.F., & Pugh, R.H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566.
Sijtsma, K. & Molenaar, I.W. (2002). Introduction to Nonparametric Item Response Theory. Thousand Oaks, CA: Sage Publications.
Thissen, D. & Steinberg, L. (1984). Taxonomy of item response models. Psychometrika, 51, 567-578.
van der Linden, W.J. & Hambleton, R.K. (1997). Handbook of modern item response theory. New York: Springer-Verlag.
Wright, B.D. & Masters, G.N. (1982). Rating scale analysis. Chicago: MESA Press.