EDP 691
Test and Scale Construction
Spring 2004: Dr. Gorin
Professor Joanna Gorin
|
||||||||||
Meeting Times
Tuesday and Thursday 10:40 – 11:55
COOR L1- 14
Course Objectives
Primary course objective:
By the end of the course students should have a general understanding of theories of test and scale construction, including underlying measurement models and associated applications. Students will complete a project, for which they will apply current best practices for scale development to generate a new scale for a specified purpose (designated by the students). In addition to the scale itself, students will be able to produce a document including a theoretical basis for the scale, a detailed description of appropriate scale development procedures, and results from data analysis. It is hoped that students will select topics relevant to their courses of study, potentially as a preliminary step to real scale development for use in their own research.
Other secondary objectives include the following:
Students will become familiar with data analysis utilizing SPSS.
Students will become familiar with the use of reference materials for currently developed tests and scales.
Students will be critical consumers of testing data and research on test and scale development.
Students will successfully work with others to complete a comprehensive project to produce a quality product (individual final documents and a group presentation).
Required Text:
Hambleton, R.K., Swaminathatn, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Sage Publications: Newbury Park.
DeVellis, R. F. (2003). Scale Development: Theory and Applications. Sage Publications: Thousand Oaks, CA.
Course Grade
Scale Development Project
Several weeks into the semester students will form groups based on common interest to develop a cognitive, attitudinal, or personality scale for the course. Completion of this project has been broken down into subcomponents, listed in the due column of the course calendar. Although the group will work together to design the project, write the items, and collect the data, all of the written materials will be submitted individually (for individual grades). A final group presentation of the project in the last two class sessions will be graded as well.
|
Date |
Topic |
Subtopic |
Due |
% of Grade |
|
20-Jan |
Introduction |
|
|
|
|
22-Jan |
Historical Development of Measurement |
|
|
|
|
27-Jan |
Basic Scaling Theory |
|
|
|
|
29-Jan |
Classical True Score Theory |
Parallel Forms |
|
|
|
3-Feb |
Classical True Score Theory |
Reliability |
|
|
|
5-Feb |
Group Project Meeting Day |
|
|
|
|
10-Feb |
Item Response Theory |
Rasch Model |
|
|
|
12-Feb |
Item Response Theory |
Other Models, Information Functions |
|
|
|
17-Feb |
Item Response Theory |
Polytomous Models, Applications of IRT |
Write-up literature review due. |
10% |
|
19-Feb |
Test Equating |
|
|
|
|
24-Feb |
Test Bias and Test Fairness |
|
|
|
|
26-Feb |
Attitude/Personality Scale Construction |
Likert and Thurstone Scaling |
|
|
|
2-Mar |
Test Construction |
|
|
|
|
4-Mar |
Sampling |
|
|
|
|
9-Mar |
Group Project Meeting Day |
|
Test Specifications and Sampling |
|
|
11-Mar |
Item Analysis |
|
|
|
|
16-Mar |
Spring Break |
|
|
|
|
18-Mar |
Spring Break |
|
|
|
|
23-Mar |
Item Analysis |
|
Write-up preliminary methodology. |
10% |
|
25-Mar |
Validity |
Messick |
|
|
|
30-Mar |
Validity |
Factor Analysis |
|
|
|
1-Apr |
Validity |
Factor Analysis |
Items completed and turned in. |
|
|
6-Apr |
Performance Assessment |
|
|
|
|
8-Apr |
Group Project Meeting Day |
|
Write-up proposed data analysis. |
10% |
|
13-Apr |
Independent Project Day |
|
|
|
|
15-Apr |
Independent Project Day |
|
|
|
|
20-Apr |
Data Analysis Day |
|
Data collection due. |
10% |
|
22-Apr |
Validity |
Facet Analysis |
|
|
|
27-Apr |
Validity |
Cluster Analysis and MDS |
|
|
|
29-Apr |
Presentation of Projects |
|
Final write-up for all projects. |
35% (Written) |
|
4-May |
Presentation of Projects |
|
|
25% (Oral) |