Beddow, P. A. (2011). Effects of testing accommodations and item modifications on students’ performance: An experimental investigation of test accessibility strategies (Publication No. 3479839) [Doctoral dissertation, Vanderbilt University]. ProQuest Dissertations and Theses Global. https://www.proquest.com/docview/898546345

ProQuest document ID: 898546345 Vanderbilt University; also located on university webpage https://ir.vanderbilt.edu/handle/1803/10548 OR http://hdl.handle.net/1803/10548

The accommodations were those naturalistically provided by Arizona to students with disabilities; in order to keep this study condition constant, California students also were provided Arizona accommodations.

Four schools in Arizona and two schools in California (U.S.) participated. The 432 grade 7 students were randomly assigned to one of four testing conditions. Students with IEPs numbered 103, and were selected to participate because they were eligible for the Alternate Assessment based on Modified Achievement Standards (AA-MAS) in their respective states. Other demographic information were reported for the participants, including sex, ethnicity, disability category, and English language learner (ELL) status. Also, 17 grade 7 math teachers assisted by selecting test accommodations.

There were two forms of this researcher-developed 34-item test drawn from the mathematics item bank of the Discovery Education Assessment. One of the test forms was the standard test, either with or without accommodations. The other form consisted of the same 34 items, but which were modified to improve their accessibility. Both forms were provided with and without accommodations, for a total of four testing conditions. Student participants also completed a survey to report about their testing experiences.

The effects of accommodations were not detected, which the researcher attributed to the fact that most students in the two accommodated conditions did not request to use them; that is, the practice in Arizona is to deliver accommodations only upon student request. Therefore, only 15 students, or 7% of participants, used accommodations, and so the effect size was small and accommodations' effect was not significant. Significant differences were found between the scores for students taking the standard form compared with students' scores on the modified form of the test. The average score of students without IEPs taking the modified form was higher than the average score of students without IEPs who took the unmodified form. Students with IEPs who took the modified form also averaged a significantly higher score than students with IEPs who took the standard form. Also, students without IEPs scored significantly better than students with IEPs, both for the standard form and the modified form of the test. Item-level analysis resulted in a tendency for the majority of items (over 70%) on the modified test to be lower in difficulty level than on the standard test. In other words, a larger proportion of students (both with and without IEPs) responded correctly on the modified items. Further, item difficulty and item accessibility were not correlated. However, item discrimination (distinction among ability levels) was positively correlated (but not at a significant level) with item accessibility. The survey findings included that all students comprehended the modified items better than the standard items. Students without IEPs comprehended items (both standard and modified forms) better than the students with IEPs. All students reported that they had to work less to answer modified items than standard items. Similarly, students without IEPs indicated a lower degree of cognitive load than students with IEPs indicated, for both modified and standard items. Students also offered their perceptions of their having learned the content, and confidence in being correct on the items. Limitations of the study were reported, and future research directions were suggested.