ISTEP+ Questions Must be Examined in Fairness to Students
Subscriber Benefit
As a subscriber you can listen to articles at work, in the car, or while you work out. Subscribe NowAs a parent, teacher and Vice-Chair of the Indiana State Board of Education, I want to make sure ISTEP+ scores are fair and accurate. It is unfortunate that the Board was forced to delay setting the pass-fail score for the test during its most recent meeting last week. I share the frustration of teachers and parents who deserve to see the test results.
From the beginning of this process, our board has insisted that we will accept nothing but the highest levels of accuracy and validity to ensure a reliable testing experience.
The most recent problem may seem very technical, but in reality it is not. When students began taking the ISTEP+ test last winter, I was joined by teachers across the state in questioning whether the online test was more difficult than the paper/pencil test. For example, on the paper/pencil test a student may be given a multiple choice math problem that requires the student to select the correct answer from four or five possible answers. Many students will know the correct answer and select it. Others may eliminate obvious incorrect choices to narrow down the potential answers. Still others may do the math for all the choices until they arrive at the correct choice, and some will simply guess; giving them a 20-25% chance of selecting the correct answer.
In the online version, the same question may be asked but the student is not provided any potential answers. They are asked to type the answer into the computer from scratch. They do not have the ability to eliminate incorrect answers, be prompted how the problem should be figured out by the looking at the potential answers, or guess.
Students are given the same amount of time to solve the problems, regardless of which version of the test they took. This clearly provides an argument for potential differences in difficulty for the student taking the online test rather than the paper/pencil version.
It is standard operating procedure to conduct a study of the test after it was taken in order to make sure the different versions were equally weighted in terms of difficulty. Educators have to know whether the formats were equitable before they can set a single pass-fail score that covers both versions of the test. In June, the State Board asked the Department of Education and its test vendor about these concerns and reiterated the importance of this particular study. At the time, I was promised that study would be complete before we began the process of setting the pass-fail score. The Department of Education’s testing timeline indicated that this study would be finished in August. Unfortunately, that turned out not to be the case.
The night before the October 14 Board Meeting, we finally received a “draft” version of the vendor’s study, dated two-weeks prior, which raised serious concerns. In that study, the test vendor’s own expert wrote “there were many flagged items in math… Many paper/pencil items appeared to be easier than the corresponding online items.” Given these many red flags in a report that was provided months after it was promised and only hours before the final vote, multiple independent and nationally recognized testing experts urged the Board to look into this problem in greater detail and delay setting a single pass-fail score that covered both tests.
Everyone at the State Board of Education hopes these red flags prove nothing but cautionary, but until we can examine the data, it is our duty to wait. As a parent, if you discovered that your child’s school gave two versions of a test, one more difficult than the other, with no difference in the score required to pass, the obvious question would be why some children had an easier test to pass while others faced a more difficult test. It is our responsibility to ensure consistent and comparable measures for all students.
Despite cries from some blaming politics, this is about fairness for our children. It is unfortunate that the Board was put into this position. It could have been avoided had the study been completed by the test vendor and provided to the Board in August as promised, but that didn’t happen. This is not about politics; it is about what is fair for Indiana children and getting the job done right.
Sarah O’Brien is a fourth-grade teacher at River Birch Elementary School in Avon and Vice Chair of the Indiana State Board of Education.