June 6, 2012
State Investigations of DPS CSAP Results Lead to Finding of Significant Test Violations at One School
All 2010, 2011 CSAP Results at Beach Court Elementary School Invalidated
The Colorado Department of Education (CDE) has completed investigations of 2010 and 2011 Colorado Student Assessment Program (CSAP) results. Denver Public Schools requested CDE’s assistance after the school district conducted a thorough analysis of CSAP materials from all of its schools and determined that there were isolated statistical anomalies that required further scrutiny. Select investigations, which included school-wide reviews at Beach Court Elementary School and Hallett Fundamental Academy and reviews limited to a particular test administration at three other DPS schools, resulted in a finding of significant test violations at one school. That finding is at Beach Court Elementary School, and all of its 2010 and2011 CSAP scores have been invalidated as a result of the inquiry. The investigations at Hallett and the three limited-incident reviews did not result in a finding of misadministration or any conclusion of wrongdoing. (Summary reports of all of the investigations are below.)
The Colorado Attorney General’s Office, in its role as CDE’s legal advisor, conducted the investigation, with the assistance of Alvarez & Marsal—a national firm that is experienced in assessment procedures and integrity.
“Beginning with the very thorough and detailed analysis of CSAP results that Denver Public Schools conducted and shared with CDE, this has been a very collaborative and careful process between the state and DPS,” said Jo O’Brien, Assistant Commissioner of Assessment, Research and Evaluation at CDE. “We appreciate the diligence and cooperation on the part of DPS officials, and we have worked hard to uphold our duty to ensure that state assessments are administered properly and that the student-achievements results that are reported to schools and families are accurate. The evidence is very clear that proper procedures were not followed at Beach Court Elementary and the serious, intentional, breaches warrant the suppression of these scores. As a result, we have invalidated all the assessments from Beach Court from 2011. We did not conclude that material wrongdoing had occurred in any other DPS school and are not invalidating any other results.”
Earlier this spring, DPS consulted with the Colorado Department of Education on an analysis of all DPS test scores and materials from the 2011 CSAP. As a result of that analysis—which included a detailed review of statistical measures in a number of different areas, including the number of times that wrong answers were erased and changed to correct answers—DPS asked the state for an independent review of test-score administering and handling of the 2011 CSAP in a very limited number of instances.
Overall, the percentage of tests with statistical concerns in DPS was almost exactly the same as the small percentage of tests that triggered similar statistical concerns in a blind statewide wrong-to-right erasure analysis. There were two schools in DPS with statistical anomalies in more than one grade or subject and three additional schools with isolated statistical anomalies limited to just one grade level or test administration.
DPS and CDE will analyze spring 2012 data when it becomes available this summer. Lessons learned from the DPS investigation will be taken into consideration in the review of the 2012 data and used in determining test policy statewide next year.