School Reform Efforts Show Mixed Results in Study

Chicago elementary schools that underwent various reform efforts since 1997 improved during the first four years of intervention, but still lagged behind the district on state exams, the University of Chicago Consortium on School Research said in a report released Wednesday.

The long-awaited report also found that high schools that implemented dramatic reforms showed little to no improvement in attendance or rates of students on pace to graduate by the end of freshman year.

The process of turning around a poorly performing school can entail changing curriculum and school leadership, firing existing staff, or in extreme cases, closing a school and starting over a year or two later. Former Chicago Public Schools chief executive Arne Duncan made these dramatic reforms a central part of his strategy for improving the school system from 2002 to 2009 and his three successors have continued the practice. Twenty-one schools have been targeted for some form of overhaul this year by Mayor Rahm Emanuel and current schools chief executive Jean-Claude Brizard.

The policy of overhauling under-performing schools is controversial and critics have questioned the educational benefits. The report released today is among the most extensive examinations of the overhaul process in Chicago.

Despite the limited progress shown in the report, CPS officials said the results reaffirm their plan to replace the staff and administrators at a record number of 10 schools next year.

“We have to stay the course,” said Noemi Donoso, the district’s chief education officer. “This study finds we have some absolutely promising practices.”

The Chicago Teachers Union, whose members lose their jobs in most turnarounds, released a statement challenging some of the report’s findings and said members of the Board of Education “continue to drain millions of public dollars from regular neighborhood schools” to “invest in charter, private-sector school models.”

Under President Obama and Secretary of Education Arne Duncan, the federal government has made turning around the lowest performing schools a cornerstone of its education agenda, doling out billions of dollars each year to districts and states to implement such measures.

But not all of the overhaul models have the same results.

Elaine Allensworth, one of the study’s lead researchers, said five research models were studied at 36 schools: reconstitution, transformation, district turnaround, turnaround by the private Academy for Urban School Leadership (AUSL), and closure and restart.

Turnaround and reconstitution are similar processes in which existing staff is fired and new leadership takes over the curriculum and learning climate; transformation entails replacing the school principal and making changes to the curriculum; restart is when a low-performing school closes and a charter school opens in it’s place a year or two later.

Consortium researchers compared schools where reform models were implemented to the schools most similar to them in both performance and student population the year before reform. Allensworth said researchers also compared schools where reforms took place with all schools on probation and to the district average and found the reformed schools were improving at faster rates.

The study did not reach any conclusions about which reform measures worked best, in part, Allensworth said, because “there aren’t enough schools in each of these reform models to make sweeping generalizations that one model worked better than another.”

Researchers found that during the first four years of reform, elementary schools on average made gains on the Illinois Standard Achievement Test in both reading and math at a faster pace than the district as a whole, but still lagged behind the district’s average composite score.

AUSL is a non-profit organization that operates 19 schools for CPS in addition to a teacher training academy. The group, whose founder, Martin Koldyke, is on the board of the Chicago News Cooperative, runs 12 turnaround schools and is set to take on six more next school year. Donald Feinstein, AUSL’s executive director, said the study’s elementary school results “clearly confirm” that his organization has a proven model for improving academic performance, but added that there is still work to be done.

“I want our schools to be at the city average and higher, but the fact is they’ve been so broken for so long,” Feinstein said. “Look where we are and look where we came from. The journey is still taking place.”

At the high schools studied, absence rates declined in the first year of intervention but returned to pre-reform rates in subsequent years. Average rates of freshman on track to graduate remained the same as those for comparable schools and did not increase more quickly or slowly than the district average.

“There’s no question that high schools are the hardest places to crack, and this evidence shows that,” said Tim Knowles, director of the Urban Education Institute at the University of Chicago. “It’s not just a Chicago problem, this is a state and national problem.”

Seven of the nine high schools studied – Englewood, Harper, Orr, Phillips, DuSable and King – experienced two or more types of overhaul since 1997. The two that did not—Fenger and Marshall—began their turnarounds in 2009 and 2010, respectively.

More recent turnaround efforts at Orr, Harper, Fenger and Marshall appear to show better results than the schools that were overhauled in 1997, but Allensworth there is not enough data to draw any conclusions.

“The chronic reform and re-reform and reform of reform that goes on in some schools is probably part of the problem,” Knowles said.

Donoso acknowledged the danger of a chronic turnaround cycle but said CPS cannot allow schools to regularly under-perform.

“Obviously, we don’t want to end up in a perpetual state of turnaround,” Donoso said. “But if we don’t get it right, we have a responsibility to these students and these communities to do something better.”

Critics have long argued that turnaround schools and charter schools show improvements because they push out poor-performing students.

Consortium researchers found, however, that the schools that experienced reconstitution, turnaround or transformation served the same students or similar populations of students after the reforms were implemented.

But schools that closed and reopened as charter schools, which enroll students through a citywide lottery, served less than half of the students that were enrolled before the reform took place, and in some cases none of the students were the same.

Tim Cawley, the district’s chief operating officer and the former chief financial officer for AUSL, acknowledged that the restart model is “highly disruptive,” which is why district leaders are focusing on turnarounds next year.

But turnarounds come with a hefty price tag.

While all reform models have been implemented with additional money, turnaround efforts require the largest investment from the district. For example, AUSL elementary schools receive $1.5 million more than their neighborhood counterparts during the first five years of turnaround and an additional $420 per pupil each year. For high schools, AUSL receives $2.4 million more than their neighborhood counterparts over the course of five years and an additional $500 per pupil each year.

With the district’s deficit projected to be close to $1 billion by 2014, Knowles said, officials should do a “careful cost analysis.”

“There’s a great need to continue to study these reforms, especially if the funding sources change after four or five years, which seems very likely,” Allensworth added.