A question often asked is 鈥渋s my school district good?鈥� Parents look for homes in 鈥済ood鈥� school districts, schoolteachers and administrators may want jobs in 鈥済ood鈥� school districts, and district employees may want to model their operations on the operations of other 鈥済ood鈥� school districts. Education researchers are also interested in identifying 鈥済ood鈥� school districts. There is a growing body of 鈥渄istrict effectiveness research,鈥� which studies the operations of school districts to identify practices that promote success in schools, such as providing sustained professional development. Frequently, however, these studies are critiqued for selecting districts unsystematically and/or for focusing on single year 鈥渟napshot鈥� research, rather than examining change over time.
The critiques of prior district effectiveness research led Alex Bowers of 麻豆原创 to ask 鈥渢o what extent can school districts be [systematically] identified from all the districts in a state that significantly outperform or underperform long-term performance trends across multiple indicators?鈥� (Bowers, 2015, p. 44). Using Ohio school districts as a sample, Bowers addressed this question in a 2010 study in which he investigated the use of hierarchical linear growth modeling for site selection, specifically, for identifying significantly unusual school districts. In a follow-up study from 2015, Bowers improved and expanded on the previous study, providing a set of recommendations for how to identify, over time, significantly outperforming school districts.
The 2015 Ohio district effectiveness study used data collected over eight academic years, 2005-2006 through 2012-2013, and included all 610 school districts in Ohio. The Ohio Department of Education collected and reported the outcome data, the Ohio Performance Index (PI) Score, in a uniform manner for all eight years, which provided a strong frame for the analysis. The Ohio PI score is a weighted average of student standardized test performance in grades 3, 8, and 10 across multiple subject tests, including mathematics, reading, writing, science and social studies. Figure 1 displays the trajectory of each Ohio school district鈥檚 PI score over twelve academic years, 2001-2002 to 2012-2013. Although the 2015 study focused on the academic years of 2005-2006 to 2012-2013, Figure 1 includes data from both Bowers鈥� 2010 and 2015 studies of Ohio school districts to allow for a more comprehensive presentation of the PI score trajectories.
Bowers used a two-level hierarchical linear growth model to identify school districts in Ohio that significantly outperformed or underperformed long-term performance trends. The model grouped districts with similar community demographics and followed their performance over time. Next, the model identified districts that significantly outperformed or underperformed the average for the districts in their group. Figure 2 presents the districts鈥� levels of performance. The districts above the top line in the plot were identified as outperforming, and districts below the bottom line in the plot were considered underperforming. The districts between the two lines were identified as at the norm, and provided a baseline for comparison. In his study, Bowers listed the names of the 15 school districts identified as outperforming and nominated these districts to be the subjects of future research. Speaking with the teachers, school administrators, and district leaders in these outperforming districts could help researchers understand how these districts consistently outperform their peers.
Interestingly, Figure 2 shows that some districts were identified as outperforming even though the Ohio Department of Education had assigned them a failing letter grade. State-assigned grades are important, but this study demonstrated the value of using a method other than state-assigned grades for guiding site selection for district effectiveness research. The hierarchical linear growth model is an improvement over previous research methods. The model controlled for district contexts, provided a baseline for comparison, and focused on change over time, presenting a more comprehensive depiction of district performance. Furthermore, Bowers鈥� model was the first to account for variation in school performance within districts. The model identified whether the district relied on a single high-performing school to raise its PI score or if a district improved evenly across all its schools. The model therefore highlighted districts that may have 鈥渋nstructional coherence鈥� through a district-level leadership effect across schools.
In the 2015 study, Bowers demonstrated the ability of hierarchical linear growth modeling to identify outperforming and underperforming school districts. His method used data the Ohio Department of Education had already collected, suggesting that it can be employed in other states. The method鈥檚 application, however, should be adapted to suit the state鈥檚 policies. For example, the 2015 study used data collected while the 鈥淣o Child Left Behind鈥� Act was in place, but the 鈥淓very Student Succeeds鈥� Act has recently been enacted and may necessitate adjustments to the application of Bowers鈥� method of site selection. That method, however, is robust and flexible and can help researchers select sites for district effectiveness studies in myriad contexts.
The is in Bowers, A.J. (2015). Site selection in school district research: A measure of effectiveness using hierarchical longitudinal growth models of performance. School Leadership & Management, 35(1), 39-61. doi: 10.1177/1942775116659462. Open Access Preprint:
The initial from 2010 is in Bowers, A.J. (2010). Toward addressing the issues of site selection in district effectiveness research: A 2-level hierarchical linear growth model. Educational Administration Quarterly, 46(3), 395-425. doi: 10.1177/0013161X10375271. Open Access Preprint: