Vanderbilt Attrition Study – What it Does and Doesn’t Tell Us

Earlier this summer, four Vanderbilt graduate students released a report, “Examining Student Mobility and Attrition in MNPS.” These students looked at a set of data provided by Metro Nashville Public Schools to find patterns in student mobility during the school year. The report was immediately hailed by charter school advocates as putting to rest questions about student attrition, particularly movement from charter schools to traditional district schools. It was reported as such by the local media. But is that the full picture? What did the lauded Vanderbilt attrition report really say? In this blog post, TREE takes a closer look.

Limited Scope

First, we note that the authors repeatedly stress that their research and conclusions were very limited in scope. They were confined to the data set that they received from MNPS. The authors looked only at student movement from one school to another during one academic year, 2012-13, and they limited their review to grades 3-8. They did not look into students leaving a school after the end of the academic year to enroll in another for the next grade level or into student movement during years other than 2012-13. They did not look at high schools or early grades. Further, they make clear that the data they received were very incomplete. Indeed, one of their primary suggestions to MNPS is that the district maintain better information on student mobility, so that more meaningful conclusions can be drawn and so that the district is better able to address the negative effects of student mobility. They offer helpful suggestions for improving the MNPS enrollment and transfer processes in order to facilitate communication between schools and the keeping of more useful information.

Highly Mobile Students Score Lower, and Mobility Harms Student Learning

Students who change schools during an academic year “scored an average of fifteen percent or more below the below the district average TCAP Reading, Math, and Science Exams. The percentage of transfer students that score in the Below Basic range is nearly double that of the MNPS average.” (Report, page 31). “Simply put, a student who transfers school during the year is likely to score nearly half an achievement lever lower on TCAP.” (Report, page 34).

The report note that the increase in school choice options increases student mobility across the district during the year. (Report, page 5). It also states that moving between schools has a negative effect in student learning. If increased mobility is an unintended byproduct of increased school choice, then the district must move to curb mobility and work harder to keep students in a school. (Report, page 53).

Burden of Student Mobility Falls Mostly on Traditional Zoned Schools

The most reported aspect of the study is that students are highly mobile across the district and exit all types of schools mid-year, including zoned, charter, and other choice schools. This does not mean, however, that the impact of students movement is the same across school type. The authors discuss the “challenge of helping transfer students adapt to a new school culture after the start of the year.” (Report, page 40). The beginning of a school year is a time when school expectations, procedures, and communities are built and set. When a student arrives mid-year, that student has missed out on this process and has a harder time adjusting to the new environment. This creates additional work for school faculty and can disrupt the classroom environment, and the authors suggest that MNPS provide more resources to schools to assist in this process.

The work of acclimating transfer students to a new classroom falls most heavily on traditional zoned schools. When a student who lives within the attendance zone shows up at a zoned school, the school must admit them. Such students may arrive with no notice to school personnel, and sometimes with no parent accompanying them. In contrast, charter and other choice schools have the option of deciding whether to admit a new student during the year. If school officials deem it disruptive to the academic year to bring in a transferring student, they can decline to do so. Further, if they decide to accept a student transferring in, then they can set a date at which the student will enroll and can prepare for the new student.

While it may be true that students are leaving schools regardless of school type during the year, the work of serving and acclimating these vulnerable students mid-year falls heavily on zoned schools. Further, since mobile students tend to be lower scoring than their peers, as noted above, and most transferring students are transferring INTO zoned schools, it is reasonable to ask whether student mobility over time has a negative effect on zoned school performance on state assessments.

Same Students?

The report did not draw any conclusions about the million dollar question surrounding school choice in Nashville. Do charter schools really enroll the “same students” as their neighboring zoned schools? There are, however, a few reasons to believe the answer is no. This question takes on increasing importance as test scores are used in a simplistic fashion to label which schools are “better” and merit replication and resources, and which schools are “worse” and therefore, some argue, should be wound down and closed. If there are key differences in the student bodies of the schools being compared, then the logic for this simplistic use of test scores falls apart.

For starters, there is the obvious fact that choice schools, whether charter or district, enroll only those students whose parents have proactively filled out a choice school application in advance. Indeed, some charter schools mandate parent attendance at orientations and conferences. Enrollment at these schools requires a certain degree of parental involvement and ability to navigate a choice system that not every student or parent has.

Further, as previously noted, transferring students, who tend to be lower-scoring on state assessments, move OUT of both zoned and choice schools during the school year, but primarily IN to zoned schools. So one type of school has these lower scoring students leaving and coming, and another type primarily has them leaving.

The report states that choice schools have a different point at which “the school can say enough is enough” when it comes to student behavior. (Report, page 39). Choice schools are able to remove students from the school for behavior at a lower threshold than traditional schools. When this occurs, and disruptive students are sent “back” to their zoned school, what impact does this have on the comparative makeup of the student bodies with regard to discipline issues and behavior? As the authors note, “For these school leaders, the behavior of students who transfer from schools of choice impacts their ability to develop a positive school culture where all students can learn.” (Report, page 38). The authors also noted that in charter schools, students who enroll after 5th grade struggle to meet behavior and academic expectations, and often leave, creating a “revolving door” of students until the school fills the seat with a “student who adapts quickly.” (Report, page 40).

Interestingly, authors state that “[s]upporting choice should include educating school staff throughout the district about the key differences between traditional and choice schools, and explain the rationale for allowing schools of choice to operate differently than traditional schools.” (Report, page 54). No details are offered as to what differences are being referenced, but certainly the public, in addition to school staff, needs to hear about these differences, their justifications, and their impact.

A Starting Point

In short, the Vanderbilt attrition student is a multi-faceted, complex first look at students mobility in a high poverty school district. It offers useful suggestions for further data collection and study, and presents interesting ideas for MNPS to consider in sharing information among schools and supporting highly mobile students. It emphasizes the importance of helping traditional schools market their successes and offerings and compete for students in a choice environment, and acknowledges that often a negative perception of a traditional school is undeserved. It finds no evidence of nefarious intent or practice among schools leaders. It does not, however, neatly resolve all questions about the impact of school choice and the validity of comparing scores of different schools, nor does it claim to do so. If you’d like to read it in its entirety, you can find it here.