Is it the school, or the students?

Posted by

Are schools that offer strong test scores highly effective, or do they mostly enroll students who are already well prepared for success? A study co-authored by MIT scholars concludes that widely circulated school quality ratings reflect the preparation and family backgrounds of their students as much or more than a school’s contribution to learning gains.

Indeed, studies show that many schools that receive relatively low ratings outperform these ratings. Research shows that traditional ratings are highly correlated with race. Notably, many published school ratings are highly positively correlated with the share of the white student body.

“A school’s average results, to some extent, reflect the demographic mix of the population it serves,” says Josh Angrist, a Nobel Prize-winning MIT economist who has long analyzed education outcomes. Angrist is a co-author of a newly published paper detailing the results of the study.

The study, which examines school districts in Denver and New York City, has the potential to significantly improve the way school quality is measured. Instead of raw composite measures such as test scores, the study uses changes in test scores and a statistical adjustment for racial composition to calculate more accurate measures of the causal effects that attending a particular school has on students’ learning gains. Can go. This methodologically sophisticated research is based on the fact that both Denver and New York City recruit students into schools in a way that allows researchers to mimic the conditions of a randomized trial.

Documenting a strong relationship between the rating systems currently used and race, the study shows that white and Asian students tend to attend higher-rated schools, while black and Hispanic students tend to attend lower-rated schools. .

Angrist says, “Simple measures of school quality, which are based on average school statistics, are always highly correlated with race, and these measures are a misleading guide to what you will get by sending your child to that school.” What can we expect?” ,

paper, “Mismeasurement of race and school qualityappears in the latest issue of American Economic Review: Insights, The authors are Angrist, Ford Professor of Economics at MIT; Peter Hull PhD ’17, professor of economics at Brown University; Parag Pathak, professor of economics at MIT in 1922; and Christopher Walters PhD ’13, associate professor of economics at the University of California at Berkeley. Angrist and Pathak are both professors in MIT’s economics department and co-founders of MIT’s Blueprint Labs, a research group that frequently examines school performance.

The study uses data provided by the Denver and New York City public school districts, where sixth-grade students apply for seats in certain middle schools, and the districts use a school-assignment system. In these districts, students can choose any school in the district, but some schools are oversubscribed. In these circumstances, the district uses a random lottery number to determine who gets a seat where.

Based on a lottery inside the seat-assignment algorithm, otherwise identical groups of students randomly attend a series of different schools. This facilitates comparisons that reveal causal effects of school attendance on learning gains, as in randomized clinical trials of the type used in medical research. Using math and English test scores, researchers evaluated student progress in Denver from the 2012-2013 to 2018-2019 school years and in New York City from the 2016-2017 to 2018-2019 school years.

It so happens that those school-assignment systems are mechanisms that some researchers have helped create, allowing them to better understand and measure the effects of school assignments.

“An unexpected benefit of our work designing Denver and New York City’s centralized selection systems is that we see how students are ration [distributed among] School,” says Pathak. “This allows for a research design that can isolate cause and effect.”

Ultimately, the study shows that much of the school-to-school variation in raw composite test scores stems from the types of students in any given school. This is a case of what researchers call “selection bias.” In this case, selection bias arises from the fact that more-advantaged families prefer the same set of schools.

“The basic problem here is selection bias,” says Angrist. “In the case of schools, selection bias is very consequential and is a huge part of American life. “A lot of decision-makers, whether they’re families or policymakers, are being misled by a kind of naive interpretation of the data.”

In fact, Reader says, the predominance of more simple school ratings today (found on many popular websites) not only creates a misleading picture of how much value schools add to students, but has a self-reinforcing effect—because Well-furnished and better-off, some families bid up the cost of housing near highly rated schools. As the scholars write in the paper, “Biased rating schemes direct families toward low-minority schools rather than high-quality schools, while penalizing schools that improve achievement for disadvantaged groups.”

The research team hopes their study will help districts examine and improve the way they measure and report school quality. To that end, Blueprint Labs is working with the New York City Department of Education to pilot a new rating system later this year. They also plan additional work to examine how families respond to different types of information about school quality.

Given that the researchers are proposing to improve the ratings by taking into account students’ preparation and improvement in what they consider a straightforward way, they think more officials and districts will be interested in updating their measurement practices. Can.

Pathak says, “We hope that the simple regression adjustments we propose will make it relatively easy for school districts to use our measure in practice.”

The research received support from the Walton Foundation and the National Science Foundation.

#school #students

Leave a Reply

Your email address will not be published. Required fields are marked *