Scrutinizing our schools: DPS student achievement lags most NC districts
January 19, 2016
This is the second in a six-part series of articles scrutinizing Durham Public Schools performance and financial priorities. Today: evaluating DPS's academic performance relative to other NC school districts. Coming up tomorrow: comparing DPS spending against other districts.
First up: how does Durham’s academic performance compare against its peers?
The short answer isn’t comforting. If you’re white, the answer seems to be that you’ll do just fine -- if you’re not, you’re literally at the bottom of the pack.
Before we go there, let’s look first at the aggregate data.
With over 33,000 students, Durham is the eighth-largest school district in the state. For a starting point, it’s useful to look at the ways in which DPS’ performance compares among the ten largest N.C. school districts.
Naturally, such an analysis depends on standardized test results from NC DPI. There’s plenty of reason to be worried about the testing-heavy regimes in school districts throughout the country, including in Durham, and some will argue this is a poor benchmark for learning achievement.
We’ve got a hunch this is not the method that DPS would choose to measure its performance by. In fact, based on the district’s response to the last round of test results, we’re pretty sure it isn’t.
Looking at school performance through a slightly different lens, the state uses complex statistical algorithms developed by SAS to measure growth for individual students and schools. After N.C. DPI found 21 out of 53 DPS schools to be “low-performing” earlier this year, the district countered that 14 of these schools met the their yearly growth metrics, and that the “school performance grades do not accurately reflect the teaching and learning going on in each school.” (Of course, even among the schools that met or exceeded targeted growth from 2013-14 to 2014-15, half of them saw actual pass rates decline.)
There are good arguments on both sides of the question of school growth measures, and about standardized testing overall, but we think it’s worth stepping back and looking at the tests in a different light: what do they tell us about how Durham students perform relative to others in the state?
After all, the biases and failings in the test do not merely affect DPS students; they are also faced by students in every other North Carolina school. In this report, we want to see how DPS students perform relative to students in other districts.
It turns out that when you use comparative data, be it achievement of grade level proficiency (GLP) or college and career readiness (CCR) -- the two testing metrics used by the state -- DPS falls at the bottom of every ranking but one:
- Overall GLP and CCR rates: 10th out of 10
- Hispanic students: 10th out of 10 (GLP)
- Economically disadvantaged students: 10th out of 10 (GLP)
- Students with disabilities: 10th out of 10 (GLP)
- African-American students: 9th out of 10 (GLP)
- White students: 4th out of 10 (GLP)
Indeed, DPS is the only district out of N.C.’s ten largest systems where fewer than half of its students met GLP in 2014-15.
The counterargument, it may be said, is that the ten largest districts do not necessarily match DPS on demographics. Union is a fast-growing, suburbanized areas with higher-SES new residents; Wake has massive high-wealth areas where students perform well; etc.
Okay. Let’s go to a different metric -- how does DPS rank among all 115 public school districts in the state? Next verse, somehow worse than the first:
- Overall GLP rates: 93rd out of 115
- Overall CCR rates: 89th out of 115
- Hispanic students: 105th out of 115 (GLP and CCR)
- Economically disadvantaged students: 105th out of 115 (GLP), 103nd out of 115 (CCR)
- African-American students: 60th out of 115 (GLP), 58th out of 115 (CCR)
- White students: 8th out of 115 (GLP), 6th out of 115 (CCR)
The contrast is, if anything, more extraordinary against the larger comparison sets. White students’ achievement on standardized tests is nearly the best in the entire state -- yet Latino students fare worse in Durham than nearly any other district.
Selection biases may play a role. For instance, the Durham MSA’s high ranking on economic inequality likely means that there are a disproportionate number of highly-educated and -compensated white families in Durham; that those who send their kids to public schools are likely to see their children excel anywhere in the state; that the performance data would look different if so many parents of means did not send kids to charter or private schools.
Only problem is, none of that moves DPS or our county one iota closer to fulfilling the basic constitutional right of all persons under North Carolina’s system of law: equal access to a high-quality primary and secondary education.
Income inequality raises still another explanation or rationalization -- pick your lens -- on the data, of course.
Is it fair to compare DPS to only large districts, or to all districts, instead of drilling into those districts that are high-poverty?
(Spoiler alert: the data are dismal any way you slice them. But let’s proceed.)
DPS in 2014-15 saw nearly two-thirds of its students qualify for the federal free/reduced lunch program. Within the state, there were 48 out of 115 districts with the same or greater levels of poverty. Against just that comparison set, DPS still was below-average:
- Overall GLP and CCR rates: 31st and 28th out of 48 respectively
Most of the districts performing worse than DPS in this measure have a F&R lunch eligibility rate at least 10% higher than DPS.
The bottom line?
Only 44% of DPS students are grade level proficient (GLP) and 35% meet college and career ready (CCR) standards. The data is relatively consistent across the elementary, middle, and high school levels.
Furthermore, large achievement gaps exist, both by race/ethnicity and by class. While 70% of white students in DPS meet CCR standards, the same holds for only 25% of black students, 26% of hispanic students, and 22% of economically disadvantaged students.
(It’s important to note that 76% of DPS students are either black or hispanic, and nearly two-thirds of DPS students are economically disadvantaged.)
In the measures related to non-white students, DPS performs worse than all the other large districts in the state; very poorly relative to all 115 systems; and worse than most districts facing equal or higher levels of poverty.
There’s one area where we lead all other districts, though: we spend more in local supplemental dollars than any of the other large districts in the state.
And, compared to those large districts most alike to DPS, we spend as much as $120 million more -- only to achieve poorer results.
It may be, as current school board chair and County Commission candidate Heidi Carter said, that our district is trying to struggle upwards at least to the average level of support schools get nationally, since North Carolina is (both recently and for years under previous Democratic leadership) a laggard in primary/secondary education funding.
Yet if we’re spending more per pupil than other districts in the state -- even those districts that resemble Durham as large, urban programs -- why are we lagging so far behind?
Source data:
- Download NC LEA Academic Performance Drilldown -- This file, from NC DPI data for 2014-15, ranks DPS against other districts for CCR/GLP on a range of demographic bases.
- Download NC LEA EDS vs GLP - This second spreadsheet compares academic performance to free and reduced lunch (F&RL) qualifying rates.
Oh, dear. Kevin, do we really need to say this again? Apparently so.
Aggregate test scores are a poor measure of school performance.
"It’s important to note that 76% of DPS students are either black or hispanic, and nearly two-thirds of DPS students are economically disadvantaged."
You put this in one sentence, three-quarters of the way down the article, in parentheses, when it really should occupy at least half the article.
Very poorly done. Do better next time.
Posted by: Michael Bacon | January 19, 2016 at 11:11 AM
Okay, on re-read, you attempt to get at this, so perhaps I'm being too harsh in language. But the content of my point stands.
If you're going to do the analysis, year-over-year improvement in test scores is at the very least a much better rubric, but it's still an extremely crude and biased measure.
The problem that a large proportion of DPS students face is poverty. Expecting the schools to completely ameliorate all of the disadvantages conferred with poverty with six hours/day of instruction is absolutely insane and I'm sick to death of reading people who should know better perpetuating this.
Posted by: Michael Bacon | January 19, 2016 at 11:16 AM
Is there any historical trend that can be teased out? I know extensive grade level testing is a product of the 1990's and early 2000's, but is there any historical data on previous tests used? This idea of struggling to get ahead sounds plausible, but how long have we been struggling?
It looks like NC has had some form of standardized testing for high school completion since 1978; end of course tests for Algebra I and English I appear to have started around 1985. (page 26 of this document: http://www.dpi.state.nc.us/docs/acre/history/accountabilityfinalreport.pdf )
(maybe I'll get bored and look into it myself later)
Posted by: Rob G | January 19, 2016 at 11:24 AM
"DPS in 2014-15 saw nearly two-thirds of its students qualify for the federal free/reduced lunch program. Within the state, there were 48 out of 115 districts with the same or greater levels of poverty. Against just that comparison set, DPS still was below-average:
* Overall GLP and CCR rates: 31st and 28th out of 48 respectively
Most of the districts performing worse than DPS in this measure have a F&R lunch eligibility rate at least 10% higher than DPS."
Michael, I wouldn't disagree that poverty conveys a deep disadvantage. However, DPS is in the bottom half of performance among districts state-wide with an equal or greater level of poverty.
I'm not expecting DPS to counterbalance all of the disadvantages that poverty confers. But it is reasonable to look at how DPS performs relative to that comparison set.
One of the things we'll be scrutinizing in the remainder of this series is, where is DPS spending the significant extra local funding that it receives from Durham taxpayers -- among the highest in the state, and giving it per-pupil resources several thousand dollars ahead of peer districts. The ultimate scrutiny I hope arrives during the DPS budget cycle and new DPS-BOCC members' tenure is, are those dollars going to the most effective places to close performance gaps and to provide a sound education to all?
Fundamentally, the thesis here is that DPS can do better with its resources (not that it needs less resources), and that the performance gap between white/non-white and well-off/impoverished students, in Durham and statewide, is appalling.
Posted by: Kevin Davis | January 19, 2016 at 12:10 PM
Kevin,
Is there any insight to be gained by looking at the economically disadvantaged cohort as a group and then sifting through that segment by race, gender, ethnicity, or geographic location within the County? Likewise, how do these slices look among those who do not qualify for free or reduced lunch?
I hold out some hope that the true achievement gap by race, controlled for poverty, will not be nearly as wide.
Posted by: Dave Neill | January 19, 2016 at 01:09 PM
Could there be interactions between race/ethnicity and free/reduced lunch rates that might account for DPS’s low performance with “economically disadvantaged students” compared to other NC districts? It would be nice to have some data about the proportion of students receiving free/reduced lunch who are Hispanic or African American across districts. I've quickly googled for some, but I haven't found any. Could it be that DPS has a disproportionate amount of students who are *both* minorities and economically disadvantaged (i.e., a double whammy)? This makes some intuitive sense to me, but I might be way off.
Posted by: Justin | January 19, 2016 at 01:28 PM
@Justin-
This is an old report but it has some interesting figures. I'm assuming it passed review by the faculty adviser of the student.
https://econ.duke.edu/uploads/assets/dje/2008/Manderscheid.pdf
Posted by: Rob G | January 19, 2016 at 02:27 PM
I look forward to seeing an analysis of where the DPS money is going compared to other districts. My experience with DPS is that money is not going to school supplies (paper shortage), athletics, or facilities. How a district with relatively flat enrollment can have dozens of 'mobile units' (a.k.a. trailers) that serve as classrooms on a permanent basis is a mystery.
Posted by: Todd P | January 19, 2016 at 10:46 PM
The Manderscheid paper is excellent and worth a read. I did my own analysis of it as part of the Two Durhams series here at BCR, nearly seven years ago. (For more info, see: http://www.bullcityrising.com/2008/03/two-durhams-sep.html)
And I still accept the underlying thesis of that article, namely that poverty is a major driver of school performance, cutting across state lines.
I've been intrigued by the lens of analysis that Alex Modestou has brought to the conversation, though, because it highlights a disparity that doesn't make sense to me across North Carolina districts.
The patterns of white advantages in resources and wealth are national, not local phenomena. I would not be surprised (disappointed, yes, but not surprised) to see white students at nominally higher passage rates on standardized tests than black or Hispanic students.
What's not explained purely by poverty to me, however, is what would lead DPS white student performance (GLP) to rank 6th/115, black student performance 60th/115, and Hispanic student performance 105th/115.
I'm skeptical that the _relative_ difference in performance, across districts, is explained entirely by poverty. I would have expected white, black and Hispanic students' relative performance at DPS versus other districts to have been more alike. If DPS ranked, say, 40th in white performance, 45th in black performance, and 39th in Hispanic performance, we'd still want to see improvements, but at least there wouldn't be questions of relative equity.
To me, this is where other questions become interesting. Do factors like discipline and in-school suspensions (which we'll just touch on, but which is and will continue to arise as a topic more broadly) in other districts explain some of the difference? Does DPS disproportionately fund programs that appeal to white parents and families while underfunding programs for Hispanic, LEP, and EC students?
The data we can access in this series don't lead to definitive conclusions on these kind of questions. They do suggest, in some cases, that we are spending significantly more in areas like programs for targeted populations than peer districts, yet see extremely low performance in those areas; and, that areas like regular classroom instruction or CTE see relatively the same or less funding as peer districts.
The bottom line, though, is in the need for more scrutiny as to why relative performance gaps are so stark, and whether DPS funding priorities can or should realign to (a) spend money more effectively and (b) narrow at least the relative performance gaps among districts.
Posted by: Kevin Davis | January 20, 2016 at 07:07 AM
Todd,
After teaching in a semi-permanent "mobile unit" immediately adjacent to a bus depot, I was surprised to learn that DPS currently has the 2nd highest 5-year-average per-pupil capital outlay costs among all 115 districts in the state. The high ranking is also unexpected given an annualized growth rate in DPS of less than 1% over the past 5 years.
http://apps.schools.nc.gov/pls/apex/f?p=1:39:1580177845480801::NO:::
Posted by: Alex Modestou | January 20, 2016 at 08:08 AM
Alex,
I am not sure if a 5-year comparison goes back far enough. This could reflect where counties are in their bond cycle of funding capital needs. Maybe Durham has been playing catch up on capital spending recently after years of neglect, or maybe this is another area where spending does not match the end results. I do think we would need to look at a 10 or 15 year cycle of capital spending to really tell the tale.
In any case, the idea that we are 2nd in the state for capital spending while having a dozen semi-permanent trailers at Riverside is curious.
Posted by: Todd P. | January 20, 2016 at 10:42 AM
I'll hold my tongue then on further comment and wait until you finish the series to see where you're headed with this. I still have some pretty substantial concerns that you're missing some awfully massive drivers of cost and performance, but let's see where this goes.
Posted by: Michael Bacon | January 20, 2016 at 10:32 PM
I'm assuming perhaps inaccurately that the students are relatively mixed in terms of race in their classes. If so then the problem doesn't seem to be teachers but factors external to the classroom. No matter how much money you throw at the teachers it won't fix the problems outside the classroom: the need for more role models, structure, expectations, etc.
Posted by: Joshua Allen | January 23, 2016 at 12:39 PM
Joshua, who wrote that teachers are the problem?
Posted by: Erik Landfried | January 23, 2016 at 10:58 PM
I agree with comments on accounting for FRPL. I made some graphs to illustrate.
https://durhamstats.wordpress.com/2016/03/01/school-performance-and-frpl-2/
Posted by: Durham Stats | March 01, 2016 at 11:49 PM