Education Disinformation Detection and Reporting Agency -- a Gerald Bracey Report on the Condition of Education | ||
EDDRA Articles Related Article: Simpson's Paradox... |
THE UNSEEN PROGRESS OF AMERICAN SCHOOLS Eight years ago, I published "The Greatly Exaggerated Death of Our Schools" in Sunday Outlook. The condition of public education in these United States has since occupied a large proportion of my professional time. While my Outlook argument made the case that American schools were doing at least as well as ever, my ensuing analyses of data made it clear that they were actually doing better than ever. It was thus with some dismay, but hardly any surprise given the enormous amount of gratuitous violence visited on our public schools, that I read about our deteriorating school system in the June 10 edition of the Wall Street Journal. While the Journal's main targets were the proposed testing guidelines from the Office of Civil Rights, along the way the editorial managed to declare that "there is no longer any pretense about the disastrous decline in education". How does the Journal know that there has been such a decline? The Journal declines to say. Where would we look to find evidence of such a decline? How about in SAT scores? No, those have been rising. And the proportion of high scorers in math was at an all-time high before the SAT scale was recentered. I mentioned this fact and listed numerous other statistics showing improvements in education in another Post article, this one on the op-ed page of December 22, 1995. I have just dug into what is perhaps both the best and worst source of information about what is or is not happening in schools, the trends from the National Assessment of Educational Progress (NAEP). NAEP trends are the worst source in that the assessments are not connected to any particular curriculum. People could be doing wonderful things in classrooms and it might not show up on NAEP. In addition, being low-stakes tests with no consequences for anyone, no one takes them very seriously. But these two facts also make NAEP the best source for information because it is untainted. No one teaches to the test. No one changes answers to raise scores. For the last 20-odd years, NAEP has been periodically testing nationally representative samples of students in a variety of subjects. The longest-term trends we have are from reading, mathematics and science stemming from 1971, 1978, and 1977 respectively. If one looks at the overall averages, one sees some improvement, perhaps not a lot. But these averages mask significant improvements for blacks, whites and Hispanics (until the most recent NAEP assessment, the sample of Asian students has been too small to generate a reliable score to be reported separately). How can all three ethnicities be gaining while the averages are relatively stable? The answer is known in statistical circles as Simpson's Paradox. It occurs because at the different points in time, the ethnicities constitute different proportions of the total sample. To look at America in 1971, the first reading assessment date, and 1996, the most recent available for trend analysis, is to look at two very different Americas, ethnically speaking. Minorities constitute a much larger proportion of the population now than they did then. This is important because, while minority scores are rising, they remain lower than the scores for whites. This means that as they become a larger part of the whole, their scores depress the average. A hypothetical example can make this statistical abstraction concrete and clear: Time 1 Time 2
490 486 average Consider the 500's at Time 1 to be scores of white students. Consider the 400 as scores of minorities. Consider the 510's at Time 2 to be the scores of white students and the 430's to be those of minorities. From Time 1 to Time 2, the scores of both white and minority students improve. Those of minorities improve much more than those of whites, but their scores remain below those of whites. Even though everyone has gained, the average from Time 1 to Time 2 falls. How can this be? This can be because minority students made up only 10% of the total at Time 1, but 30% of the total at Time 2. The table presents the results of NAEP trend analyses for reading, mathematics and science for whites, blacks and Hispanics. It shows not only the average scores, but also the scores of low performers (5th percentile) and the scores of high performers (95th percentile). For virtually all comparisons, the scores are up. The increases for black students can only be called spectacular. Consider the gains for 17-year-old black students in reading: 35 points for the lowest scorers, 26 points for the average and 20 points for the best students. The gains for Hispanics are significant, but more modest. They might well be moderated by the continuing immigration of students whose first language is not English. It is one thing to converse in a second language, it is quite another to read it well enough to take tests, especially a multiple choice test where the test maker tries to trick you into choosing a wrong answer. I invite the reader to pore over these tables. I invite the editors of the Wall Street Journal to print a retraction. I invite the critics of education to shut up. | |
|
© 1999 Gerald Bracey Last updated June 27, 1999 |
Web Services by | |