The state of Florida is apparently about to launch the first large-scale use of vouchers. The legislation has passed the house and is expected to pass the senate today, Friday, April 30, 1999. Since Governor Jeb Bush is its sponsor, he no doubt will sign it. Under the plan, all schools in Florida will be graded A to F. Students at schools with failing marks can obtain vouchers allowing them to attend private schools at state expense.

Here's what the research to date says about the wide-scale use of vouchers: nothing.

What has been done so far consists of small-scale demonstrations which, even if they were valid, would say nothing about what vouchers would mean if used on a large scale. And the research on voucher programs to date has been very much in dispute. It does not have an honorable history.

The first research report on vouchers was conducted by John Witte, a political scientist at the University of Wisconsin, who was hired by the State of Wisconsin to evaluate the voucher program in Milwaukee. After five years of evaluations, Witte and his team concluded that there was essentially no difference in the achievement of Milwaukee Public School Students and students attending private schools at state expense.

This interpretation was challenged by Paul Peterson of Harvard University who has managed to become the Right's "designated researcher" for all voucher programs in the country. Peterson has said that he does not trust any data that he has not himself collected. In the instance of Milwaukee, Peterson claimed that choice worked, it just took four years for the effect to show up.

Peterson has also written that voucher advocates are "a small band of Jedi attackers, using their intellectual powers to fight the unified might of Death Star Forces led by Darth Vader, whose intellectual capacity has been corrupted by the urge for complete hegemony ("Monoply and Competition in American Schools", p. 73, in William Clune and John Witte (Eds.) Choice and Control in American Education, Volume 1, 1991, Stanford Series on Education,). Such statements lead some people to question Peterson's objectivity.

Peterson's handling of the Milwaukee data does not inspire confidence. He initially released his "findings" as an AP wirestory. That story just happened to run the same day as Peterson and protégé Jay P. Greene authored an op-ed piece in the Wall Street Journal (one of the writers at Slate magazine, while extolling the WSJ's reporters, called the editorial page a "viper's nest of right-wing vitriol". That seems accurate enough). That day just happened to be the same day that Presidential candidate Robert Dole addressed the Republican National Convention on education and called for vouchers. Within a few days, Lamar Alexander and Bill Bennett had both gotten themselves on TV talk shows to extol the merits of Peterson's study and Rush Limbaugh had sent the study across the airwaves.

As if a single study could prove anything!

Peterson's study was especially problematic since it was impossible to tell from his actual paper what statistical procedures he had used (thus violating a premiere research canon). Other researchers have since been unable to replicate Peterson's findings. Actually, the few tables that Peterson provided contradicted some of his conclusions. About a year later I heard him speak at the Cato Institute where he was using a "cleaned up data set" that made the findings seem more firm.

Princeton University economist, Cecilia Rouse, was able to find a significant gain for math but not for reading. And, more important, she was unable to find any positive impact when she used scores for the same students for two. The effect was ephemeral.

This seems to be the conclusion that Michael Timpane, former President of Teachers College at Columbia and now with the RAND Corporation is also approaching. Timpane was asked by the Pew and Spencer Foundations to examine the data as a neutral observer.

For my part, after looking at Peterson's original paper, I noted (in the Sixth Bracey Report on the Condition of Public Education, Phi Delta Kappan, October, 1996) that if a positive outcome existed, and I was by no means certain that it did, it might say absolutely nothing about the positive impact of choice. By the time Peterson looked at the results of four years of the program, he was looking at a most unusual group of students: poor, inner city kids who had been in the same school for four years. This stability alone could produce positive outcomes.

My gut feeling, however, is that no issues can be resolved in Milwaukee because the data are so bad: the are so many missing test scores from the kids and so many missing surveys from the parents that were this a less politically charged affair, people would call the data garbage and walk away from it (fewer than 1/4 of the participants had both parent survey responses and test scores).

Peterson claimed to find positive outcomes for the New York City voucher program as well. Here Peterson has tried to buy some respectability outside of Right-Wing circles by affiliating with Mathematica Policy Research, an organization with offices in Princeton and Washington, DC.

The positive outcome here, to my way of thinking is problematic because it compared the results for the voucher-using children with a non-existent other group. This other group is a "similar" group of students who did not receive vouchers, but who would have used them if they had. This group is constructed using statistical means which, quite frankly, I don't fully understand. But, to my way of thinking, basing conclusions on hypothetical motivation of non-existent groups is problematical (especially when you don't tell people that's what you've done).

The effects, if they exist, are quite small in the short term. Peterson contends that if the effects were sustained each year, they would be quite significant and this is true. As Timpane said, though, education has a glorious history of showing modest short-term gains which disappear in the long run. This Peterson study can be found as "An Evaluation of the New York City School Choice Scholarships Program: The First Year," data.FAS.Harvard.edu\PEPG\. (There is no "www" in the address).

And, perhaps more important, a demonstration, using a small number of students says nothing about what would happen if the experiment were expanded to a whole system where large numbers of poor students were involved, where special education students and limited English proficient students would be included. This is something even Peterson has acknowledged.

This means that the Florida experiment, unless it dies in court (its constitutionality is already being challenged), will be important because it will operate on such a large scale in comparison to everything that has preceded it. It will be important, of course, only if a thorough and disinterested evaluation is conducted. Peterson says he doesn't trust anyone else. I don't trust Peterson, the "Jedi attacker." It looks like there will be lots of room for dispute.

Unfortunately, vouchers have become such a cause celebre on the political right that those pushing vouchers are not asking if that reform, even if it has positive outcomes, is the best thing to do for poor kids. A study by Alex Molnar of the University of Wisconsin at Milwaukee indicates that small class sizes might well be more effective. Molnar's study "Smaller Classes, Not Vouchers Increase Student Achievement" is available from the Keystone Research Center, 412 North Third street, Harrisburg, PA, 17101.

Speaking of class size, we do have a large scale experiment in this. Actually, we have two. The best is from Tennessee's Project STAR. This state-funded experiment looked at regular classes, small classes, and regular classes with full-time teachers' aides. The beauty of the experiment was that teachers and kids were randomly assigned to the three treatments within each building, meaning there was no chance of any selection bias.

The other large scale experiment on class size is ongoing. It is called California. Unfortunately, there are many horror stories emanating from the Golden State about the difficulties in finding qualified teachers and/or classroom space to meet the mandate.

It is also true that the classes in Milwaukee's voucher schools were considerably smaller than in the public schools. Any positive outcome could well be related to smaller class size rather than choice.