The number Francis read out at Ann Arbor on 12 April 1955 was believed at once, by almost everyone, and it has not been seriously doubted since. That is rarer than it sounds. Most of the vaccines then in use had been introduced on far thinner evidence; the polio researchers who praised the trial that spring pointed out that the country now knew more about the Salk vaccine than it knew about typhoid vaccine after fifty years of use or tuberculosis vaccine after thirty. The reason it was believed is the design, and the design was not the one the Foundation had wanted.
Why a trial at all
Polio was never a common disease. In the early 1950s it caused about six percent of the deaths among American children aged five to nine. What made it feared was its selectiveness and its unpredictability: it struck children, it left some of them in braces or in respirators for life, and it came in epidemic waves that no one could forecast, so that a town that had escaped one summer waited in dread for the next. In the United States as a whole the reported incidence ran at about 50 cases per 100,000 people a year through the decade, and it varied wildly from year to year and place to place.
That variability is the whole problem of evaluation. Paul Meier, the statistician who later made the trial a textbook case, put it with a chart of the annual reported incidence from 1930 to 1955. Had a vaccine been introduced nationally in 1931, the fall in incidence to less than a third of the previous year’s would have been taken as proof of a highly effective vaccine; the same false proof was on offer in 1935, 1937 and 1952, years in which nothing had been introduced at all. Nor would it have helped to vaccinate one state and compare it with another: an epidemic might strike Chicago, as one did in 1956, in a year when New York was quiet. And the disease was hard to diagnose. The classic case in a respirator was unmistakable, but most cases were fever and weakness, which are the symptoms of many illnesses, and the line between weakness and slight transitory paralysis was drawn differently by different physicians, and drawn, inevitably, with some knowledge of how much polio was about.
So the trial had to be large, and it had to be controlled. Suppose, Meier wrote, the vaccine were fifty percent effective and the incidence 50 per 100,000. With 40,000 children in each of two groups, one vaccinated and one not, the expected cases would be about 20 and about 10, a difference that could easily be chance. With 100,000 in each, 50 and 25, which would persuade. And because the real criterion would be paralytic polio, which is about half of all diagnosed cases, and because the trial areas might have a quiet season, the groups had to be bigger still. In the end the study population was 1,829,916 children.
The plan the Foundation wanted
The National Foundation for Infantile Paralysis had paid for the vaccine and it would pay for the trial, and its first plan was simple. Offer vaccine to every second-grader in the participating schools, and watch the first- and third-graders in the same schools as the comparison. Meier called this the observed control approach. It had real merits. All three grades were in the same towns and the same season, so an epidemic that reached the second grade would reach the first and third; the ages were close enough that the average of the two outer grades should stand well for the middle one. It was also cheap, and it asked nothing of a parent except permission to vaccinate.
Several state health departments that were expected to take part would not have it, for two reasons. The first was the diagnosis. Every physician in a trial area would know that only second-graders had been offered vaccine, and in deciding whether a feverish seven-year-old had polio or something else he would naturally ask whether the child had been vaccinated. Any tendency to decide the doubtful cases in the vaccine’s favour would produce a spurious result, and there would be no way afterwards to tell how much of the result was spurious. The second was the volunteers. The families who would hand in a permission slip were not the families who would not; it turned out later that they were better educated and better off, and that their children were absent from school more often. Which way that would push the polio rate nobody could say in advance, and that was the point.
The critics proposed a scientific experiment instead: assign children to vaccine or to control by a chance mechanism, so that the two groups differed in nothing but the vaccine, and give the controls an injection that looked exactly like the vaccine, so that no one who examined a child afterwards could know what he had been given. The objection to this was that parents would not consent to their child’s being injected with salt water when the child might instead have been given a vaccine that was probably helpful, and that an injection of anything carried a small risk if the child happened already to be incubating polio. The proponents answered that parents, properly approached, would consent, and that the injections would be given before the polio season, when that risk was vanishingly small. “Certain health departments took a firm stand,” Meier wrote: “they would participate in the trial only if it were such a well-designed experiment.” The Foundation gave way in part. In about half the areas the trial was run the health departments’ way, with placebo controls; in the rest it was run the Foundation’s way, with the alternate grades observed.
What the placebo areas built
In the placebo areas every child in the first three grades whose parents consented was given three injections, in April, May and June 1954, from a vial that bore only a code number. Half the vials held vaccine and half held salt solution, and the assignment was by chance. The nurse who gave the injection did not know which she was giving; the parents did not know; the physician who saw the child in August did not know; and the laboratory that received the child’s blood and stool did not know. Only when every reported case in the trial had been classified, as not polio, doubtful, non-paralytic, paralytic or fatal, was the code broken and each case matched to what the child had received.
The classification itself was built to resist the vagaries of diagnosis. Every case of polio-like illness reported by a local physician was examined by one of the trial’s physical therapists at the time and again two months later, when the second muscle examination measured residual paralysis rather than the first impression; a specimen went to one of twenty-seven laboratories for virus isolation and antibody tests; and a diagnostic group at Ann Arbor read the file. The Evaluation Center’s statisticians, several of them on loan from the Bureau of the Census, kept the records on punched cards, and the University of Michigan’s tabulating service ran them.
Chance assignment did two things at once, and Meier was careful to separate them. It balanced the groups on every factor that might affect polio, including the ones nobody had thought of, better than any deliberate matching could; and it made the theory of probability legitimately applicable to the result, so that one could say how often so large a difference would arise from the assignment alone. “For most of us,” he wrote, “it is something of a shock to be told that competent and dedicated physicians must be kept in ignorance lest their judgments be colored by knowledge of treatment status.” The concern was not that any physician would distort a finding; it was that an uncertain decision, made many times, drifts.
The numbers
The study population, by the summary report’s table, divided like this. In the placebo areas, 749,236 children: 200,745 vaccinated, 201,229 given placebo, 338,778 whose parents had not consented, and 8,484 who had started the course and not finished it. In the observed areas, 1,080,680: 221,998 second-graders vaccinated, 725,173 first- and third-graders watched, 123,605 second-graders whose parents had declined, and 9,904 incomplete.
Across the whole trial 1,013 cases were reported and 863 confirmed as polio, 685 of them paralytic and 15 fatal. In the placebo areas there were 358 confirmed cases: 57 among the vaccinated, of which 33 were paralytic, and 142 among the placebo children, of which 115 were paralytic. As rates, that is 16 paralytic cases per 100,000 vaccinated against 57 per 100,000 on salt water. No vaccinated child in the placebo areas died of polio; four placebo children did. In the observed areas the vaccinated second-graders had 38 paralytic cases, 17 per 100,000, against 330 among the 725,173 controls, 46 per 100,000; none of the vaccinated died and eleven of the controls did. Within families the vaccine held too: of 223 vaccinated children exposed to a case in the household, one fell ill; of 244 placebo children so exposed, eight.
Francis put the two estimates side by side and said which he trusted. From the observed areas alone “the vaccine could be considered to be 60 to 80 per cent effective against paralytic polio, 60 per cent against type 1 polio and 70 to 80 per cent against disease caused by types 2 and 3.” But: “There is, however, greater confidence in the results obtained from the strictly controlled and almost identical test populations of the placebo study areas. On this basis it may be suggested that vaccination was 80 to 90 per cent effective against paralytic poliomyelitis, that it was 60 to 70 per cent effective against disease caused by Type I virus and 90 per cent or more effective against disease from Type II and Type III virus. The estimate would be more secure had a larger number of cases been available.” Against bulbar polio, the form that reaches the brain stem and kills, the estimate in the placebo areas was 81 to 94 percent; against spinal paralysis it was lower, about 60 percent, with a lower limit of 39. The weakness against Type I agreed with what the laboratories had already found, that most lots of vaccine were less potent against that type, and the effectiveness of different lots, as measured by cases, varied considerably.
The volunteer effect the critics had feared is visible in the table. In the placebo areas the children whose parents had not consented had a paralytic rate of 36 per 100,000, well below the 57 of the placebo children, who were volunteers like the vaccinated; in the observed areas the second-graders who had declined had 35 against the controls’ 46. Had the Foundation’s plan been used alone, the vaccinated second-graders would have been volunteers compared against grades that were not, and the comparison would have carried that difference inside it with no way to take it out. “Were the observed control information alone available,” Meier wrote, “considerable doubt would have remained about the proper interpretation of the results.” The difference the vaccine made was large and definite, but not so large that it could have survived the doubt.
Safety, as the trial saw it
The trial also looked for harm, and found none. Of the children vaccinated, 931 had minor reactions, and of the placebo children 939, the same rate, four in ten thousand. Of the so-called major reactions none could be attributed to the inoculation; there were nine among the vaccinated and thirteen among the placebo children. There was no evidence that the vaccine was itself a source of infection: the cases among vaccinated children did not cluster in the weeks after injection, did not attach to any lot, and did not favour the left arm, where every injection was given. In Massachusetts, where the reported disease had been so mild that doubt existed whether much of it was polio at all, poliovirus was recovered from only three of 44 cases, and unidentified “orphan viruses” from 31. The Public Health Service’s chronology, written a year later with the Cutter cases in front of it, summarised the trial’s safety finding in one sentence: the results “indicated that there was no evidence of disproportionate frequency of paralytic poliomyelitis in vaccinated children up to four weeks after injection, nor of selective localization of paralysis.”
That finding was true, and it was the finding that was carried to Washington on 12 April. It was a finding about the lots of the spring of 1954, made by two firms under triple testing, after the early lots had failed and the trial had been held back a month. Fairfax County, Virginia, had been the first place in the country to inject the commercial vaccine that spring, in the middle of a measles epidemic that sent frightened parents to the telephone with every fever. Its health officer, Harold Kennedy, told the Evening Star on 12 April 1955 why he had never worried: “I was satisfied there was no danger—just look at the high caliber of the Medical Advisory Committee to the national foundation. Men like that certainly would not issue any vaccine if they had the slightest notion it might be dangerous.” Of the 2,277 Fairfax second-graders vaccinated, one, a nine-year-old girl, had caught polio that summer; she could walk. “Anyone familiar with vaccine knows that none is 100 per cent effective,” Kennedy said. “The best that could be hoped for is 80 to 90 per cent effectiveness.” Her brother was due for his injection that spring.
Sources
- Thomas Francis, Jr., and others, An Evaluation of the 1954 Poliomyelitis Vaccine Trials: Summary Report, University of Michigan, 12 April 1955, as printed verbatim by the Associated Press in the Evening Star (Washington), 12 April 1955, pp. A-1, A-2 and A-4, and as tabulated by Meier. 12 April p. 1, p. 2, p. 4
- Paul Meier, “The Biggest Public Health Experiment Ever: The 1954 Field Trial of the Salk Poliomyelitis Vaccine,” in Statistics: A Guide to the Unknown, 1972. webpages.math.luc.edu
- United States Public Health Service, Technical Report on Salk Poliomyelitis Vaccine, June 1955, chapter 1 and Appendix A. Global Health Chronicles
- Neal Nathanson and Alexander D. Langmuir, “The Cutter Incident,” I, American Journal of Hygiene 78, 1963, pp. 16 to 28, on the analysis of the field trial’s own cases for association with vaccination. Global Health Chronicles
- Mary Lou Werner, “Fairfax Health Officer Repaid for His Work,” Evening Star, 12 April 1955, p. A-3. loc.gov