The entry ends with the mirror corrected and the telescope working. This piece follows the three and a half years between the announcement of the flaw and the first corrected pictures, from the record: the Senate hearing of 10 July 1990, in which the people responsible argued over the test that was not done before anyone knew what had gone wrong; the board’s findings; the fix; the settlement; and the servicing mission of December 1993.

The week the pictures came down

The Hubble Space Telescope went up in Discovery on 24 April 1990 and was released the next day. The project’s policy was that no first-light image would be recorded until the fine guidance sensors had found best focus, and the sensors were struggling: the pointing control was disturbed each time the spacecraft crossed the terminator and the solar arrays flexed, and the star trackers made several false acquisitions, so that only three of the first sixteen succeeded. The first picture came on 20 May. The Wide Field/Planetary Camera then took images at a chosen secondary-mirror position and “the initial image analysis indicated significant defects”; since the secondary had only been moved along the axis, “it was still believed at the time that corrections could be made by tilting or decentering the mirror.” The wavefront sensors in the guidance system were turned on; they and the camera both showed a large spherical aberration; calibration tests showed it was not inside the camera; the secondary was tilted and decentred and nothing improved. Star images taken on 14 June and 21 June were matched against computer models with 0.5 of a wave of spherical aberration, and fitted. The error appeared on axis and off it alike, with no significant coma, which placed it in the primary mirror. The project manager was told, and on 21 June the contingency plan was put into effect. NASA announced the flaw on 27 June.

Two panels were formed. The Independent Optical Review Panel, six optical scientists, Duncan Moore of Rochester and Aden Meinel of the Jet Propulsion Laboratory among them, was briefed on 5 July and signed nine findings: about half a wave of spherical aberration in the telescope assembly; not correctable with any of the existing controls; coma and astigmatism, if found, correctable; replacement instruments “can be corrected for the spherical aberration error of the telescope assembly so that the original performance targets can be met”; the error’s size to be determined from the images; the backup primary and the number two secondary to be tested; some optical elements of the replacement instruments to be replaced or added; a wavefront sensor of wider range to be considered; and a simulator of the flawed telescope to be built so that the new instruments could be tested before launch in 1993. The second panel was Lew Allen’s board, established on 2 July, which met in Washington on 5 and 6 July.

10 July 1990

The Senate’s Subcommittee on Science, Technology and Space, chaired by Albert Gore of Tennessee, had heard Lennard Fisk on 29 June. On 10 July it heard him again, with NASA’s deputy administrator James Thompson, the head of space flight William Lenoir, and George Rodney, and then the two contractors. The members had spent the morning in a classified session on other things the United States had put in orbit, and Gore was careful about what he could say.

He opened with numbers. Between 1970 and 1985 NASA had cut its quality assurance staff by 70 percent, and Marshall by 87 percent. In twelve years the telescope had had fifteen programme managers across Marshall, Goddard and headquarters. Larry Pressler of South Dakota asked the layman’s question: why had the telescope never been tested as a system? Thompson: “Well, you are right, it was not tested. The optical system is an all-up system. At that time, we did not feel it was necessary.” Fisk explained that there had been many end-to-end tests of the spacecraft, but not “a combined test of the two mirrors working together”, and that the judgement had been right: “we were building the world’s greatest mirrors here, and therefore we would have had to build essentially the world’s greatest test facility to test the world’s greatest mirrors.” Then he conceded the thing the board would later put in writing: “we are going to find that there were other tests that we could have done and should have done that would have revealed this problem.”

Gore disagreed with where that left it. “I think it is not just a technological decision, I think it is a philosophical issue. Regardless of how confident you are in the testing of the components separately, there is something about a final assembly test that gives you a reality check, a fail-safe procedure, that can catch problems which get through the component testing.” He had learned that morning that of the two bidders, Kodak and Perkin-Elmer, “One of the bidders did propose a final assembly test. The winning bidder did not.” NASA had chosen Perkin-Elmer for its strength in the pointing system, which was thought the harder problem, and “what went wrong is, in my opinion, NASA did not require that both bidders include in their bid proposal a final assembly test”. Then the passage the hearing is remembered for: “When you build a ship, you test it in the water before it is accepted. When you build an automobile, you drive it before you buy it. When you build almost anything, the final test is the final product. When you are putting on a play, every performer goes through the part of the play that that person has to perform. But the director, or the person responsible for it, before opening night, they say let us have a dress rehearsal to see how everything fits together. That, in my opinion, should have been required before the Hubble was flown.”

Thompson, who had directed Marshall from 1986, agreed in retrospect, and said that when the board found the fault “we will all look back and say, gee, we could have done that test for a fraction of what the pain has been to go through it. And I will lead that parade.” Fisk made the optician’s point, that the design “is a very standard design for an optics system. There is nothing exotic about this thing,” and that “had the test on each mirror individually been correct, then by definition they would have worked together”. Gore asked whether the decision had been reopened after the award. Fisk said the testing policy had been “common knowledge” and pointed to a record of 1979 or 1980 in which the project manager told the science group that no end-to-end test was being done; there had been “no ripple”. Thompson, who had chaired quarterly reviews from 1986, said: “I never heard word one.” Gore asked for the document. Later, on the shuttle’s leaking fuel umbilicals, he said a sentence that belongs to the whole afternoon: “You shot the messenger, in effect.”

John Rich of Hughes Danbury read a chronology, contract in October 1977, blank ordered from Corning in November, received a year later, grinding and polishing complete in December 1980, coating complete in February 1982, the assembly delivered in October 1984, and a defence: “The mirror configuration test we developed and relied heavily on and NASA approved was the only test that we judged to be sufficiently precise to meet the program’s requirements.” Asked why the company had not bid a final assembly test, he said: “Senator, unfortunately, I found out something here today, and that is that Kodak did put in their bid a final assembly test. So that is new information to me.” Gore: “Let me state openly here that the cost of the final assembly test and facility for same included in the Kodak bid was $10 million, not $100 million, not $150 million, but $10 million on a project that turned out to be $1.5 billion.” Rich’s letter of 3 August, for the record, added that he was “not aware of any evidence that judgments made in the 1977 to 1982 time frame were not sound”, and that “the final assessment on this issue must await the more detailed fact finding by Dr. Allen and his colleagues.”

Rodney’s answers for the record came after the board’s work had begun. “We are unable to explain why quality assurance was not more intimately involved in the calibration and certification of the reflective null corrector in the 1978 and 1979 time frame.” And: “The level of testing performed on the HST appeared appropriate at the time based on the assumption that the individual elements were thoroughly tested and met their individual specification requirements. As we now know, this assumption was incorrect.”

The prescription

Within two weeks of the hearing the project’s priority had shifted from cause to cure. On 23 July the programme scientist, Edward Weiler, told reporters that NASA hoped to have the prescription for the corrective mirrors within about two weeks, that they would be “about the size of a nickel”, and that “determining an accurate prescription (for the new camera’s mirrors) is the No. 1 priority of this program right now.” The replacement Wide Field and Planetary Camera had been in development long before the flaw and was due on orbit in June 1993; NASA now looked at bringing it forward by three to six months for about 10 million dollars. A doughnut-shaped mask over the outer mirror was considered and ruled out. The board’s measurement of the 1.3 millimetres, confirmed in August and explained by the field cap in September, gave the prescription its ground truth, and the old interferograms through the refractive corrector, reprocessed, gave a second opinion that agreed.

For the instruments that could not be rebuilt in time, NASA designed the Corrective Optics Space Telescope Axial Replacement, COSTAR, a unit the size of a large refrigerator carrying small mirrors ground with the opposite error, to be put in the place of the High Speed Photometer and to feed three of the other instruments corrected light. It cost 50 million dollars. A review team under Thomas Stafford, the Apollo and Gemini astronaut, first met in September 1992 to look at the servicing mission, and reported itself “very concerned about relaying overly optimistic expectations to the public”.

The settlement

Hughes Aircraft had bought Perkin-Elmer’s optics division in December 1989 and, as part of the purchase, its liabilities. Through 1990 and after, Perkin-Elmer “resolutely denied any wrongdoing” and held that NASA had been given all the data. On 4 October 1993 the Justice Department settled: 25 million dollars to NASA from the two companies, Perkin-Elmer paying 15 million in cash and Hughes forgoing 3.5 million in remaining on-orbit fees and about 6.5 million in fees on future work. The department’s statement said Perkin-Elmer “knew or should have known of the defect”, and released both companies from further liability under the False Claims Act; the agreement said it was not to be construed as an admission of liability by either. Perkin-Elmer’s chairman said the company believed it had acted responsibly and chose to avoid a costly legal fight.

On 16 November 1993 the House Subcommittee on Space heard the accounting. NASA’s general counsel, Edward Frankle, put the agency’s “actual, monetary damages … from the defect in the mirror” at 87 million dollars, which “certain legal defenses” prevented it from recovering, and called the settlement “fair and reasonable”. The inspector general, Bill Colvin, said that “obvious evidence of errors existed at all stages of the manufacturing process of the primary mirror.” Perkin-Elmer had been paid some 440 million dollars for the optical assembly and the fine guidance sensors. The chairman of Hughes, C. Michael Armstrong, said: “It is our understanding that NASA knew everything that Perkin-Elmer knew about the polishing of the mirror. But no one at NASA or Perkin-Elmer ever told Hughes Danbury about a potential problem with the mirror when Hughes Danbury became involved with the Hubble. If they had, Hughes would not have gone through with the purchase or taken over the Hubble contract without proper indemnification.”

Five spacewalks

Endeavour lifted off from Pad 39B at 4:27 in the morning of 2 December 1993 with Richard Covey in command, Kenneth Bowersox as pilot, Story Musgrave as payload commander, and Kathryn Thornton, Claude Nicollier of the European Space Agency, Jeffrey Hoffman and Thomas Akers. Nicollier caught the telescope with the arm on the third day. Then, on five successive days, two pairs of astronauts went out in turn. On 5 December Musgrave and Hoffman replaced two rate-sensing units, two electronic control units and eight fuse plugs. On 6 December Thornton and Akers fitted new solar arrays, after one of the old ones would not retract and was let go. On 7 December Hoffman and Musgrave took out the Wide Field/Planetary Camera and slid in its replacement, with the corrective optics inside, and fitted two magnetometers. On 8 December Thornton and Akers removed the High Speed Photometer and put COSTAR in its place, and installed a co-processor for the telescope’s computer; “I think everybody’s going to breathe a sigh of relief,” Akers said as the 640-pound unit went in. On 9 December Hoffman and Musgrave replaced the solar array drive electronics, fitted a redundancy kit to a spectrograph and covered the old magnetometers. The five walks took 35 hours and 28 minutes. Endeavour boosted the telescope to a higher orbit, released it, and landed at Kennedy on 13 December.

13 January 1994

Five weeks later, in a room at Goddard crowded with reporters, NASA released the first pictures. The telescope had been expected to take some thirteen weeks to reach full performance, but the difference was plain enough to show early. The chosen object was the spiral galaxy M100 in Virgo: a frame from the old camera taken on 27 November, days before the mission, in which only the brightest objects could be seen and the faint stars needed to measure the galaxy’s distance were lost to the aberration, beside a frame from the new camera, raw and unprocessed, in which they stood out; and, for comparison, the old frame sharpened by computer, which recovered some edges but no faint stars, and the same field from the 200-inch telescope on Palomar on a good night. Barbara Mikulski, the senator for Maryland who had fought for the telescope’s money, held up two pictures of a star, before and after, and declared, in the words NASA’s own history gives her, that the trouble with Hubble was over. Weiler said the telescope had been fixed beyond the project’s wildest expectations. In Paris the same day the European Space Agency called the correction of its Faint Object Camera “a total success”.

Afterwards

COSTAR did its work for fifteen years. In May 2009, on the last servicing mission, it came out to make room for the Cosmic Origins Spectrograph, the last of the original instruments having been replaced by ones with their own corrections, and it went to the National Air and Space Museum. Kodak’s backup mirror had gone there in 2001, never coated, transferred from storage at Danbury when NASA decided it was worth more on display than in its crate. The telescope has made more than a million and a half observations. When the James Webb Space Telescope’s mirrors were assembled in 2017, NASA took the whole optical telescope to the vacuum chamber at Houston built for the Apollo spacecraft and tested it as a system at its working temperature, with an interferometer that could see the entire primary; the test team said the campaign applied the lessons learned on Hubble, which could be repaired, where Webb, a million miles away, could not. Lew Allen died in January 2010, at 84. The test Gore called a dress rehearsal had become, for the next great telescope, the thing nobody argued about.

Sources

• United States Senate, Committee on Commerce, Science, and Transportation, Subcommittee on Science, Technology, and Space, Hubble Space Telescope and the Space Shuttle Problems, hearing of 10 July 1990, S. Hrg. 101-1087: every quotation of Gore, Pressler, Thompson, Fisk, Rodney and Rich, Rich’s letter of 3 August 1990, and Rodney’s answers for the record. govinfo.gov
• Lew Allen and others, The Hubble Space Telescope Optical Systems Failure Report, NASA-TM-103443, November 1990: Chapter VI on the checkout and the onboard data, and Appendix B for the Independent Optical Review Panel’s nine findings and signatories. ntrs.nasa.gov
• Rebecca Kolberg, “NASA moves to speed up Hubble repair,” UPI, 23 July 1990: Weiler’s words, the nickel, the three to six months and the doughnut mask. upi.com
• UPI, “Hughes, Perkin-Elmer settle Hubble Space Telescope lawsuit,” Danbury, 4 October 1993: the terms and the chairman’s statement. upi.com
• Irwin Goodwin, “Hubble Space Telescope makers pay for costly mirror mistake,” Physics Today, November 1993, page 80: the Justice Department’s statement, Perkin-Elmer’s denials, the 440 million dollars, Frankle’s “fair and reasonable” and Armstrong’s statement. physicstoday
• Richard M. Jones, “Science Committee Hearing on Hubble Space Telescope Repair Mission,” FYI, American Institute of Physics, 23 November 1993: the hearing of 16 November, the 87 million dollars, Colvin’s testimony and the Stafford team. aip.org
• NASA, “STS-61,” mission page: the launch, the crew, the five spacewalks and the landing; and Irene Brown, “Hubble telescope outfitted with corrective mirrors,” UPI, Houston, 8 December 1993: COSTAR’s cost and Akers’s words. nasa.gov, upi.com
• NASA and the Space Telescope Science Institute, “Galaxy M100 resolution comparison,” January 1994, with its caption; NASA’s Hubble history pages on the first servicing mission and the mirror flaw for the press conference of 13 January 1994, Mikulski’s and Weiler’s words, COSTAR’s removal in 2009 and the museum; and the European Space Agency’s release 3-1994 of 13 January 1994. science.nasa.gov, mirror flaw, esa.int
• National Air and Space Museum, “Mirror, Primary Backup, Hubble Space Telescope,” object A20010288000. airandspace.si.edu
• Stephen Clark, “James Webb Space Telescope completes critical end-to-end test,” Spaceflight Now, 15 January 2018. spaceflightnow.com
• Jet Propulsion Laboratory, “JPL mourns passing of former director Lew Allen Jr.,” January 2010. jpl.nasa.gov