Fred Hoyle's best ideas lit up the scientific world, but his worst meant devastating humiliation for astronomy's brightest star, says Simon Mitton
At the end of the Second World War, British astronomy was in a pitiful state. The country's astronomers returned to their universities and observatories to a demoralising reception. Lecturers had little time for research as they struggled to cope with rooms packed with students.
Observational astronomers fiddled with outdated instruments in a foggy climate, while their US counterparts trained the world's largest telescopes at California's cloudless skies. And in the whole of the UK, there were barely a dozen theoreticians to work out what all the observations might actually mean.
Theoretical astronomy, in which Britain had once led the world, was a shadow of its former self. But out of the gloom strode an unlikely hero, Fred Hoyle, a blunt Yorkshireman with a distaste for the establishment. His creativity dazzled his demoralised colleagues and he became the beacon for the rebirth of British cosmology, the effort to understand the nature of our universe.
Yet the story of Hoyle's career is not just one of triumph and brilliance - in many ways it was a Greek tragedy. The conflicts that arose from his theories and unusual methodology often sparked high drama. At his best, his work should have won him a Nobel prize. His approach to science was original, incisive and remarkably visionary.
But the ideas he pressed with such passion to shed light on some of the biggest mysteries in science could on occasion be just plain wrong. And as damning evidence mounted about his mistakes, the chorus of critics, in part stirred up by his forthright, iconoclastic style, sounded ever more plangent. They would fall silent only at the conclusion of the saga, with his shock departure from Cambridge University in 1972. By then, though, he had made theoretical astrophysics flourish in Britain as never before.
Hoyle was ever the outsider. From the moment he arrived at Emmanuel College, Cambridge, in 1933, he was unwilling to accept what he saw as arbitrary authority and old-fashioned academic procedures. On his first day, he announced to his astonished tutor that he was switching from physics to mathematics; the college would not have admitted him had they known his true intention. But Hoyle had correctly foreseen that mathematics would equip him better than laboratory work for a career as a theorist.
At mathematics, the grammar school boy from Bingley displayed astonishing talent, even by the highest standards of Cambridge. He skipped the second year completely and graduated with the highest marks in his class. This achievement won him a position as a research student in nuclear physics, where his supervisors would include Paul Dirac, the quantum physicist who won the Nobel prize in 1933.
War interrupted Hoyle's studies but did not staunch his creativity.
Cambridge was drained of scientists and mathematicians, who used their talents in the service of their country. Hoyle worked on naval radar despite his limited knowledge of electronics. But well aware that he was not capable of fighting Hitler with a soldering iron, he instead set about developing the theory that underpinned this crucial technology. His brilliance would soon make its mark.
There was a fatal flaw in the defensive radar used by the Royal Navy. While it could measure the distance of an incoming enemy aircraft, it was unable to determine the height. That made it impossible to direct defending aircraft to the correct position quick enough for a counterattack. There was no time to modify the equipment, so Hoyle set about finding another way to tease out the vital information from the signals.
He approached the matter as though it was another problem in theoretical physics, such as understanding radioactivity. It did not take the young physicist long to realise that measuring the changes in the strength of the radar beam as an aircraft closed in could betray its height. From then on radar operators were issued with a set of graphs designed by Hoyle from which they could gauge altitude. It was a simple solution and one that proved significant for naval defence by reducing the effectiveness of air attacks. Hoyle never received due public recognition for this achievement.
The experience was valuable, though, not least because radar had introduced him to two Austrian emigres who were to become longtime collaborators in his exploration of the cosmos. Tommy Gold and Hermann Bondi had both been interned by the British authorities in Canada, but were then brought back to Cambridge. There, Hoyle recruited them for his theory group at the Admiralty Signal Establishment.
The three men clicked. In Hampshire, they shared a modest cottage in which they would spend evenings solving problems in astrophysics and cosmology.
Hoyle would typically set the pace on which puzzle to try next.
They returned to Cambridge in 1945. Hoyle, now a junior lecturer, threw himself straight into one of the great mysteries of physics - the origin of the elements. Astronomers knew that hydrogen and helium were the main components in the chemical make-up of normal stars - just 2 per cent of the mass of the Sun is composed of heavier elements. But it was uncertain just how that 2 per cent - composed of all the other elements from lithium to uranium - was produced.
The few astrophysicists who had pondered their source believed they must be byproducts of an explosive start to the universe. This intrigued Hoyle, who always felt that the secrets of nuclear alchemy lay inside the stars rather than being primordially ordained.
Towards the end of the war, during an official visit to the US to share radar secrets with the Americans, he made a trip to meet the influential astronomer Walter Baade. Baade suggested that Hoyle work on supernova explosions: "Maybe a star is a nuclear weapon," was how he put it.
Emboldened by this thought, Hoyle set out to investigate the nuclear reactions that might take place at the very high temperatures found inside massive stars. Drawing on data that had just been publicly released by the atomic weapons programmes, he worked alone at St John's College, Cambridge.
A single page of his notebook from 1945 captures the moment Hoyle cracked the problem. In this hitherto unrecognised volume, the young scientist reveals the secret for cooking up elements at high temperatures.
He shows how nuclear reactions inside very hot stars can turn carbon into oxygen, silicon and so on.
His theory neatly matched reality, as its predicted distribution of the elements corresponded well with their abundance in the natural environment.
But there was little interest from the scientific community when Hoyle published his findings in 1946.
In applying nuclear physics to the stars, he was far ahead of his time. It would be another eight years before people took notice. But when they did, the idea was rapidly accepted. With a handful of other scientists working in the US - the experimentalist Willy Fowler, the astronomer Margaret Burbidge and her husband, the theorist Geoffrey Burbidge - Hoyle had demonstrated that all the heavy elements were blasted out in stellar explosions. By 1957, the origin and distribution of the entire periodic table of chemical elements could be accounted for. Hoyle showed that we are all the children of stardust.
In explaining the origin of the elements, he taught British astronomers another lesson. His frequent visits to the California Institute of Technology had proved a great success. And through them, he showed his UK colleagues that they must stop treating Americans as dangerous competitors and start meeting them on equal terms as collaborators.
But it was back in England, as part of the Cambridge trinity with his wartime friends Bondi and Gold, that Hoyle hatched his most outrageous idea. In 1946, Gold had speculated that the universe might have no beginning and no end. Initially Hoyle dismissed this as crazy. But as he toyed with the concept, he became attracted by its merits. The three started to flesh out its properties, not least of which was the implication that new matter would have to appear spontaneously to fill the voids left as the universe expanded.
Bondi and Gold described this cosmology as "steady state" because the resulting universe would appear the same no matter when it was observed.
Hoyle described it as a "continuous creation" arrangement, a phrase that would cause the outspoken rationalist trouble with churchmen. But by this time, the rest of the scientific community was being strongly drawn to the alternative explosive origin for the universe.
Hoyle's star continued to rise. Early in 1947, a BBC producer invited him to give a 20-minute lecture on continuous creation for the Third Programme . He spoke confidently, using simple analogies to help his listeners picture his ideas. In the course of the programme, he likened the exploding universe - a model he rejected - to a "big bang". The words stuck.
Further radio lectures followed and they were extraordinary for their time. With his gritty Yorkshire manner, warm self-confidence, picturesque language and the grandeur of the cosmos as his subject, he captured a radio audience in Britain as never before. His fireside chats brushed aside the rather donnish tradition of BBC lectures.
These radio talks were the prelude to a brilliant career as a writer of popular science and science fiction, through which he drew many students into astronomy. His first novel, The Black Cloud , tells the story of a cloud of interstellar gas that plots an unwelcome course through the solar system, calculated to cause enormous damage to our planet. Hoyle describes the reactions of incompetent politicians and able scientists so convincingly that the reader is seduced into thinking these events really could happen. It became a bestseller.
Broadcasting and literary success brought Hoyle world fame and welcome income. The public had gained an appetite for his thoughts. But some of his peers treated him as a bumptious upstart. They resented his cocky manner and rejected his outlandish ideas.
All eagerly awaited a way to test the two rival theories of cosmology: steady state versus big bang. The stakes were high. Despite all of Hoyle's other achievements, his reputation rested on the answer to this most fundamental of questions.
The man who was to take on the task was Martin Ryle, the great Cambridge pioneer of radio astronomy. The two men might have shared a university and a hunger for knowledge but otherwise they could not have been more different. Ryle was born into a distinguished academic family and had enjoyed a privileged private education. He was a hands-on physicist, and had no great liking for the outspoken theorist. And he regarded continuous creation as sheer speculation. In this patrician man, Hoyle had found a worthy adversary.
To learn what kind of a universe we inhabit, Ryle designed a radio telescope to survey sources of radio waves far beyond our Milky Way galaxy.
He correctly realised that the past history of the universe is written in how these sources are distributed in space. If Hoyle was right and there was a state of continuous creation, the radio sources would be similarly spaced out no matter how distant. But if he was wrong, if the universe had emerged from a big bang, then radio sources should be more common at great distances than they are nearby.
The new telescope set the stage for a remarkable conflict. Ryle chose the occasion of his 1955 Halley Lecture at Oxford to announce his conclusions.
To thunderous applause, he told his distinguished audience that the results of his survey could not be explained by a steady-state universe. The matter was resolved - Hoyle was wrong.
Repulsed by the manner in which he had been flatly denounced and with no means of independently checking Ryle's data, Hoyle was not ready to give in. Then, to his delight, rival groups of radio astronomers reported that they could not reproduce the Cambridge results. This first clash had a damaging consequence for Ryle: Hoyle, by now the world's most famous theorist, had grounds to refuse to believe the results of Ryle's next two radio source surveys. Nevertheless, Ryle's group continued its public attacks on the steady-state theory. A great deal of blood was spilt at the monthly meetings of the Royal Astronomical Society.
Finally, in 1961, on his fourth attempt, Ryle produced damning evidence that the universe had evolved from a big bang. This spelt the death of the steady-state theory. Unveiling his result in a stage-managed press conference, Ryle somewhat unfairly invited his rival to defend his pet theory in the light of data that he had never seen before.
Hoyle was dealt a shocking humiliation in front of the world's media. He stormed out without responding. Within hours, newspapers proclaimed Ryle's triumph, the tabloids alluding to the biblical account of Creation, much to Hoyle's annoyance.
Despite backing a doomed theory, Hoyle's position in British theoretical astrophysics remained supreme. His output of papers was absolutely staggering, and a succession of brilliant research students joined him in Cambridge to work in stellar evolution and cosmology.
Hoyle positioned them at the frontiers of research, encouraging them to take credit for their discoveries. They were well equipped, too. From 1959, his group started to use digital computers to model the life of a star. He quickly grasped how important such machines would be for theoretical astronomy, and he secured access to what was then the most powerful digital computer in the world.
Theory was the platform on which Hoyle transformed UK astronomy, initially through his own work on the lives of stars, but subsequently by founding a major research centre. In 1967, his Institute of Theoretical Astronomy opened in Cambridge. From this new base, he initiated a sweeping research programme. During the summer months, visitors from the US flocked to Hoyle's institute, attracted by both the excellent computing facilities and world-class collaborators. It was a far cry from those bleak postwar days.
But it was not to last. Hoyle's Cambridge career came to an abrupt end in 1972. Although he gathered his own brilliant team within the institute, he had typically failed to make the right connections with other professors in physics and mathematics. Some were no doubt resentful of the resources being poured into theoretical astronomy.
Hoyle was aware of the ill feeling some directed his way and became increasingly paranoid that forces were conspiring against him.
Astonishingly, he misinterpreted a series of management decisions by the university as an attempt to push him out. Having jumped to a dubious conclusion about envious colleagues, he decided to deprive his supposed foes of their moment of victory by resigning before they could sideline him.
That he was wrong about this was a bitter irony. Hoyle's resignation was seismic: British astronomy had lost its brightest star. Cut adrift from the Cambridge hothouse, Hoyle strayed into academic territory for which he was ill-equipped. His speculations that diseases are seeded by infections from comets added nothing to a reputation that had begun to decline from the mid-1970s. He attacked the Natural History Museum for showcasing a fossil (archaeopteryx) that he considered to be a fake. This spat unnecessarily harmed the museum's standing for a while, and lost Hoyle even more support.
On August 20, 2001, Hoyle died aged 86. Despite his mistakes, his legacy is still very much with us. He persuaded everyone to regard the US as a source of friendly colleagues. He led the way in popularising science, found new ways of encouraging young talent and pushed endlessly for better observational facilities. And through the strength of his vision and through his own example, he pulled British astronomy out of its demoralising doldrums.
Simon Mitton is a fellow of St Edmund's College, Cambridge, who studied under Ryle and worked with Hoyle. His book Fred Hoyle: A Life in Science is published by Aurum Press this week, £16.95.