As the Sir Humphreys meet to recall the Thatcher era, Harriet Swain looks at the task facing official historians in sifting through memories and miles of archives
As distinguished former and serving civil servants assembled at Churchill College, Cambridge in November for a series of witness seminars, they found in their way a large covered box. The idea was that the contents of the box, unveiled at the opening session, would help them recall the 1980s and the administration they had come to discuss. Inside it was a handbag.
It is ironic that such an object could so successfully represent the period and place dominated by these cerebral, and overwhelmingly male, dealers in words. For these are the men who make history - at least the kind of history that for a long time was the only kind that was considered to matter. Here was former Cabinet Secretary Lord Armstrong, who travelled to Australia to try to prevent publication there of the MI6 memoir Spycatcher . Here was Lord Waldegrave, who served as a "wet" Cabinet Minister under Baroness Thatcher and Sir John Major. Over there was Lord Wilson, Cabinet Secretary and head of the home Civil Service in 1998, and there, being ushered into the final session, was the current head Sir Gus O'Donnell.
More than any other grouping, it is civil servants who fill the archives on which later historians depend. This is true of senior figures who earn knighthoods for their ability to formulate a policy, draft a document or issue a briefing note, and of penpushers diligently recording demands on a local health service. But they rarely form the subject of histories themselves.
This may be one reason why so many eminent figures turned up to take part in the seminar series, part of a forthcoming official history of the Civil Service commissioned by the Cabinet Office. "Usually, they don't have a public voice," says Rodney Lowe, the official history's author. "They see politicians writing various versions of things and they want to get in and get their version of the truth on record." Many were there for more altruistic reasons. For them, says Lowe, taking part in the official history programme is a continuation of a life spent serving the public.
Certainly, Lowe's history is designed to be of service, to both future historians and policymakers. He says it has two main functions: to reinforce collective memory and to aid independent academic research "by giving truth a quick start".
This was a phrase coined by Sir Keith Hancock who set up the Government's civil history programme in 1941. Before then, official histories, which began in 1908, had concentrated exclusively on "compiling the naval and military history of the nation". From the 1940s onwards, these histories covered aspects of the home front during the Second World War, such as civil defence and food. They later included intelligence, but it was not until 1966 that the range of histories was extended to cover periods of peace. Today, it is peacetime policy that dominates. Histories in preparation include studies of privatisation (a few years ago there was one on nationalisation), the development of North Sea oil and gas, and UK accession to the European communities.
Individual government departments also commission their own studies of the past, but these are on a more piecemeal basis. The aim of the Cabinet Office programme is to cover subjects of general interest that fall within the remit of more than one department and involve at least some closed papers. The idea is to provide a reliable secondary source for use by historians unable to access the primary material because of the 30-year embargo period before which papers relating to an administration are transferred en bloc to the National Archives, or because papers remain closed for reasons such as intelligence.
Ideally, the official historian - chosen as a respected expert in the field - will also interview the key personnel involved. For this reason, the choices of subject tend to obey what one seminar delegate described as the "Goldilocks" principle. Approach them too soon and the events will be too hot. Leave it too late and the trail will have gone cold, as will some of the bodies.
Deciding what to publish is a protracted process that starts with a call for ideas from all main government departments. Every ten years, an official Cabinet Committee on Official Histories, chaired by the Cabinet Secretary, meets to whittle down the resulting long list to seven or eight potential runners. These are ratified by a Privy Council group made up of Conservative, Labour and Liberal representatives - currently Lords Healey, Howe and Rodgers. If they are happy with the list, it goes for approval to the Prime Minister, who is also responsible for formally appointing the chosen historian.
The next official Cabinet Committee meeting is likely to take place early next year and Tessa Stirling, head of the histories, openness and records unit in the Cabinet Office, says she has started compiling a long list, which for the first time will invite suggestions from people from outside Government.
Until recently, she feared for the future of the programme, in spite of its minimal cost - £190,000 in 2003-04, the latest year for which figures are available. Now, she says, there appears to be a change in mood, a renewed feeling that history matters. A recent endorsement of the official history programme by the powerful Public Administration Select Committee has helped to put the programme on the map.
Present-day civil servants were in no doubt of the relevance of the November seminar series for themselves and their colleagues. Lowe's two volumes look at the history of the home Civil Service between 1966 and 1997, focusing on the planning and policy behind its reform in response to changing expectations from politicians and the public. The seminars tackled the 1981 Civil Service strike, which lasted five months and broke the consensus culture of industrial relations in the service, and the abolition of the Civil Service Department - the prototype of Yes Minister' s Department of Administrative Affairs - which followed. The final seminar, which resonated most clearly with today's officials, looked at the Next Steps initiative, which envisaged a small central core of civil servants supporting ministers and managing departments together with a new range of executive agencies, including privatised bodies. This initiative led to the highly successful government agency Companies House and the less successful Child Support Agency.
Lowe's task is not an easy one. Although less than 5 per cent of official records are permanently preserved, this still amounts to 2.4km of archive every year from which he will have to make an informed selection. He also has to negotiate a path of academic freedom within the boundaries that the job of official historian brings. This means complying with a synopsis given to him by Cabinet Office officials and attending a project board every year, which checks on his progress. The finished history goes through a clearance process that involves being checked by every relevant department "to make sure he hasn't got anything really, really inaccurate", Stirling says. "We aren't trying to knobble the historian." She says official historians cannot breach the Official Secrets Act and they are asked not to dwell on personal differences between individuals - "particularly civil servants who cannot answer back". Nor can they quote large chunks of Cabinet minutes. Otherwise, they are free to write what they want, on the understanding that Stirling is free not to publish it - not something that has ever happened. Lowe says he has never felt restricted.
History, and the truths it tries to expose, is slippery enough. Lowe says the advantage to an official historian of getting records early and being able to compare the recollections of key players with written records is that often the two are shown to be quite different. "Sometimes the people involved are deliberately editing things," he says. "But often they aren't. They've just forgotten."
This became clear during the seminars when witnesses appeared carrying political memoirs of the period to aid recall of the events in which they had featured. Others referred to what colleagues had written about the period rather than relying on their own memories. And, for many, the passing of time, together with the knowledge that it was all being recorded, seemed to have mellowed their approach to confrontations that had once been fierce. So here were unions and management describing surprisingly similar impressions of the past. There were officials who had once argued from diametrically opposed positions finding that their views were not so different after all.
Speaking afterwards, Lord Waldegrave said how important it was for official histories to exist and expressed confidence the exercise had got somewhere near the truth of what happened in the early Thatcher years. "Very carefully, we are reconstructing what seems to me just about right," he said.
Talk afterwards among the modern mandarins suggested that it had not been quite the full picture. They highlighted points that had been skimmed over and reflected that witnesses who went out of their way to be nice to each other were probably disguising major past differences.
Such nuances will not appear in the transcript of the event, to be published on the web this month after participants have approved their words. Lowe says that if the transcript does not reveal the whole truth it could provide the basis for more probing questions in later interviews. But in the end, the truest essence of that period, notwithstanding the hours of talk and mountains of paperwork it produced, could perhaps be contained in that handbag.