Research intelligence - Domesday scenario: denied

A US/UK scheme aims to give digital archivists the tools they need to keep up with technology. Paul Jump reports

October 28, 2010

In 1986, the BBC marked the 900th anniversary of William the Conqueror's famous land survey with a Domesday Project of its own.

But by 2000, the videodiscs containing the information gathered - submitted by researchers and schoolchildren - had become so antiquated that a huge investment of time and effort was needed to read them.

Simon Wilson, a digital archivist at the University of Hull, cites the story as an example of how easily digital archives can be overtaken by advances in technology.

"With paper records, as long as the material is kept dry and in a controlled environment, we know we can return to it years later without any problems," he said. "But with digital material, archivists have to tackle the issues surrounding software, hardware and media going out of date within a few years."

To do so, Hull has joined the universities of Stanford and Yale in a project funded by the Andrew W. Mellon Foundation and led by the University of Virginia Library. It will develop a framework for archiving "born-digital" content, such as word-processed documents, web pages and images.

Mr Wilson admitted this was not the first project to address the issue. However, previous approaches, spearheaded in the UK by the British Library and the National Archives and in the US by the National Science Foundation and the Library of Congress, had been geared to specific institutional needs. As a result, they were not necessarily suitable or affordable for the academy.

The born-digital project, by contrast, would see the four "very different" institutions forge a common approach that any university could adopt, built around open-source file formats not reliant on particular software to be read.

He stressed that archiving was about more than just putting files on servers, and said the challenge would be working out how to adapt curation techniques developed over decades for paper archives - while dealing with the "scary" volume of digital content that already exists.

A fragile sense of history

Another challenge, Mr Wilson said, was to preserve the "sense of history" of born-digital documents in an era in which hardware and software were constantly evolving.

"In three years, we may have to convert all files into a new format, but we will need to make sure we aren't making too drastic a change that will alter the historical interpretation, such as altering the appearance of a blog or web page," he said.

"We will need to keep an audit trail of changes and take care with the metadata, such as information about when the document was last viewed."

Adam Farquhar, head of digital library technology at the British Library, said another challenge for archivists was dealing with scientific datasets, which often amounted to unexplained columns of figures.

"If they are measurements, what instruments were used? Who took the readings? It is often not recorded very well," he said. Things would change only when datasets were accorded the same status as published articles and their creators received similar credit when cited, he added.

An American by birth, Mr Farquhar said some of the "very best work" in digital archiving had been carried out in Britain, due in part to its "cultural awareness of the value of historical material".

He also welcomed the Joint Information Systems Committee's (Jisc) recent decision to join the Open Planets Foundation - a group of major European research institutions, technology companies, national libraries and archives interested in digital archiving.

Neil Grindley, programme manager for digital preservation and records management at Jisc, said he hoped the move would accelerate development - partly through greater sharing of the problems institutions have faced.

He said that although digital preservation remained an "emerging field", Jisc was keen to encourage universities to invest in it and engage with pressing issues such as e-journal archiving. Although archives' full value took decades to be realised, they could also serve an important short-term role as a backup for universities' digital treasures.

As for encouraging academics and others to submit material to digital archives, Mr Wilson said the key would be to stress that if they wanted to avoid the BBC's Domesday scenario, they should put content in an archive "sooner rather than later". But he admitted that he feared lots of material had already been irretrievably lost.

"If you ask people whether they have computer files from 20 years ago, they look at you as if you were stupid," he said. "We are having to make up lost ground and a black hole may appear when people ask for stuff from 10 years ago and it is not there."

Please Login or Register to read this article.

Register to continue

Get a month's unlimited access to THE content online. Just register and complete your career summary.

Registration is free and only takes a moment. Once registered you can read a total of 3 articles each month, plus:

  • Sign up for the editor's highlights
  • Receive World University Rankings news first
  • Get job alerts, shortlist jobs and save job searches
  • Participate in reader discussions and post comments
Please Login or Register to read this article.