“Maintaining the Long-term Reliability of Digital Information”

IIMDr. Jason Kaminski

Digital Dark Ages?

‘We seem at times, to be living in what Umberto Eco has called an “epoch of forgetting.” Within this hyperbolic environment of technology euphoria, there is a constant, albeit weaker, call among information professionals for a more sustained thinking about the impacts of the new technologies on society.’

(Terry Kuny, 1997)

Digital Doomsday?

BBC's 1986 Domesday Project

An electronic version of William I's Domesday Book, originally published in 1085.The project cost GBP2.5 million.17 years after its creation the Domesday Book ‘Mark II’ could not be read as the technology used for the project became obsolete.The original version can still be read in the Public Record office, 900 years after its creation.

Information & ‘Deep Time’

“Can we convey the general emotional message in some direct way, independent of language? Suppose we erect some aerodynamically streamlined monoliths with gaps between them. These resonate in the wind, sending forth a hollow, mournful note. Most likely, such wailing rocks could establish a legend about the site that transcends language. There’s the rub-getting through to cultures and languages we cannot anticipate. So how can we expect them to share our (often unspoken) assumptions, and thus read our warnings? Generally, we can’t. But there are ways of shaping a message so that it has some plausible chance of sailing intact across the great ocean of deep time”.

Technical Obsolescence

Digital World

• 93% Of All Information Produced Each Year Is Digital

• A project at the University of California aimed to quantifythe amount of information published in the world each year.

• They estimated that 93% of the information produced each year is stored in digital form.

• Hard drives in stand-alone PCs accounted for 55% of total digital storage shipped each year (1999 figures).

What is e-Permanence?

The aim of ‘electronic permanence’ or ‘digital preservation’ is to enable long-term usability, accessibility and reliability for digital information based on sound digital information management and archiving practices, as well as specialised preservation techniques.

The ‘Business Case’ for Digital Preservation

• Protection of the commercial value of information ‘assets’ (business intelligence/knowledge)

• Protection of competitive advantage derived from ownership of information;

• Avoidance of duplicated physical and intellectual effort;

• Avoidance of risk where loss of evidence may expose an entity to liability;

• Compliance with legislation requiring long-term retention of certain types of records; and

• Maintenance of a long-term business history as a source of strategic business intelligence.

Is Digital Obsolescence Inevitable?

No. Increasingly, Governments and private organisations that own and manage information with a high degree of intrinsic long-term value are turning to specialised techniques in the fight against digital obsolescence.

– Mining– Science and research – e.g. pharmaceuticals, medical– Government – e.g. Australian Bureau of Statistics,

PROV, NAA– Construction – e.g. infrastructure planning and

development– Cultural – e.g. manuscripts, images

Primary Digital Preservation Techniques

Migration Transfer of digital content to current and/or open-source digital formats to reduce the complexity of strategic migration of digital information between applications, systems and media

EncapsulationAssociating digital information with detailed metadata records in order to record the complete technical and cultural/business context of digital information to support decipherment.

Emulation Computer ‘emulators’ translate original software instructions so that they can be executed on new platforms; obsolete software can then run on new technical platforms ‘in emulation,’ eliminating the need to retain examples of obsolete hardware and software.

Digital Preservation & Information Management

Elements of migration and metadata encapsulation techniques, coupled with good strategic governance of corporate information systems, as well as the use of appropriate technologies for data storage and retrieval, will achieve the primary aims of permanent preservation of digital information.

Digital Preservation Standards

There are two main facets of digital preservation that require control according to industry standards:

• Archival media and formats; and

• The archival ‘process,’ including systems and metadata

architecture, and business process models.

Media and Formats

• Digital Media & Microfilm ISO Standards• A comprehensive methodology for evaluating the

stability and longevity of formats using a range of criteria, called ‘INFORM,’ has been developed by the Online Computer Library Centre in Dublin, Ohio.

• Formats and media standards for specific content types (including audiovisual, sound, image, text, data, spreadsheets, and magnetic formats and media) issued by the ‘Preserving Access to Digital Information’ working group of the National Archives of Australia.

Preservation Process: OAIS

Digital Preservation & Legal Compliance

• Public and Corporate Record-keeping;

• Privacy;

• Copyright;

• Evidential Integrity; and

• Inter-jurisdictional data-transfer.

Digital Evidence

The Australian Commonwealth Electronic Transactions Act, 1999 requires that:

• In all cases—having regard to all the relevant circumstances at the time of the communication, the method of generating the electronic form of the document [must provide] a reliable means of assuring the maintenance of the integrity of the information contained in the document; and

• In all cases—at the time the communication was sent, it [must be] reasonable to expect that the information contained in the electronic form of the document would be readily accessible so as to be useable for subsequent reference.

Digital Preservation Technology Types

Computer Output Microform

Analogue-Digital-Analogue – physical medium with very long life (100-500 years) – human readable – no dynamic functionality

Digital Archives

‘Born Digital’ – high quality components – centralised management (migration, encapsulation) – dynamic/functional

Digital Preservation Technology Products

Digital Archives• IBM DIAS [i] • Hitachi’s Content Archive Platform [ii] • NetApp Snaplock [iii] • EMC Centera [iv] • Hewlett Packard RISS [v]

[i] Refer to http://www-5.ibm.com/nl/dias/[ii] Refer to

http://www.hds.com/press_room/press_releases/gl060612a.html[iii] Refer to

http://www.b2net.co.uk/netapp/network_appliance_snaplock.htm[iv] Refer to http://www.emc.com/products/systems/centera.jsp[v] Refer to

http://h18006.www1.hp.com/products/storageworks/riss/index.html

Digital Preservation Technology Products

Computer Output Microform• Fuji - AR-1000 [i] • Image Graphics - Micrographics Electron Beam Recorders [ii] • Kodak – Reference Archive Writer 9600 [iii] • Global Information Distribution (GID) COM system 6880; [iv]• Staude – DigiFiche [v]• Anacomp - XFP2000 and IDP1600 [vi]

[i] http://www.fujifilmusa.com/JSP/fuji/epartners/Products[ii] See http://www.igraph.com/EBRecorders/EBRs.htm[iii] See http://www.kodak.com/US/en/dpq/site/TKX/name/i9600_product[iv] See http://www.gid-it.com/us/html/produkt.php?rowname=com[v] See http://www.e-staude.com/[vi] See http://www.anacomp.com/mir/mirSystems.asp

Digital Preservation ‘Architecture’

Corporate Information Repositories & Data Structures

Electronic Records Management or Enterprise Content Management System

Integration

Export of encapsulated and standardised digital information objects to Open Archival Information System (OAIS)

Underlying digital and/or computer output microform digital archives technologies

Format standarisation and metadata encapsulation workflows

Permanent analogue records

Dynamic content

Metadata

Targeted Migration Strategy

Digital Preservation Case Studies

• GovernmentPublic records management: Long-term Governance – Heritage - Duty of Care – Public Assets Management - Continuity

• Private IndustryMining Sector:Private Asset Management - Business Intelligence – Long-term Risk Management – Long-term Business Continuity

Reference Information

• ‘The State of the Art and Practice in Digital Preservation’ (Lee, Slattery, Lu, Tang & McCrary, 2002) Journal of Research of the National Institute of Standards, http://nvl.nist.gov/pub/nistpubs/jres/107/1/j71lee.pdf

• ‘Overview of Technological Approaches to Digital Preservation and Challenges in Coming Years’ (Thibodeau, 2002), http://chnm.gmu.edu/digitalhistory/links/pdf/preserving/8_37e.pdf

• ‘Challenges to Digital Preservation and Building Digital Libraries’ (Ross, 2003), http://www.ifla.org/IV/ifla69/papers/209e-Ross.pdf

• ‘Is digital preservation an oxymoron?’ (Surface, Caplan, Horton & Halbert, 2005), http://portal.acm.org/citation.cfm?id=1065385.1065422

• ‘Thirteen Ways of Looking at...Digital Preservation’ (Lavoie & Dempsey, 2004), http://www.dlib.org/dlib/july04/lavoie/07lavoie.html

Further Discussion