washington state digital archives presented by: adam jansen digital archivist washington state...

Washington StateDigital Archives

Presented by:

Adam Jansen

Digital Archivist

Washington State Archives

360-586-4893

ajansen@secstate.wa.gov

Public Records

As defined in RCW 40.14

ANY records that have been made by or received by any agency of the state of Washington in connection with the transaction of public business

Records Retention

Any destruction of official public records shall be pursuant to a schedule approved under RCW 40.14

Why?...

The foundation of democracy in America is government accountability to the

people

What the Digital Archives is not

• Not mass storage for active business applications & data

• Not remote back-up for state & local government networks & data

The Digital Archives will:

• Preserve electronic records with long-term legal, historical and/or fiscal significance

• Assure platform-neutral retrieval 50, 100, or more years from now

• Provide security back-up of certain permanent electronic legal records (courts, vital records, land records, etc.)

Business Need

• Comply with statutory & regulatory mandates

• Avoid loss of legal & historical records

• Manage risk (avoid litigation losses)

• Preserve rare paper records

• Centralize access to permanent electronic

records for government

• Improve access for citizens

8 Requirements for Preservation• Readable• Retrievable• Intelligible• Encapsulated• Reconstructible• Identifiable• Understandable• Authentic

* From Authentic Electronic Records by Charles Dollar

} Hardware

} File Format

} Content Management

File Formats

Digital Archives Multi-pronged approach:

• Maintain native format, wrapped

• Render XML formatted version, wrapped

• Acquire original hardware and software

Phased Implementation

• Phase I – SAN Roll-Out with OSOS & one local government record series as beta test

• Phase II – Four to 6 state & local government agencies

• Phase III – Agencies with electronic records of high archival value

• Phase IV – All remaining state & local government agencies as they are ready

Management Issues• Authenticity of record• Chain of custody• Metadata• File naming conventions• Corporate Culture• Start with e-mail, web page• Use existing retention schedules• Educate• Shift AWAY from desktops• Management Software is a must!

• DoD 5015.2-STD compliant system

• Wrap original file in native format

• Wrap XML copy

• Apply metadata & XML for indexing,

searching & retrieval

• Provide chain of custody & authenticity

‘Content Management’

‘Content Management’

• Microsoft Solution

• Custom Coded .Net front end

• SQL Server back end

• BizTalk translation utility

• SSH Tectia for secure transport

Ingestion Process• Microsoft BizTalk 2004

• Transforms, adds metadata based on business rules

• Creates ‘deep storage’ copy wrapping original file in XML, with Hash

• Creates ‘web’ version of original file

Deep Storage XML SchemaRecord Common

•Who

•What

•When

•Where

•Original File

•‘web’ file

•Security

•Fixity

Vital Records• Type

Birth• Date of

• Father, Mother

• Hospital

Front End

• Developed in VB.Net by Microsoft/EDS

• Web based solution

• Allows uploading of documents, searching and ordering of certified copies

www.digitalarchives.wa.gov

Web Design Wire Frame

Back End• SQL Server 2000

• Multiple databases, similar construction to XML schema– Record Common– Record series specific

• Security Roles– Locks at office, agency, global level– Record Series, Record Field or Record

Web Archiving• Custom Built Solution• Saves binary streams and BLOBs into SQL

Database• Multiple streams, Assist with Archiving• Allows predefining of internal fragments, levels,

maximum file size, secure authentication• Command line interface• Can be used to ‘spider’ email• Web Services allows current architecture for

retrieval

Admin Pages• Allows viewing of confidential information

• E-Transmittal process

• Viewing of open orders

Risk Management& Quality Assurance

• Strong executive sponsorship & involvement

• Regular weekly project team meetings

• Outside expert: GlassHouse Technologies

• Targeted recruitment & training of staff

• Proof of concept testing

• External Quality Assurance during procurement, installation & testing

• Phased implementation

• Competitive procurement process among high-quality, experienced prime vendors

Proof of Concept Tests

• 97% of legacy file formats tested were successfully converted to XML with no change in format, feel or function

• Top-four levels of OSOS web site (5,015 files & 250 MB) were spidered remotely in 8 minutes

• E-mail on Novell GroupWise & Microsoft Exchange servers was successfully archived onto a remote server

Seven-Year Growth Estimates

Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7

SAN 03.0TB 08.6TB 15.6TB 024.6TB 037.1TB 051.6TB 074.6TB

TapeLibrary 10.0TB 28.0TB 45.0TB 103.0TB 160.5TB 255.0TB 350.0TB

-------------- ----------- ----------- ----------- ----------- ----------- ----------- -----------

TOTAL 13.0TB 36.6TB 60.6TB 127.6TB 197.6TB 206.6TB 424.6TB

Summary

• The Digital Archives is essential to meet statutory & regulatory mandates to preserve & make accessible legally & historically significant electronic records.

• The Preferred Alternative is the best solution for a stable, cost-effective, long-term storage & retrieval system.

• The project has a high probability of success by using proven technologies & experienced vendors.

• Ready to proceed to the procurement phase

Digital Archives Eastern Washington University, Cheney, Washington

Adam JansenAdam JansenDigital ArchivistDigital Archivistajansen@secstate.wa.govajansen@secstate.wa.gov