national digital information infrastructure and preservation program (ndiipp)

National Digital Information Infrastructure and Preservation Program (NDIIPP)

CNI Project BriefingDecember 5, 2005

www.digitalpreservation.gov 2

NDIIPP Elements

Technical Architecture

 Research

Preservation Partners

www.digitalpreservation.gov 3

Lead Institutions for the 8 Partnerships

• California Digital Library: Web political content• Emory University: MetaArchive of Southern digital culture• UC Santa Barbara: NGDA Geospatial Data• NC State University: Geospatial Data• U of Maryland: Dot.com business records• U of Michigan: DataPASS--Social science data• U of Illinois, OCLC, State & academic libraries, NCSA: State

government publications, among other content• WNET/PBS: Digital television(red=represented in today’s panel)

www.digitalpreservation.gov 4

Goals for the Partnerships• Identify/select/collect content;

communicate strategies for doing so• Probe intellectual property issues• Collaborate broadly in developing a

shared technical architecture• Study economic sustainability• Identify and share best practices• Learn how to build and incrementally

improve a preservation network

www.digitalpreservation.gov 5

Network of Networks• LC pleased to see interconnections

between partners, other networks• Some examples:

– Association of Research Libraries– Digital Library Federation– Coalition for Networked Information – UK Joint Information Systems Committee

www.digitalpreservation.gov 6

Broad Categories of Challenges

• Technical

• Legal

• Social

• Economic

www.digitalpreservation.gov 7

MetaArchive Project Emory Univ. & partners

• Distributed preservation network of six institutions now deployed and functioning

• Conspectus database (based on UKOLN RSLP and DCC) of all collections archived in the network of nodes

• Now working on developing a long-term Cooperative to support our network

• Negotiating many issues about organizational roles and responsibilities in such a Cooperative

• Examining various organizational structures for the Cooperative, including unincorporated association for interim period of 1-2 years, and 501(c)(3) for subsequent long term; now drafting inter-institutional contracts

www.digitalpreservation.gov 8

North Carolina Geospatial DataArchiving ProjectProject Goals• Preservation of state and local geospatial data

resources• Engaging existing spatial data infrastructureEmerging Challenges• Rethinking content: not just data--also cartographic

representation and geospatial documents• Technical challenges: handling complex objects in a

repository; spatial databases• Engaging the industry: making the case for historic

and temporal data--gathering use cases

UCSB—Geospatial Data: NGDA Overviewhttp://www.ngda.org

www.digitalpreservation.gov 10

UCSB NGDA: FIRST YEAR PROGRESS AND ISSUES Designed NGDA architecture; combination of new and available:

Storage subsystem (Isilon, Archivas) Server to connect components and manage ingest Existing ADL as interface for access and federation Major challenge: Developing Geospatial Format Registry

Collections issues in archiving geospatial imagery: Developing collection policies and provider contracts Definition of “at-risk” digital information Identifying good collections to test/prototype system:

• UCSB: 3 TB CaSIL data; Stanford: .5 TB, Rumsey Collection

Major challenge: Multi-layered data environment and poorly documented formats

Team building challenges: People who understand mission and have technical skills Developing a common vocabulary

Year Two goals: Web-accessible prototype for demonstration Economic sustainability models Collections (MODIS, Landsat, Shapefiles, DOQQ)

www.digitalpreservation.gov 11

ECHODepository: Illinois, OCLC, State & Academic Libraries, NCSA• Year 1 Accomplishments:

– Selection rationale (Arizona Model)– OCLC Web Archiving Workbench Tools– Repository evaluation– Repository interoperability: Hub & spoke

model – Partnership building: NCSA, CDL, others

• Challenges:– Scalability & flexibility of software &

storage architectures– Import/export commonalities: METS

www.digitalpreservation.gov 12

CDL: Web at RiskA Distributed Approach to Preserving our Nation’s Heritage

• Project Goal: develop a web-archiving service to enable institutions to continue their historic collection, management, and preservation roles.

• Collection Challenges: web-based events (Katrina)– The nature of events on the web– Technical and social issues– Selection, collection & classification – Balancing rights and acquisition needs– Understanding information retrieval

www.digitalpreservation.gov 13

Michigan: Data Preservation Alliance for the Social Sciences (DataPASS)• Goals:

– Build Partnership to Identify and preserve at-risk digital social science content

– Successful creation of a database of content

• What have we learned? – Identification/selection daunting – Technology challenging – Aided by social science metadata

standards– Building a partnership very useful

www.digitalpreservation.gov 14

Emerging Issues

• Natural tension between collaboration and tradition of institution-specific approaches

• Much interest in joint infrastructure, but developing shared services will take time

• Existing preservation methods are being stretched

• Balancing a focus on project goals while also considering lots of interesting ideas

www.digitalpreservation.gov 15

Learning and Incremental Development

• We need to get broad feedback and continue learning

• Still no “silver bullet” solution to digital preservation

• NDIIPP considering all viable approaches, working toward gradual development of decentralized, interoperable architecture

• Our partners are modeling this approach

www.digitalpreservation.gov 16

Contact Information:• LOC: William Lefurgy• CDL: Patricia Cruse• Emory: Martin Halbert• Illinois: Beth Sandore • Michigan: Myron

Gutmann• NC State: Steven Morris• UCSB: Sarah Pritchard

wlef@loc.govPatricia.Cruse@ucop.edumhalber@emory.edusandore@uiuc.edugutmann@umich.edusteven_morris@ncsu.edupritchar@library.ucsb.edu