protecting the bits: preserving your digital records · protecting the bits: preserving your...

Protecting the Bits Preserving your digital records

Mark Myers Senior Electronic Records Specialist

Texas State Library and Archives Commission

1

Texas Digital Archive (TDA)

bull Created in 2015bull Repository for

electronic archival state records held by TSLAC

bull Preserves and provides access to electronic records

2

3

Currently the TDA hasbull ~ 83 million recordsbull lt 60 TBbull Born-digital amp

Digitized materialbull Records from all

branches of TX state government

bull Wide variety of file typesbull Textbull Photosbull Audiobull Video amp Filmbull Databasesbull Artifacts (kind of)

httpswwwtsltexasgovtexasdigitalarchive

TSLACrsquos Mission

bull Preserve the record of government for public scrutiny

bull Secure and make accessible historically significant records and other valuable resources both for print and electronic documents

4

ProtectingElectronic State Records

Using ldquoprotectingrdquo to meanbull Securing bull Preservingbull Making them accessible

5

bull Unlike some paper records digital records do not survive without constant attention

bull Biggest challenge facing electronic records - CHANGE

Preservation Challenges

Technical obsolescence

Hardware software media

Media DeteriorationMagneticOpticalSSDflash

Technological DependencyHardware software etc

20

When Electronic Records Are at Risk

bull Software is obsolete or getting to end of life

bull Dependency on specific hardware or system

bull Non-standardspecialty formats used

bull Systems being replaced

bull Records no longer in use

bull Records are on removable media

bull Starts with Records Managementndash Records Retention Schedules

bull Identify what records you have and what purpose they serve

bull Establish appropriate retention periods to manage volumendash Dispose of records at the right timendash Protect records that need to be retained

ndash Recordkeeping systemsbull Organize and Categorize Recordsbull Centralized controlbull Managed Access and storage

ndash Policies and Proceduresbull People know how to use the systembull Train people on the culture of management

Get control over your records

Securing records in the TDA

134

Step one ndash getting the records

10

What do we take

bull The same kind of records that you transfer in paperndash Records with an ldquoArdquo designation on the schedulendash Records with an ldquoRrdquo designation (need to be reviewed by an archivist

prior to transfer)

bull After active administrative use endsbull ldquoOriginalrdquo format ndash especially with images audio video

ndash Or the ldquoofficialrdquo or ldquopublishedrdquo for text-based records

What do we takebull Can take records in any format

howeverndash Proprietary formats that require

specialexpensive licensendash Encrypted records ndash other forms

of DRMndash Older ldquolegacyrdquo formats ndash Legacy media

bull We need to talk first

13

Step two - build the foundation

14

bull TDA is built in the Amazon Government Cloudndash Amazon S3 online

storagendash Glacier near line storage

bull Encryption in transit and at rest

bull Rolling integrity checks

15

Select security tags for files

3 tagsbull Publicbull Openbull Closed

Can be set at multiple levels

Add checksums to the files

16

Integrity = Protecting the bits

Example of Bit Rot

Preserving electronic records

17

Texas Digital Archive (TDA)Functional Requirements

bull Ingest electronic records from state agencies in any format

bull Characterize validate and track digital formats over time

bull Monitor integrity and security of digital filesbull Migrate to sustainable formats for preservationbull Collect and store all appropriate metadata about the

recordsbull Allow for proper access to records that meets the

needs of our usersndash Provide records in access formats based on user needs

bull Follow appropriate archival standards for preservation

18

Long-Term PreservationSustainable Formats

Formats expected not to change or change slowly

bull Widely supported amp usedbull Easily Transferable

ndash Compatible with other applicationsndash ForwardBackward Compatibilityndash Version Control

bull Non-proprietary or openpublished standards

Types of Standards- StateEnterprise- NationalInternational Standards (ANSI ISO)

- Industry Standards

- ldquoDe factordquo Standards

21

Ingest Workflow ndash how we bring a file into the repository

bull Runs a series of ldquomicro-servicesrdquo on the files

bull Most important for long-term preservation being the ldquoCharacterizerdquo step

22

Characterize step indentifies the file format

Plugged in to an international registry of file formats

23

24

25

Migration example

These are MS Word documents marked as the Original

System allows us to normalize to a variety of file formats

26

Migration example

After migration now have a ldquoPresentationrdquo format in PDFPresentation format is the one that gets pushed out to the public

Redaction of an Outlook calendar entry in a PST file

Redaction of an email

27SAA 2015

28

We also watermark images and films for extra protection The watermarked image is the presentation copy

Providing Public Access to electronic records

29

Access = Preservation

Public Access Portal

bull Preservica Universal Accessndash Free ndash provided as part of preservation system ndash WordPress front-end ndash Linked directly to back-end Preservica repositoryndash Can control access through security settingsndash Displaysearchndash Customizable look and feelndash Browse read search

Searchbull Searches everything

bull Metadata (seenunseen)

bull Full-text searching of text files

bull Facets and filters let you narrow the search

Content viewers

Photos Video

Audio

Benefits of transferring records to the TDA

bull Relieves the preservation burden from the agency

bull Saving taxpayer money ndash Reducing agency storage and migration costs

bull Reducing agency staff time for PIA requestsbull Increases transparency by making more

records accessiblebull Preserves and protects electronic state

records a secure trusted repository

34

Want to transfer electronic records to the TDA

bull First step ndash Contact usbull Complete our electronic

records surveybull Identify the series to

transferbull Negotiate the timing and

method transfer

35

36

-If you have worked with an appraisal archivist you can contact them as well

- Laura Saegert ndash Assistant Director for Archives

- lsaegerttsltexasgov- 512-463-5500

-Mark Myers ndash Senior Electronic Records Specialist

- mmyerstsltexasgov- 512-463-5434

Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions


bull Created in 2015bull Repository for

electronic archival state records held by TSLAC

bull Preserves and provides access to electronic records

2

3






TSLACrsquos Mission



4



5








20















134


10

What do we take


prior to transfer)








13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

3






TSLACrsquos Mission



4



5








20















134


10

What do we take


prior to transfer)








13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

TSLACrsquos Mission



4



5








20















134


10

What do we take


prior to transfer)








13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions



5








20















134


10

What do we take


prior to transfer)








13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions








20















134


10

What do we take


prior to transfer)








13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions















134


10

What do we take


prior to transfer)








13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions


10

What do we take


prior to transfer)








13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

What do we take


prior to transfer)








13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions






13


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions


14





15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

15





16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

16


Example of Bit Rot


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions


17








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions








18









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions









21




22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

22



23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

23

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

24

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

25

Migration example



26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

26

Migration example




27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions



27SAA 2015

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

28



29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions


29








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions








Content viewers

Photos Video

Audio







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions







34





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions





method transfer

35

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

36






Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

Questions

37



Slide Number 3

TSLACrsquos Mission







What do we take

What do we take

Slide Number 13


Slide Number 15

Slide Number 16




Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28




Slide Number 32

Content viewers



Slide Number 36

Questions

protecting the bits: preserving your digital records · protecting the bits: preserving your...

Documents