CT Digital ArchiveBrown Bag

Friday Dec. 16, 2016

MaintenanceC TDA just completed a maintenance sprint which upgraded Islandora to “head”, Drupal to the latest 7.4x, and testing Amplidata as a potential storage solution to replace Dell’s EqualLogics.

Maintenance for this year consisted of….

• Four 2 week maintenance sprints: All Islandora and Drupal systems are brought to “head”, meaning any bug fixes or patches implemented after the latest release.

• Monday and Wednesday 3am -6am: Any patches or updates to servers are done in the early morning on Monday and/ or Wednesday.

• Sunday morning: All systems are restarted to flush out temporary files and ensure the system is working normally. This occurs around 5am.

• Any Highly critical updates and/ or patches can be applied immediately.

• 2 week maintenance work involves approximately 100 hours minimally.

A Year of G rowhtThe C TDA has seen tremendous growth over the past year. The number of digital assets grew over 412,547. This represents an increase of 45.26% or an approximate growth rate of 3% per month. The number of records harvested to aggregate discovery services extends beyond 49,923 records or a 43.96% increase.

0

50000

100000

150000

200000

250000

300000

350000

400000

450000

Records Digital Assets

Content Growth in 2016

Jan-16 Nov-16

Happenings from C TDA G overnance

Advisory C ommittee:

• This committee met on 1-22-2016, 6-10-2016, 10-13-2016 and will meet on 12-22-2016.

• 2016 News:• C SL received a supplemental award for their newspapers project• C TDA was a sponsor of the C LHO 2016 conference• C TDA hosted the 1st User conference one day event• The C TDA Participants’ List was created as a means for participants

to communicate with each other• C HS received a grant to create a management system specific for

museum needs• Maritime C T contacted C TDA about using content form the repository

Technical Roadmap Discussion:• The Technical C ommittee was replaced by the Technical Roadmap

Discussion• Held every 4 th Monday of the month unless noted otherwise, the TRD

discussion infrastructure, storage, and C TDA technology

New in Education/ Training

C TDA created a YouTube C hannel where videos on how to add and manage content can be found. A video introducing C TDA in general is also there. The how-to documents grew on C TDA’s SlideShare space. 24 presentations and documents were added to Slideshare in 2016 and has one follower. Since Aug. 2016, 19 videos have been uploaded. There are 2 subscribers. Every video has been viewed at least once and the most viewed 45 times!

The 1st User C onference brought together 22 attendees from all over C onnecticut. 88.9% of attendees rated the event as above average. Attendees requested more lightning talks, hands-on workshops, and more time for focused discussions!

The in person Open Refine and MODS workshop brought over a dozen people from all over C onnecticut. This workshops was repeated for staff at C SL.

Features & Improvements in 2016

• The custom transformation for harvests was re- implemented for correct harvesting of C TDA records

• The information on the Fedora describe page was updated – used for systems and harvesting

• The code for the external batch ingest service was ported to dev, stage and production

• C ustom pages for “not found” and “access denied” were created for C TDA C ollections site

• Rolled out G IS cartographic tool, On this day, batch derivative trigger and ordered newspaper issue batch

• Redirected http:/ / ctdigitalarchive.org to http:/ / collections.ctdigitalarchive.org

• C reated video tutorial series uploaded and accessible on the new C TDA YouTube channel

• Added languages to OC R (C roatian, Hungarian, Lithuanian, Slovenian, Swedish, Danish, and polish)

• C ompleted content migration from old C TDA to new C TDA Drupal site

Features & Improvements in 2016

• C reation of new outreach and marketing materials including C TDA flier, logo, bookmarks

• C reation of a cron job that will restart system each week to address XMLUsersFile module error

• C reation of a cron job to flush out batch queue files and decision to disable batch queue.

• Moderately critical updates to XML sitemap, PHP vulnerabilities, ImageMagic

• C ritical patch for Islandora

• File level restore solution implemented

• Rolled out Simple Map and Islandora C ompound Parent Metadata Display

• Re- added ability to ingest current MS office file formats to the Document C ontent Model

• C reation and later approval of storage growth document

Features & Improvements in 2016

• C reation of new C SL manage site on the management server

• Fixed email delivery settings for all production sites

• Implementation of monitoring tools like Splunk and PRTG

• Administrative access granted to Fedora Admin C lient

• Newspapers moved to C TDA shared folder

• Applied G oogle security patch

• Rolled back to Jquery module to 7.x.3.0- alpha2

• C reation of an AWS test instance

• 4 Maintenance Sprints

• Work to create an external batch ingest service

• Final testing for DPLA harvests

CTDA & DocumentationC TDA has the following outlets for documentation:

• G itHub: This has the public repositories for metadata, outreach/marketing materials, uconn themes, governance agendas and notes, and our system maps.

• ITS G itHub: This has the private repositories of code written by our developers that contains information about our systems. For security reasons, this code is only accessible to ITS developers.

• YouTube: The CTDA channel has the video tutorials.

• SlideShare: This has our how -to documents, presentations, and metadata documents.

• C TDA Website: Documentation can be found on CTDA technology, namespaces, governance, and policies such as MOU or C ontact form.

• HelpSpot: The ticketing system keeps a record of all interactions which is used to generate reports at the end of the year.

• Other: Signed agreements are kept on UC L servers which are accessible only by C TDA staff (C TDA folder); documents that have any passwords or other compromising information are also kept in the C TDA folder; until recently the annual review document has also been kept in the C TDA folder; storage growth document is on Office365 and has been moved to the C TDA folder.

CTDA Development as Surveyed by Participants and Users

At the end of the calendar year, C TDA sends out a series of surveys to evaluate services. “Development Ideas for the C TDA” was one of those surveys. This survey’s goal was to seek feedback on how to improve tools, resources, or content display and presentation.

• 72.8% of participants found their experience of C TDA average to excellent.

• 75% of users surveyed found their experience average and 25% excellent.

• Both participants and users respondents found the resources such as the videos and how to documents to be average to excellent.

CTDA Services as Surveyed by Participants

“C TDA Services: Evaluation 2016” was our second end of year survey. This survey’s goal was for participants to rate C TDA services such as education/ training sessions, email lists, or ticketing system.

• All respondents found C TDA education and training sessions to be average to excellent.

• 87.5% of respondents rating the ticketing system average to excellent.

• All respondents rated the C TDA Newsletter average to excellent with 62 .5% rating it as above average.

• 50% rated the C TDA Alert list as above average and all respondents rated it from average to excellent.

CTDA G overnance as Surveyed by Participants

• 40% of respondents rated the C TDA Participants’ list as average and 40% as above average.

• 20% rated the Advisory C ommittee meetings as below average. No respondents rated these meetings as excellent and most rated it either above average or average.

• All respondents rated the Technical Roadmap discussions as above average.

CTDA Reporting as Surveyed by Participants

The last survey of the year sought to get participants’ needs and wants in regards to reporting. Here’s the list:

• G enerate report based on specific criteria such as creator’s name, collection name

• List of items ingested either per collection or per participant

• Number of objects by object file format type

• Amount of G B added each month

• Number of items added per month

• Find and replace based on report that can be exported as a spreadsheet, the metadata changed, and that report reloaded and records updated

• Reports accessible for participants when logged in

Participants’ Would like More…

• More in person training sessions

• Training sessions for beginner, intermediate, and advanced users

• Include more participants’ stories in Newsletters

• Include more discussion and input in the Advisory C ommittee

• Improve searching

• Improve stability and performance

• C reate batch ingest tools

• Display owning institution and rights information more visibly

• Reports

CTDA Statistics on Sites Uptimes

We began tracking site uptimes and loading times through PRTG since May 2016. A public dashboard was also created. The average uptime percentage is 98.572% and average page load is 529 msec.

0 200 400 600 800 1000 1200 1400

CTDA Collections Presentation

CTDA Collections Manage

UConn ASC Presentation

UConn ASC Manage

CSL Presentation

CSL Manage

Bibliomation Presentation and Manage

CHI Presentation

UConn Research Data

Channel Uptime Percentages & Average Page Load times

Average Loading Time (msec) Uptime Status (%)

CTDA Ticketing System Numbers

• C TDA Requests• The average number of requests per month is 68.5 with a total number of

requests of over 754.• The average time to respond to a ticket for the 1st time is 25.9 minutes.

Tickets Take Time To Resolve

• The average time to resolve a ticket is 67.6 hours.

• The majority of tickets take 11 to 20 minutes to resolve with those taking more than a week in second place.

Ticketing System: Uses

The ticketing system is the primary communication tool for users and participants to:

• Inquire about the rights of an image

• Report an issue

• Request a welcome orientation

• Request services such as data curation services

We’ve received tickets from around the world!

Statistics

C TDA gathers statistics on its websites through G oogle Analytics that are visualized through Tableau and shared publicly on C TDA’s main website.

Statistics for each site or C TDA channel cover:• C ontent Use in sessions and pageviews, visits, average visit duration, pages with highest

number of pageviews and most popular landing pages• Navigation in medium and source of visits, most popular referral pages, and keywords used

in searches• Audience Metrics in sessions by country of user, devices used to access content and web

browsers used

C TDA also now records the number of views for each object. These views can be seen when looking at a collection. Only those objects that have been viewed 1 or more times will appear in the “Most Viewed”. This is accessible to everyone.

CTDA & Staff

Throughout the year, C TDA has both interns and student employees who help with a number of projects ranging from online tutorials, marketing materials, or metadata.

Our largest project was to correct metadata for DPLA. This involved both updating the custom transformation and fixing over 22K records; and yes the majority of these records were fixed manually. We also cleaned up type of resource, held by statements, and some rights statements for UC onn.

We have a new intern, C arolyn Moritz, from Florida State University who will be investigating data remediation and creating MODS records from spreadsheets.

Development Timeline 2017

Image Credits• Beach hall (1929), C onstruction, C onnecticut Agricultural C ollege, http:/ / hdl.handle.net/ 11134/ 20002:199701581

• Repairing Laurel Street Bridge, Hartford, http:/ / hdl.handle.net/ 11134/ 40002:18627

• Old State House (C ity Hall) and Federal Building (PostOffice), Hartford, http:/ / hdl.handle.net/ 11134/ 40002:19334

• School children on parade, Hartford, Memorial Day, http:/ / hdl.handle.net/ 11134/ 40002:19019

• Bicyclists in C olumbus Day parade, Main Street, Hartford, October 12 , 1910, http:/ / hdl.handle.net/ 11134/ 40002:19029

• Housatonic Landscape, http:/ / hdl.handle.net/ 11134/ 30002:2715

• Flood of 1936 : Hotel Bond lobby, Harford, http:/ / hdl.handle.net/ 11134/ 40002:19212

• C anning School, http:/ / hdl.handle.net/ 11134/ 20002:199703036

• Esso Service C enter, Derby and Forrest Roads, New Haven, http:/ / hdl.handle.net/ 11134/ 20002:199702965

• Refueling in Westport, http:/ / hdl.handle.net/ 11134/ 20002:199725127

• Men looking at urban-suburban school integration (busing) report, http:/ / hdl.handle.net/ 11134/ 50002:993

• Directory Assistance, http:/ / hdl.handle.net/ 11134/ 20002:199701269

• United States Army dirigible, balloon or airship with crowd of onlookers, http:/ / hdl.handle.net/ 11134/ 40002:14208

Questions?ctda@uconn.edu