integrated data management network: project tracking deanne dipietro data manager, california lcc
TRANSCRIPT
The Team
Lei Ann WilsonFort Collins Science Center
Tim KernFort Collins Science Center
Deanne DiPietroCalifornia LCC
Zhahai StewartCalifornia LCC
Matt HellerGreat Northern LCC Emily Fort
National Climate Change & Wildlife Science Center
Josh BradleyArctic LCC
DD
Goal of the IDMN Project Tracking Task
“Support cataloging and tracking of LCC funded projects, and improve the integration of project information for use across the LCC Network.”
Multiple Needs
• National level: for the National LCC to analyze what the LCC Network is funding, doing, and with what partners.
• The Public: to discover LCC projects relevant to their work
• LCC level: for LCCs to manage detailed project information including proposals, funding, contacts, deliverables, contracts.
Information Needs for the National LCC Project Catalog
• All LCC projects in one place• Links to more detailed info• Types of deliverables• Types of end users• Funding by category• Partner match• # of projects co-funded by multiple LCCs• # of projects in progress/completed• # of projects by subject area• Projects by Congressional district• And more….
Two kinds of project information
• Project Cataloging: basic, standardized information about LCC projects, with links to more detail on LCC websites
• Project Tracking: detailed information about every aspect of a project needed to manage it from start to completion; typically expands on project cataloging with deliverables, contacts, deadlines, etc.
Project Tracking LCC level
Project Cataloging
National level
The Challenge
Diversity among LCC approaches, styles, and technical capacity
The need to have one view across all the LCCs’ projects.
DD
Dual Approach1. Develop a unified project
metadata catalog via a data exchange and aggregation system,
2. Help LCCs with choices and adoption of diverse project tracking software that support national participation.
DD
Previous work:
Simple National Project CatalogSpring 2013
Gathered, standardized, and assembled project metadata from all 22 LCCs
Collaboratively developed a common standard for the data, got buy-in from all LCCs
Worked with each LCC to incorporate the standard with their current methods for managing this data
http://lccnetwork.org/DD
Existing National Level Project Database(as of Spring, 2013)
• 395 projects from 22 LCCs • Filter by LCC or Primary Category, full-text search• Links to detailed info on LCC website• Download the spreadsheet
Simple National Project Catalog Content Standard
Basic InformationLCC NameUnique ID for project (within LCC)Project TitleDescriptionProject StatusProject Start DateProject End DateProject URLProject Lead person name, emailProject Lead OrganizationPartner OrganizationsTarget Audience/End UsersFiscal Years of AllocationComments
Duplicate ReportingOther LCC co-funding this projectIs this project cataloged in ScienceBase?
FundingTotal Funding by Your LCCYour LCC Funding by Year and SourceTotal In-kind, matching, or other fundsMatching or In-Kind Funds by Year and SourceNotes about matching and in-kind contributions
Classifying and groupingProject CategoryDeliverableSubject KeywordsGeographic KeywordsCongressional Districts
DDRed = has a controlled vocabulary
Previous work:
Simple National Project Catalog
The SNPC Spreadsheet tool. Each LCC data manager was provided their data back in standardized form.
Previous work:
Simple National Project Catalog
DB or CMS
non-std spreadsheet
(or none)
LCCs with databases incorporated the standard and wrote CSV exports.
LCCs with spreadsheets or nothing yet were helped to adopt the standard spreadsheet.
At the end of this effort, everyone was theoretically on the same page and ready to easily update the National Project Catalog using the standard.
Excel, for analysis
Webinterface
(all LCCs)
Standardized metadata in
CSV
Standardized metadata in
CSV
Project Metadata Exchange and Aggregation
IDMN Objectives:
• Develop a design of a data exchange architecture
• Address the needs of the National LCC while allowing the LCCs to keep their individual data systems
• Implement prototypes to refine the protocols and provide reference implementations
Standardized metadata in
JSON
Generalized Architecture
National LCC Project Catalog
Spreadsheet with National Catalog Fields
LCC (master records)
Aggregation Site (derived records)
Standardized metadata in
CSV
Exchange Formats
Local DB/Application with National Catalog Fields Web
interface(all LCCs)Web
interfaceExports
Aggregation Site Decision: Prototype in ScienceBase
• Already has wide acceptance as a common platform for LCCs
• Willing to commit staff time to this effort
• Opportunity to integrate with other ScienceBase services
• Challenging to modify an existing system that serves multiple purposes, but worth it in the long run
ScienceBase
• Data catalog and collaborative data management platform with project cataloging capabilities
• Central search and discovery application• Data Repository• Used by many different communities for many different
purposes• Has a front end web application, DEPTH, to create project
metadata that complies with the national LCC standard• Web services for facilitating other applications which provides
added flexibility to display data
LW
ScienceBase as a master record source AND the aggregation site
LCC National Project Catalog
atScienceBase
Master LCC project records
Aggregation Site (derived records)
Import via standard
Exports
Spreadsheet with National Catalog Fields
Local DB/Application with National Catalog Fields
ScienceBase DEPTH
Direct entry
Status of ScienceBase Pilot Effort
• Edit in ScienceBase using DEPTH for master record project metadata entry and management (now being tested by core group)
• Batch upload of standard records using JSON-based REST Services (still in dev/testing with North Pacific)
• Export of data for use in LCC webpages (still in dev/testing with Desert LCC) LW
Prototype Architecture (interim arrangement)
Simple National Project Catalog
at lccnetwork.org
The Simple National Project Catalog continues to serve the purposes of aggregation and translation of LCC project data to the exchange standards while we develop and test things at ScienceBase.
Spreadsheet with National Catalog Fields
Local DB/Application with National Catalog Fields
CSVCSVCSV
JSON
Spreadsheet
IDMN Objective:• Investigate possible project cataloging or
project tracking systems which could be integrated into the overall framework, existing or in development.
Informing Selection and Adoption of Project Tracking Software in the LCCs
Supporting LCC Software Adoption• Reviewed the range of
viable options for managing project metadata and contributing to the national catalog
• Created information sheets to assist in informed choices
Option Comparison Summary
SNPC Spreadsheet
ScienceBase (DEPTH)
Arctic PTS
Great Northern PTS
*Customizing a CMS (Content management system)
Public Web Display
No Yes No No Yes
Web Data Entry
No Yes Yes No Possible
Technical Expertise Needed for Administration
No No High Low High
Project Tracking
No Some Yes Yes Possible
Customization Options (data, Reporting, etc.)
No Limited Yes Yes Possible
Automated National Data Integration
No Yes Planned Planned Possible
Ongoing Development
No Yes Yes Yes N/A
Contact Name Zhahai Stewart, Deanne DiPietro
Lei Ann Wilson
Josh Bradley
Matt Heller
N/A
● Five ways to accomplish project tracking and contribute to the national project catalog
● LCCs have diverse needs, no one solution is best for all of the LCCs
Simple National Project Catalog Spreadsheet
● Microsoft Excel● Low technical overhead● You can customize it (just keep
the standard fields intact)● You can upgrade later using
the CSV format
National LCC Project Catalog
Just send it
when requested
Standardized metadata in
CSV
Link: http://lccnetwork.org
Great Northern Project Tracking System (based on Arctic Project Tracker)
Manual entry
OR
ingest Simple National Project Catalog spreadsheet ● Microsoft Access
● Low/moderate technical overhead● Project tracking features (i.e. lcc strategic elements)● You can customize it (entry forms, tables, reports,
etc.)● Sciencebase linking/export options● Batch uploads of lccproject.org data
National LCC Project Catalog
Standardized metadata in
.CSV
Link :https://www.sciencebase.gov/catalog/folder/53324b07e4b08fca49cf2d9bWebinar: http://greatnorthernlcc.org/event/462
ScienceBase using DEPTH
Manual entry via DEPTH**
Batch upload*
Display of information on LCC website*
● Use via WWW● Low technical overhead● Integration with RFP and project
deliverable management applications
● limited customization
* = still in beta** = currently under evaluation
National LCC Project Catalog
Auto-majically
Standardized metadata inCSV or JSON
Arctic Tracker Project Tracking System
National LCC Project Catalog
Standardized metadata inCSV (JSON
export planned)Manual entry
Link http://arcticlcc.org/ptsVideo: https://www.youtube.com/playlist?list=PLDO3wp3QE78u_uBRoth4s7MBxMZi9pJSG
● Used by Arctic LCC and Western Alaska LCC● JS, PHP, and PostgreSQL● High technical overhead● Project tracking features (i.e. Tasks, Funding/Invoice Contracting
Workflows, etc)● You can customize it (free/open source)● Browser based multi-user edit support
Custom CMS-based System
Example above: California LCC’s Climate Commons website
Other LCCs doing their own systems include:South Atlantic, Gulf Coastal Plains and Ozarks, Upper Midwest and Great Lakes
● High technical overhead● You have to build it ● Incorporate the standard and
create an export to provide to the National Catalog
LCC National Project Catalog
Standardized metadata inCSV or, soon,
JSON*
* v1.0 is currently being tested
IDMN Project Tracking Accomplishments
• Established a metadata standard and formats for machine-readable exchange
• Described interim and final architectures
• Prototyped the import of project metadata into ScienceBase
• Developed a data entry form in ScienceBase DEPTH
• Evaluated compatible methods for LCC Project Tracking
• Recommendations for further work
Recommendations
● Need a follow-on effort to create a production system that supports LCC project cataloging and contribution to a National LCC Project Catalog
● List of technical specifications found in our “Roadmap” report, including:
○ use ScienceBase as the production site, add the functions that the Simple National Project Catalog is now serving,
○ refinements to the standard, ○ automate tasks now done manually.
See Confluence under IDMN Project Tracking Team for our Roadmap report
What’s Next
• Effort continuing as a subcommittee of the DMWG
• National Project Catalog Update: call for data in November
• Continuing to use lccprojects.org with ScienceBase projects integrated while we work toward the long-term goal of having it all in ScienceBase
Acknowledgments
Sean Finn, GN LCCJosh Bradley, Arctic LCCDeanne DiPietro, CA LCCEmily Fort, NCCWSC/USGSMatt Heller, GN LCCSally Holl, Desert LCCTim Kern, USGSZhahai Stewart, CA LCCLei Ann Wilson, USGS
Thanks to the National LCC for
funding the IDMN Project
[email protected][email protected]
Thank you!