DWR GIS Data Steward Training

Crash Course Version

March 13, 2014

Overall Course Schedule

- Overview of Enterprise GIS

- Data Promotion WorkflowInitial Stewardship PlanComplying with DWR GIS Data StandardFinal Production Packet

- Publication

- GeoPortal

- Long-Term Stewarding Issues

- Workflow Focus: Developing Metadata

- Workflow Focus: Calculating Positional Accuracy

What is “The Enterprise”???

The Enterprise

- Architecture

Servers, software, the network, databases

- Data

Storage, classification, geodatabase, origin (current/new, legacy, ex-DWR)

- The DWR GIS Data Standard

- Roles & Responsibilities

Stewards, Sub-stewards, Geodatabase Managers, Oracle instance managers, other IT support, QA/QC, Governance, EGC, end users

Why a Stewarded Enterprise?

- minimize duplication of effort, use of incorrect choice of dataset

- facilitate easier location of best, current authoritative data

- facilitate everyone working off “same sheet of music”

- reduce storage space requirements

- ability to leverage providing data as service to improve performance & reduce network demand

- assign responsibility for improvements to be made in an organized manner

- promote and provide documentation

- clarify distribution/access policy

- decisions about dataset edits are made more consistent

Ultimately, enterprise deployment of stewarded data is chosen to save time, save money, and promote use of consistent, documented data to improve the quality of everyone's work.

It requires up-front investment, but should yield back-end savings.

Demo DWR GIS Atlas

The Enterprise

“Enterprise” is not synonymous with “centralized”. While enterprise systems may or may not be centralized, our chosen approach for DWR Enterprise GIS is federated.

Data Types

Project Data vs. Program Data vs. Enterprise Data (as used in this training class)

Data Type Project Program Enterprise

Data stored on your workstation Yes No No

Data stored on the enterprise architecture, but not “enterprise data”

No Yes No

Data stored on the enterprise architecture, but is “enterprise data”, and stored in Program-specific geodatabases

No Depends Yes

Data stored on the enterprise architecture, but is “enterprise data”, and stored in DWR GIS Atlas

No No Yes

Stewarded Data Origin

- Legacy Data, DWR-sourced

- Current/New Data, DWR-sourced

- Data originating from outside DWR

DWR Governance Board

Enterprise GIS Planning Committee (EGC)

Division of Technology

Services

Danny Luong & Nicholas

Perez

EGC Sub-committees

Steward 1Sub-

Steward

Steward 3

Steward 2

Sub-Steward

Sub-Steward

Misc DTS

Entities Outside DWR Providing Data to DWR

DOE Photogrammetry

DWR GIS Atlas geoDB Administration

Roles & Responsibilities

- DBA

- Geodatabase Manager

- Data Steward

- Data Sub-steward(s) (if applicable) - QA/QC Reviewer

- EGC & IT support

- End users

The Overall Stewardship Process

- Become a steward

- Develop ISP (Initial Stewardship Plan)

- Get approvals from DWR Enterprise GIS Committee (EGC) for ISP

- Post data to development environment, with help from DOE

- Bring data up to DWR GIS Data Standard

- Conduct QA/QC

- Submit final dataset promotion packet to EGC

- Once EGC approves, work with DOE to get development copy of data posted to production environment of DWR GIS Atlas

- Post data to GeoPortal

- Once published, maintain/update data as necessary

Enterprise Data Promotion Workflow

Initial Checklist Steps

1. Decide to become a data steward or accept a request to become one.

2. Identify appropriate relationships and explore capacity for staff to fill various roles. Should there be sub-stewards? Who? Who will be the ArcGIS server/geodatabase administrator?

3. Identify the correct candidate dataset for eventual promotion to DWR GIS Atlas. Compare various datasets. What are strengths and weaknesses of each? Discuss options with sub-stewards and select best candidate dataset. Begin developing draft data model. Identify final products (feature data service, map service, etc.).

4. Initiate formal indication to DWR Enterprise GIS Committee (hereafter EGC) of intention to steward dataset.

5. Develop Initial Stewardship Plan.

6. Secure stewardship roles and obtain signoffs.

Promotion Workflow Checklist Steps

7. Submit candidate dataset.

8. Finalize draft data model and description documents (next slides will illustrate).

9. Develop Geodatabase Management Plan.

10. Develop Archive & Maintenance Plan.

11. Submit Initial Stewardship Plan to EGC.

12. EGC’s Enterprise Architecture & Applications sub-committee (EA&A) will review and approve Geodatabase Management Plan and Archive & Maintenance Plan. Revise these if necessary.

13. EGC’s Data Standards sub-committee (Data Standards) will review and approve candidate dataset selection and approve Data Model. Revise these if necessary.

14. EGC signs off on Initial Stewardship Plan.

ISP Forms

Data Model Example 1

Data Model Example 2

Data Model Example 3

Promotion Workflow Checklist Steps

7. Submit candidate dataset.

8. Finalize draft data model and description documents.

9. Develop Geodatabase Management Plan.

10. Develop Archive & Maintenance Plan.

11. Submit Initial Stewardship Plan to EGC.

12. EGC’s Enterprise Architecture & Applications sub-committee (EA&A) will review and approve Geodatabase Management Plan and Archive & Maintenance Plan. Revise these if necessary.

13. EGC’s Data Standards sub-committee (Data Standards) will review and approve candidate dataset selection and approve Data Model. Revise these if necessary.

14. EGC signs off on Initial Stewardship Plan.

EGC Signoff? Congratulations, you are now a steward!!!

Enterprise Data Promotion Workflow

Promotion Workflow Checklist Steps

15.Ensure everyone involved in stewardship of dataset has appropriate accounts/permissions.

16.Ensure everyone involved in stewardship of dataset has necessary training.

17.Coordinate with any outside agencies, as necessary.

18. Determine permissions, data security policy, and security management approach.

19.Work with DOE to get identified candidate dataset copied into the development geodatabase environment.

19.Develop and/or edit dataset to comply with current version of DWR GIS Data Standards.

Promotion Workflow Checklist Steps

21. Complete all metadata for dataset.

22. Have independent QA/QC conducted. Obtained signoff of independent QA/QC certification.

23. Develop any end user or data sharing agreements, if necessary.

24. Complete Standards checklist.

25. Update DWR GeoPortal.

26. Submit Final Dataset Production Document to EGC.

Don't know if data can be shared? Government Code Section 6254, part of what is known as the “Public Records Act” tells you what is exempt from the public domain. Or ask Sonny Fong. Or a DWR attorney. But don't just guess.

Development vs. Production Environments

The “Green Cans of Tuna”

Notice: this is how it can be seen as “federated”

Demo Development Environment

& Transactional/Multiuser Editing Guidelines Document

Final Dataset Production Forms

DWR Spatial Data Standard

It is all about QUALITY

Enterprise Data Promotion Workflow

Promotion Workflow Checklist Steps

27. EGC reviews document and submittal. If EGC indicates changes, revisions, or additional work, complete those changes and re-submit to EGC.

28. EGC approves dataset for publication.

29. Work with DOE to migrate data from development environment into DWR GIS Atlas production environment.

Development vs. Production Environments

The “Green Cans of Tuna”

Notice: this is how it can be seen as “federated”

Promotion!

Promotion Workflow Checklist FINAL Steps

31. Update DWR DataSpace default template.

32. Maintain dataset on ongoing basis.

When it comes to GIS, incorporating change benefits from a clear line of communication.

Ongoing Changes to the Data

Maintenance and Archiving Issues

- Incorporating improvements and/or directing the process

- Staying abreast of data updates from outside DWR

- When to update metadata, QA/QC, positional/attribute accuracy

- Are the data dynamic or static?

- Versioning

- Legal considerations

Back to the Committee?

Enterprise Data Promotion Resources

- Enterprise GIS Committee (EGC) and sub-committees

- Dwrgis.water.ca.govExisting enterprise data documents…can be used as models to followMeeting notesKey documents

- Specific documents, including:ISP and Final Dataset Production packetsEnterprise Data Promotion WorkflowDWR Spatial Data StandardsCalculating Positional Accuracy

- Software resourcesArcGIS DiagrammerGeoPortalPositional Accuracy Assessment Tool extension

- Your colleagues

Questions? Discussion?

GeoPortal

Metadata

Positional Accuracy & PAAT