better managing your gis database presented by sol mccrea california geological survey for ca...

BETTER MANAGING YOUR GIS DATABASE

PRESENTED BY

SOL MCCREA

CALIFORNIA GEOLOGICAL SURVEY

FOR

CA NATURAL RESOURCES AGENCY

GIS DAY

THURSDAY, NOVEMBER 13, 2014

sunrise | Malcolm MacGregor PhotographyGolden Gate Bridge

INTRODUCTION

http://malcolmmacgregorphotography.com/tag/sunrise



• The possibility of having a successful GIS Database hinges on:

• The availability of power technology and

• How people better leverage its use

• The biggest deterrence to this often is:

• Our current perceptions and

• Existing work arrangements

INTRODUCTION

A BRIEF PROFESSIONAL BIOGRAPHY

• Started career mostly using existing GIS data to create maps/analyze data

• ArcView

• Additional third party GIS software

• Soon I needed more tools for various types of data creation/analytics:

• MS Access/SQL Server

• DOS Batch files

• ArcInfo

INTRODUCTION

THE ROLES AND RESPONSIBILITIES OF THE GIS PROFESSIONAL ARE RAPIDLY EVOLVING:

• I initially was doing GIS Analytical work, the type of work that now non-GIS people increasingly do.

• I now am learning to function as a Systems Integrationist who needs to:

• Come up with business solutions and complicated workflows

• Utilize cutting edge technology

• Effectively work with a variety of colleagues and clients

INTRODUCTION

THIS PRESENTATION WILL TRY TO ANSWER THE FOLLOWING QUESTIONS:

• What is GIS and who are GIS professionals?

• Who is responsible for creating a GIS Database?

• What is the lifecycle of a GIS Database?

• How do you go about implementing some of the processes outlined in the lifecycle?

• What feedback can the audience give regarding these issues?

INTRODUCTION

WHAT IS GIS AND WHO ARE GIS PROFESSIONALS?

• Cutting edge technology

• A visual tool which is a part of many people’s workflow

• A way to query data that’s unrelated base on its spatial relationship

• A powerful way to resolve increasingly complex business/scientific questions

• A bridge between business/science and technological implementation

GIS IS/CAN BE:

GIS PROFESSIONALS ARE/CAN BE: (PART 1 OF 2)

•Artist that create beautiful and illuminating maps

•Skillful at using useful applications and tools

•Capable of accurately inputting spatial and tabular data

•Collaborative when working with subject matter experts

•Knowledgeable about the subject matter that they are working with


GIS PROFESSIONALS ARE/CAN BE: (PART 2 OF 2)

•Creative at coming up with work-around solutions

•Capable of working with and creating spatial and tabular databases

•Knowledgeable about platform and infrastructure issues

•Capable of creating applications and tools


• Layer less maps

• Mylar maps

• Digital maps

• Interchange files

• Coverage files

• Shapefiles

• Data driven digital maps

• Personal Geodatabase

• File Geodatabase

• SDE Geodatabase

• Graphic Artists

• Drafters

• GIS Technicians

• GIS Analysts

• GIS Database Administrators

• GIS Programmers

• GIS Managers


EVOLUTION OF DATA FORMATS AND MAPPERS

www.ezbordercrossing.comPicture of the Lewiston Queenston Bridge

WHO IS RESPONSIBLE FOR CREATING A GIS DATABASE?

http://www.ezbordercrossing.com/

GIS DATABASES ARE NOW CAPABLE OF:

•Becoming more powerful than ever before

•Having more accurate and simplified input processes

•Providing more precise information to support better business decisions and to avoid lawsuits


WITH INCREASED CAPABILITIES I RECOMMEND: (PART 1)

•An increase in the utilization of different types of specialists needed

•Shared/joint ownership and responsibilities by the various entities

• Subject matter experts

• GIS professionals

• IT specialists


•Professionals should only scope in area of expertise

•All entities should be invited (or aware of) planning meetings

•Better communication which leads to better results


WITH INCREASED CAPABILITIES I RECOMMEND: (PART 2)

THE GROWING PROMINENCE OF GIS PROFESSIONALS:

•Not too many years ago this profession did not exist

•The people operating in this profession are typically underutilized

•They have great potential as bridge builders between subject matter experts and IT specialist


www.funonthenet.inDubai tallest Arch Bridge

WHAT IS THE LIFECYCLE OF A GIS DATABASE?

http://www.funonthenet.in/

IN ORDER TO SUCCESSFULLY CREATE A GIS DATABASE IT HELPS TO: (PART 1 OF 5)

• Have effective communication with the client/project lead to identify business needs (see next section)

• Have a good understanding of the subject matter

• Properly utilize planning tools (Data Flow Diagrams, Data Dictionaries) (See next two slides)


IN ORDER TO SUCCESSFULLY CREATE A GIS DATABASE IT HELPS TO CONSIDER QA/QC ISSUES : (PART 2 OF 5)

•Set up Domains and Subtypes to limit user input

•Determine the type of versioning/archiving you want to use

•Plan on using Replication to separate the Working GIS databases from Final GIS database (See Data Flow Diagram)



•Effectively communicate business needs to IT specialist during implementation phase of development (see next section)

•Create database template structure (see next section)



• Plan set up of database infrastructure:• Cloud based

• Agency or Department based

• Set and user and administrator interfaces:• Desktop

• Smartphone, Tablet

• GPS Devise



• Thoroughly test database system and interface with mock data

• Create metadata, system, and procedure documentation

• Have Client test GIS Database

• Gauge client satisfaction and system functionality


IN ORDER TO SUCCESSFULLY MAINTAIN A GIS DATABASE IT HELPS TO: (PART 1 OF 2)


• Double check user inputted data using versioning

• Create custom administration tools (see next section)

• Create custom QA/QC cross reference tools (see next section)

IN ORDER TO SUCCESSFULLY MAINTAIN A GIS DATABASE IT HELPS TO: (PART 2 OF 2)

• If a table in your database links to file names, locate orphan files (see next section)

• Continually update metadata, system, and procedure documentation

• Create custom reports


WHEN DECOMMISSIONING A GIS DATABASE IT HELPS TO:

•Archive all data for possible future reference


http://sfappeal.com/2011/08/get-used-to-it-old-eastern-span-of-bay-bridge-will-take-years-to-take-apart/Bay Bridge

HOW DO YOU GO ABOUT IMPLEMENTING SOME OF THE PROCESSES OUTLINED IN THE

LIFECYCLE?

http://sfappeal.com/2011/08/get-used-to-it-old-eastern-span-of-bay-bridge-will-take-years-to-take-apart/

A FEW TIPS WHEN CREATING CUSTOM TOOLS USING PYTHON:

• Use ArcGIS ModelBuilder and export code into Python

• Test your code line by line in the Python window

• Create a tool box and link your code to a specific tool window

HOW DO YOU GO ABOUT IMPLEMENTING SOME OF THE PROCESSES OUTLINED IN THE LIFECYCLE?

CREATING A DATABASE TEMPLATE STRUCTURE:

• File Geodatabase are great but you can’t reorder the fields

• I get around this by setting up my templates using the Personal Geodatabase format (*.MDB)

• This way I can reorder the fields when I open it in MS Access

• I’ve heard that the ArcGIS Diagrammer allows you to reorder fields but haven’t had a chance to test it.


HOW TO FIND ORPHANED FILES IN FOLDERS THAT ARE REFERRED TO IN A DATABASE: (PART 1 OF 2)

• You want to first get a list of all the files by using the DOS command DIR

• Create a text file and place it where you want the DIR output to be

• Type the following code and modify it for your purposes:

• DIR "Z:\Documents" /o /b /s > DIR_Output.txt

• Save the text file and change the *.txt to *.bat


HOW TO FIND ORPHANED FILES IN FOLDERS THAT ARE REFERRED TO IN A DATABASE: (PART 2 OF 2)• Double click this file and look for the DIR_Output.txt file

after it runs

• Using MS Excel open up the DIR_Output.txt parsed by the “\”

• Bring the resulting table into GIS

• Join this table to your table where you’re checking to see if there are any orphans using a flag field technique.


A FEW USEFUL RESOURCES:

• The book Getting to Yes: Negotiating Agreement Without Giving In has been a great resource for me to learn how to communicate more effectively.

• Google it

• Take advantage of your ESRI Support Account.


IN REVIEW HERE’S WHAT WE’VE COVERED:

• What is GIS and who are GIS professionals?

• Who is responsible for creating a GIS Database?

• What is the lifecycle of a GIS Database?

• How do you go about implementing some of the processes outlined in the lifecycle?

• What feedback can the audience give regarding these issues?

www.autoevolution.comNavajo Bridge, Grand Canyon

WHAT FEEDBACK CAN THE AUDIENCE GIVE REGARDING THESE ISSUES?

http://www.autoevolution.com/

COMMENTS FROM AUDIENCE:

WHAT FEEDBACK CAN THE AUDIENCE GIVE REGARDING THESE ISSUES?

THANK YOU FOR PARTICIPATING!

Contact information:

Sol McCrea, CFMResearch Program Specialist II (GIS)Forest and Watershed Geology Program | California Geological Survey (CGS) | CA Department of Conservation801 K St, MS 13-40, Sacramento, CA 95814Tel 916/323-9284 | FAX 916/323-9264 | [email protected]

mailto:[email protected]