electronic data management and workflow

Imagine the result

Electronic Data Management and Workflow

Introduction

Jane Kennedy

Project Chemistry & Data Quality

ARCADIS U.S., Inc.New Orleans

EDD Management and Data Workflow Overview

• ARCADIS data management approach• Planning & setup• Data acquisition• Laboratory EDD prep and submittal• EDD receipt and review• Data distribution• Questions

Electronic Data Management

• Why manage data electronically?

• Consistency

• Confidence

• Efficient access to information

ARCADIS Approach to Data Management

PLANNING

Plan data acquisition,  

management , quality, and reporting systems

ACQUISITION

Acquire data efficiently

MANAGEMENT

Manage data with tools appropriate to output 

requirements

VALIDATION

Evaluate data quality 

compliance with project 

plan

REPORTING

Provide streamlined reporting and data 

accessibility to users

Increase productivity of technical staff

ARCADIS Data Management Systems

• EQuIS 3.2• Desktop or server

• EQuIS 5• Enterprise system• Automation of EDD

management• Internally developed system – Access platform

• Server• Microsoft Excel

• Desktop

• Involved variety of disciplines and stakeholders in selection• Data managers• Corporate IT• GIS personnel• Project teams• Senior management• Clients

• Evaluated overall system applications

EQuIS 5 Deployment

A data acquisition and management strategy defines:

•Coordination of appropriate resources

•Project quality assurance process

•Data deliverables

•Communication pathways to ensure data usability and accessibility

Planning

• Project staff• Provide information to

labs and DMs

Stakeholder Participation

Database

•Field Data

•3rd Party Info

•Lab Data

•Boring Logs

•Maps/Figures

•Data Tables

• Data managers• Receipt, import, query and export

• Data visualization and project Team• Communicate data export content requirements

to DMs

Project Planning Documents

• Work Plan, SAP, FSP, QAPP• Document performance requirements• Summarize project activities• Define project goals• Establish data quality objectives

• Permits• Corrective action goals• Risk standards (RECAP)

Data Management Plan

Data Management Plan• Based on project planning documents• Define personnel responsibilities• Set up work flow• Data acquisition strategy• Establish project nomenclature• Create reference values• Detail storage and archive

Sample Collection Planning

• Establish location and sample nomenclature• Identify locations• Monitoring wells • Soil depths• Sample matrix• Trip blanks, equipment blanks• Field duplicates

• Provide information to field team and data manager prior to sampling

Database Set Up

• Prepare chemistry and geology database• Use applicable project nomenclature• Acquire geographic coordinates• Provide project specific criteria to data manager

• Permit limits• Screening standards/RECAP limits• Corrective action goals

• Identify export requirements• Trend plots, contours, charts• GIS or CAD

Field Data Acquisition

• Establish field data requirements• Well construction• Geologic information• Water quality parameters• Water levels

• Identify data acquisition format• Manual entry onto forms with transfer later• Electronic acquisition with nightly upload

• Geographic coordinates• Survey or GPS

• Routes of data transfer• Who has

responsibilities?• What format?• Quality Control Review

prior to submittal to Data Manager

• Geographic coordinates• Get them to the Data

Manager in a timely manner

Uploading Field Data

Chain Of Custody Documentation

• Complete COC with as many EDD expectations as possible• Sample ID• Matrix• Sample type• Samples to be

used for site specific QC

Communicate Project Requirements to Laboratory

• Laboratory is a partner in the project success• Communication prior to sample collection is

crucial• Data quality requirements• Performance expectations• Deliverables • Communication tree

What the Laboratory Needs

• List of samples and performance criteria

• Sample nomenclature• Select EDD Format prior to sample

submission• DEQ format• Consultant format

• Send project reference values• Confirm lab has programming

completed to generate required EDD

Laboratory EDD Preparation

• Understand the requirements• Spend the time to develop the 4-file EDD NOW• Minimize manual entry• Report and EDD MUST match

• Rounding routines• Manual data entry peer review

• Data Checker (EDP) = zero defects• EDD requires specific naming convention• Follow the rules or it will get kicked back

• Contact client for direction• Don’t make anything up• Review reference values for

other options• Get email confirmation of

directions• Include additions changes in

email submission of EDD

Reference Values Missing

Potential EDD PitfallsSample Table

• Confirm sample ID and handwriting interpretations• Do not add any suffixes to the sample ID unless

directed by consultant• Sample Delivery Group (report number) must be

populated• Lab may be required to populate start and end depth

for soils• Sample receipt date must be populated

• Sample type is critical because some samples require listing parent samples

• Field duplicates – lab may need to use sample type of “N” to clear checker

• MB = Material Blank not method blank (LB)

Potential EDD PitfallsSample Table (continued)

Potential EDD PitfallsTest Table

• Subsample amounts must be populated• Lab name must be populated• Sample dates and times must match sample file• Percent moisture cannot be null for solids• Analysis type must be appropriate for dilutions,

re-analyses• Understand the use of T, D, N for total and

dissolved field• Caution with methods where multiple parameters

reported (e.g. Method 300 or 352.3)• Time format can cause problems

Potential PitfallsResult Table

• Do not add suffixes to the CAS number• Only 1 result is reportable = Yes (multiple dilutions)• If the detect flag is yes, the result value cannot be

null• Units fields cannot be null for populated fields

requiring a definition of value units• Subsample amount must be populated• Quality control data must be included in the

appropriate field

Additional Laboratory Challenges

• Can’t load project reference values

• Data Checker continues to show errors

• Do NOT try to re-write the export every time you have samples

• Make sure to save the export to laboratory system

• LIMS changes - CAUTION

EDD Submission

• Data checker (EDP) = zero defects• Document problem resolution• Follow EDD naming convention• Email EDD to appropriate venue(s)

DATABASE

Email

Web

FTP

Lab

PDA

FieldEDD

LabEDD

SubmitterNotice

ManagerNotice

Data Receipt:• Variety of information received in Electronic

Data Deliverable (EDD) for import into the database

EDP

• EDDs received by project data manager or via direct upload system

• EDP confirms completeness and compliance (3.2 v 5)

• Do not correct the EDD• If it is not compliant, return

to lab

Consultant EDD Management

Database

Data Manager

•Field Data

•3rd Party Info

•Lab Data

•Boring Logs

•Maps/Figures

•Data Tables

EDD Upload to Project Database

• Manage EDD upload schedule• Group work in batches

• Tracking is critical• Report and EDD may not arrive

at same time• Verification / validation of

conformance

Consultant Database Updates

• Verify sample IDs• Laboratory data flag definitions• Run update queries to move qualifiers• Add location codes to link samples• Field information

• Field parameters• Parent samples for field dups

• Perform project specific queries

Confirmation of Content

• Export data to crosstab format• Review a percentage of data against lab reports

• Define percentage in project plan process • If deficiencies are identified return to lab for

correction• Rounding Routines• Significant Figures• Data Flags

Track Changes to the Database

• Identify fields in the database to document changes by data managers• Historical data source• Laboratory Flagging changes• Special data qualifier definitions• Validated, Verified, or Neither• Validator comments/reason for qualification

• Updates of project information

• Screening tools are available• Critical to have appropriate QC

information• Populate data fields appropriately• Understand the limitations• Qualified personnel review of

screening tool reports

Electronic Data Quality Screening

‘My’ Format AND LEADMS

• Create LEADMS EDD export from data system

• If managing 2 EDDs (client specific and LEADMS)• Ensure any changes in the

desktop version are transferred to LEADMS

• Perform confirmation QC to verify

DATABASE

Email

Web

FTP

Charts,Graphs

‘Pretty’ Reports

Tabular data,Crosstabs

Exports for data analysis

and visualization

Exports to Data Users

Database

Data Manager

•Field Data

•3rd Party Info

•Lab Data

•Boring Logs

•Maps/Figures

•Data Tables

Data Exporting

• Quality in = Quality out

• Exports can be automated, scheduled or team coordinated

• Provide information to the DM to yield efficiency

• Database monitors incoming data and generate reports based on arrival of:

– New data• Anytime there is new data for facility 123, build

Report A and email it to me and all my group.– New detections

• Anytime there is new Arsenic data greater than 15 ppb for facility 123, build Report B and email it to just me and my client.

– Date• Build Report C for facility 123 and email it to the

entire group on the 15th of every month.

.

Data Export Automation

Data Export to Visualization Programs

Thank You

Jane Kennedy

ARCADIS U.S., Inc.Phone: (504) 832-4174

Cell: (225) 205-8256Email:

jane.kennedy@arcadis-us.com