Implementing the ITS Archive Data User Service in Portland, Oregon

Robert L. BertiniAndrew M. ByrdThareth YinPortland State UniversityIEEE 7th Annual Conference on Intelligent Transportation SystemsWashington D.C. October 4, 2004

2

Objectives

Introduction to Data Archiving Project

National ITS Architecture and ADUS

Introduce PORTAL (Portland Transportation Archive Listing)

Describe Architecture

Describe Database Processing and Storage

Review Online Interface Features

Conclusions and Next Steps

3

IntroductionGuiding Principles

“Data are too valuable to only

use once.”

4

IntroductionGuiding Principles

“Management of the transportation system cannot be done without knowledge of its performance.”

5

National ITS Architecture Major Components

Travel and Traffic Management Public Transportation Management Electronic Payment Commercial Vehicles Operations Emergency Management Advanced Vehicle Safety Systems Information Management Maintenance and Construction Management

6

Overview: Relational Diagram of ITS Architecture

Source: Guidelines for Developing ITS Data Archiving Systems

7

Archived Data User Service (ADUS) is BornITS Architecture 1999

USDOT Vision for ADUS: “Improve transportation decisions through the archiving and sharing of ITS generated data.”

Principles of ADUS to Achieve Vision All ITS deployments should consider data archiving Archive data to maximize integration with other data

sources and systems Archive data in a way that eases retrieval for those with

access Provide information that is integral to transportation practice

8

Archived Data User Service (ADUS) is BornADUS Standards

Operational Data Control: by managing operations data integrity

Data Import and Verification: through historical data Automatic Data Historical Archive: with a permanent data

archive Data Warehouse Distribution: which integrates the planning,

safety, operations and research communities and processes data for these communities

ITS Community Interface: by providing a common interface to all ITS users for data products specification and retrieval

9

Who Can Use Archived ITS DataStakeholders

Transportation Planning Transportation System Monitoring Air Quality Analysis MPO/State Freight and Intermodal Planning Land Use/Growth Management Planning Transportation Administrators and Policy Analysis Traffic Management Transit Management Construction and Maintenance Safety Planning and Administration Commercial Vehicle Operations Emergency Management Transportation Research Private Sector

10

Data ResourcesPotential Archiving Applications

Traffic Surveillance

Fare/Toll Systems

Incident Management

Traffic Video

Environmental

CVO

Traffic Control

Highway/Rail

Emergency Response

ITSITSDataData

ArchivesArchives

ITSITSDataData

ArchivesArchives

Performance MonitoringNational reportingPerformance-based planningEvaluationsPublic Reactions

Long Range PlanningTRANSIMSIDASFour step modelsTransit routes

Operations PlanningIncident managementER deploymentSignal timingTransit service

Travel Time ForecastingCustomized route planningATIS Advisories

Other Stakeholder FunctionsSafetyLand useAir qualityMaintenance management

11

ADUS Architecture:Implementing a Successful Data Archive

A Few Examples of Existing Systems California PeMS - only statewide system Puget Sound (WSDOT/TRAC) - started small and

successfully expanded San Antonio, TX TransGuide, Datalink system

Lessons Learned from Existing Systems Begin with a single data source Provide data through the web or CD subscription Create a user friendly interface Save raw data Make aggregate data available to users Implement data quality control measures Create adequate documentation of system and metadata

12

Portland Regional ADUS:PORTAL(Portland Transportation Archive Listing)

13

PORTAL

Purpose:

Implement the U.S. National ITS Architecture’s Archived Data User Service for the Portland metropolitan region

Cooperation with Various Agencies–Oregon Department of Transportation–Metro (Portland’s regional planning agency)–The City of Portland–TriMet (Portland’s regional transit agency)

14

PORTALPSU is the Designated Regional Archive Center Through regional cooperation,

Portland State University is the regional center for collecting, coordinating and disseminating variable sources of transportation data and derived performance measures.

Portland ITS data is warehoused at raw and aggregate levels.

An online user interface for extracting relevant performance measures in real time and historical data has been implemented.

15

Portland’s Regional Infrastructure

77 CCTV Cameras 18 Variable Message Signs

(VMS) 436 Inductive Loop Detectors 118 Ramp Meters TriMet Automatic Vehicle

Location (AVL) System and Bus Dispatch System (BDS)

Extensive Fiber Optics Network

16

Portland ADUSData Sources

Current Oregon Department of Transportation (ODOT)

Freeway loop detector data Incident data (future)

National Oceanic and Atmospheric Administration (NOAA) Weather data

Future Portland Metropolitan Transit Agency (Tri-Met)

Automatic vehicle location (AVL) poll data Bus dispatch system (BDS) data

City of Portland Traffic signal count/speed data

Washington State Department of Transportation (WSDOT) ODOT Weigh-in-motion data

17

PORTAL Architecture Fiber Optic Data Connection

Rich Johnson, City of Portland

18

PORTAL ArchitectureData Flows

ODOTTMOC

Loop Controllers

Fiber

PSUADUSServer

Fiber

WebServer

TriM

et

WS

DO

T

City

of

Por

tlan

dFuture

BackupServer

Operations Planning PublicUser Classes

19

PORTAL ArchitectureDatabase Back End/ Web Front End

Database Back End SQL relational database 200 MB per day of raw data 75 GB per year High capacity disk array 5 MB compressed for download

Off-site backup server Daily backups Uninterruptible power

Web-based interface Easily accessible User-friendly PHP language

20

PORTALDatabase Processing and Storage

Raw Data Storage Useful for research Large data sets Previously difficult to manage and organize

20-second loop detector data collected Volume, Speed, and Occupancy collected indefinitely Transferred via fiber optic cable from ODOT TMOC to PSU

relational database

21

PORTAL Database Processing and Storage

Data Aggregation 5 and 15 minute aggregates Performed on 20-second data every night at 3:00 am Data appended to a table for relevant month Flow, speed and occupancy characteristics retained VMT, VHT, travel time, delay added to 5-min and 1 hour

tables Useful for research and planning purposes

22

PORTAL Data Fidelity

Quality Control

PORTAL uses Daily Statistics Algorithm (DSA) developed by the California PeMS database

DSA identifies and marks suspect or erroneous errorsof four types

Type 1: Occupancy and flow are mostly zeroType 2: Non-zero occupancy and zero flowType 3: Very high occupancyType 4: Constant occupancy and flow

PORTAL Web InterfaceHomepage

24

PORTALDatabase Processing and Storage

Users have the ability to: sample raw data store data permanently in accordance to their specifications use aggregate data at desired resolutions

Outputs onscreen table plotted on relevant graphs downloaded on comma separated value (CSV) format for

permanent offline storage

Data Analysis and VisualizationContour Plots: Speed

Data Analysis and VisualizationTime Series Plots: VHT

Data Analysis and VisualizationTime Series Plots: VHT

Data Analysis and VisualizationData Fidelity

29

Next StepsExpanding Archive Functionality

Data from: TriMet Washington State DOT City of Portland ODOT WIM Data

Additional processing tools and performance measures

Acknowledgments

National Science Foundation Oregon Department of Transportation City of Portland TriMet Portland State University Oregon Engineering and Technology

Industry Council

31

FeedbackYour Opinions and Suggestions

Please let us know what you think about existing or future:

Data sources Performance measures Data visualizations and summaries

Contact us:

bertini@pdx.eduhttp://www.its.pdx.edu