applied scientific research - egypt scholars...civil engineering (intelligent transportation...

Applied Scientific Research:From Lab to Field

Samah El-Tantawy, PhDAssistant Professor

Faculty of Engineering, Cairo University

2/07/2018

Applied Scientific Research: From Lab to Field

My Academic Career Journey

Electronics and Communications

Engineering, B.Sc.

Engineering mathematics,

M.Sc.

Civil Engineering (Intelligent

Transportation Systems), Ph.D.

AI Applications (Transportation,

Education,…)

2

Basics of Scientific Research

Agenda

• I: What is research? The Scientific Research Process

• II: Why is Research?

• III: A Journey of An Applied Scientific Research Example

From Lab to Reality

• IV: When to Start Research?

• V: What do you need to know before starting your

research?

3


I. Scientific Research Process

4


Have you conducted scientific research before?

5


Research is a Key Driver of Innovation

7

II. Why Do We Need to Conduct Research?

Why should we conduct Research?

• To develop knowledge for professions.

• To develop effective policies.

• To solve practical problems.

• To make informed decisions.

• To increase the knowledge base of larger society.

Huge amounts of daily life and experience in our society are based on what we have learned using the logic and evidence involved in scientific research.

8


III. Taking Research From Lab to Field

9

Smart Traffic Lights that Learn !

Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers

M A R L I N

Smart Traffic Lights that Learn !

Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers

M A R L I N

Baher Abdulhai, Ph.D., P.Eng.

Director, ITS Centre and Testbed, University of Toronto

Co-founder of Pragmatek Transport Innovations Inc.

Samah El-Tantawy, Ph.D.

Post Doctoral Fellow, University of Toronto

Co-founder of Pragmatek Transport Innovations Inc.

Applied Scientific Research: From Lab to Field 11

1. Make An Observation

Traffic Congestion: Urban CANCER

0

5

10

15

20

25

0

2

4

6

8

10

12

Jakarta Sydney Chicago Area NY-NJ Cairo Los Angeles Toronto

Pp

op

ula

tio

n in

Mill

ion

s

Co

st in

$ B

illio

ns

Cost of Congestion Population


Traffic Congestion Problem

Tahrir Square, 1914

Tahrir Square, 2014

13

2. Define a Problem Statement and

Formulate a Research Question


Traffic Congestion Solutions

More Supply

Less Demand

Intelligence

16


Intelligent Transportation Systems

Intelligent Transportation Systems (ITS)

Advanced Travelers Information Systems

Advanced Public Transit Systems

Advanced Traffic Management Systems

Ramp Metering Signal Control Route Divergence

Fixed-Time Control Actuated ControlAdaptive Traffic Signal

Control (ATSC)


3. Gather InformationLiterature Review


Research Database

21


ATSC in North America

22

MARLIN-ATSC: Level 4

Evolution of “Adaptive” Traffic Signal Control

Level 0• Fixed-Time

and Actuated Control

• TRANSYT• 1969, UK

Level 1• Centralized

Control, Off-line Optimization

• SCATS• 1979,

Australia • >50

installations worldwide

Level 2• Centralized

Control, On-line Optimization

• SCOOT• 1981, UK • >170

installations worldwide

Level 3 • Distributed

Control, Model-Based

• OPAC, RHODES• 1992, USA• 5 installations in

USA

Level 4• Distributed

Self-Learning Control

• MARLIN-ATSC• 2011, Canada

23

23

Issues with Leading ATSC Technologies

• Expensive• Not scalable • Not robust

Centralized

• Relying on an accurate traffic modelling framework

• the accuracy of which is questionableModel-Based

• Increasing the complexity of the system exponentially with the increase in the number of intersections/controllers

Curse of Dimensionality

• Requiring highly skilled labour to operate due to their complexity.

Human Intervention

Requirements

24

24

4. Hypothesis Construction


In a Nutshell• Grand objective

– Intersections "talk to each other"

– Each is affected by what is happening upstream

– Each affects what is happening downstream –

– Whole network control in one shot from a grand brain is the dream

• Issue

– Intractable theoretically

– Too complex practically

– Requires massive and very expensive communication

• Solution??

– Decentralized

– Self learning: agents learn to control their local intersection

– Game theory based: agents learn to collaborate

26

5. Define Methodology and Do Experiments


Traffic Signal Control ProblemMarkov Decision Process

28

Current State

Next State ??


Markov Decision ProcessDynamic Programming

29

• The Transition Probability Matrices (TPM)

• (-1)* Delay

•Extend Green (a1)•Switch to Minor Street (a2)

• Queue Length• Short (S1)• Long (S2)

State Action

Transition ProbabilityReward

Queue Length

Optimal Control Policy that

maximizes the expected long-term

rewarda1

a2Curse of Modeling Curse of Dimensionality


Smart / Intelligent Machine?

SMA

RT

can

:

Sense

Interpret

Learn

React

Optimize

Adapt

Help

30


The Most Intelligent Machine Known to Man?

31

Artificial Intelligence MethodReinforcement Learning

Reward (savings in delay)

environment

action

observation (queue Lengths)

MARLIN

32

Q a1 a2

s1 -10 -5

s2 -3 -15

Q Table

Model-Free Algorithms

Model-Based Algorithms

Solution Form

Theoretical Framework

Problem Identification

Stochastic Control Problem

Single Agent (Stationary Environment)

Markov Decision Process

Optimal Control Policy

Dynamic Programming Methods

Reinforcement Learning

Multi-Agent (Non-Stationary Environment)

Stochastic Game

Optimal Joint-Policy

Game Theory Methods

Multi-Agent Reinforcement Learning

Single vs. Multiple Agents:MARLIN Independent vs Integrated33

33

Corridor Synchronization Vs. Network-Wide Coordination

Collaboration with each adjacent intersection in the

neighborhood

Collaboration with each adjacent intersection in the

neighborhood

34

Corridor SynchronizationCorridor Synchronization

34


MARLIN-ATSC Framework

Agent(Learning and

Decision Making)

Environment(Observation and Acting)

Environment-Agent Interface

(Synchronized Interaction)

RL

In

pu

t P

ara

me

ters

RL

In

pu

t P

ara

me

ters

Sig

na

l D

ata

P

ara

me

ters

Sig

na

l D

ata

P

ara

me

ters

35


6. Data Collection and Analysis

Applications Simulation Testbeds on Toronto

Applications Simulation Testbeds on Toronto

37


Traffic Simulator

38


Testbed Networks and Applications

Isolated Int.

→ Different RL Design Parameters

5- Int. Network

→ Transferabilityand Different CoordinationApproaches

Large-Scale

→ Scalability and Network-wide

Coordination Effect

39


Isolated IntersectionBay and Front (Downtown Toronto)

Zone 1

Zone 2

Zone 3

Zone 4

0%

5%

10%

15%

20%

25%

1 2 3 4 5 6

Dem

an

d A

rriv

al P

erce

nta

ge

Time Interval (10min)

Uniform Profile For All Phases Phase 1 (NBL-SBL) Profile Phase 2 (NB-SB) Profile Phase 3 (EBL-WBL) Profile Phase 4 (EB-WB) Profile

40


5-Intersections Network

4 Bay & Front

2 University/

York & Front

3 Wellington

& Bay

1 Bay & Lake

Shore

5 Yonge &

Front

4 Bay & Front

2 University/

York & Front

3 Wellington

& Bay

1 Bay & Lake

Shore

5 Yonge &

Front

IN

Wellington & BayIE

Yonge & Front

IW

University/

York & FrontIW

Bay & Lake Shore

IC

Bay &

Front

41


Large-Scale Application in Toronto

42


7. Make Conclusions


Large-Scale ApplicationAverage Delay % Improvements

MARLIN-IC vs BCMARLIN-IC vs BC

% ImprovementArea 1

Area 2

Area 3

44


Large-Scale ApplicationAverage Route Travel Time for Selected Routes

0

1

2

3

4

5

6

7

8

1 2 3 4 5 6 7 8 9 10 11 12

Av

era

ge T

ra

vel

Tim

e (

min

)

Time Interval (5 min)

Gardiner EB

BC MARL-TI MARLIN-IC

0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8 9 10 11 12

Average T

ravel

Tim

e (

min

)


LakeShore EB to Spadina NB


0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8 9 10 11 12

Average T

ravel

Tim

e (

min

)



BC MARL-TI MARLIN-IC0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8 9 10 11 12

Average T

ravel

Tim

e (

min

)



BC MARL-TI MARLIN-IC0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8 9 10 11 12

Average T

ravel

Tim

e (

min

)




45

Hardware-in-the-Loop Simulation (HILS)

Hardware-in-the-Loop Simulation (HILS)

Samah El-Tantawy Kasra Rezaee Hossam Abdelgawad Tamer Abdulazim Baher Abdulhai


Controller Interface Device(CID)RS485 to USB

Traffic Signal Controller

RS485 -SDLC protocol

USB -SDLC protocol

Industrial Computer

Ethernet -NTCIP protocol

Paramics Modeller

MARLIN-HILS Architecture

47


8. Communicate Results


Writing a Research Proposal/ Publishing a Paper

7/23/2018 49


MARLIN RecognitionIEEE ITSS Best Ph.D. Dissertation Award, 2013

Canadian Institute of Transportation Engineers Scholarship, 2010

INFORMS George B. DantzigDissertation Award, 2013

ITS Canada Scholarship, 2011

51


MARLIN Featured in Media!

52

52

Towards System Implementation Towards System Implementation

53

MARLIN Field Implementation

Ethernet

Switch

Industrial Computer

running MARLIN Traffic Controller

54

Lessons Learned for Future Consideration


MARLIN Final Edit.mp4


IV. When to Start?

57


IV. What do you need to know before starting your research?

7/23/2018 59

Applied Scientific Research: From Lab to Field7/23/2018 60

Applied Scientific Research: From Lab to Field7/23/2018 61


Lessons Learned

62

• Research Never Ends– Stay on Top of the Literature

• Passion and Handwork are the Keys

• Adapting toYour Audience Academia Vs. Industry Vs. Government

“If Opportunity Doesn’t Knock, Build A Door “ Milton Berle

Being Out of Your Comfort Zone

MARLIN is One Piece of the Puzzle

Samah El-Tantawy, [email protected]

Assistant ProfessorFaculty of Engineering, Cairo University

27/07/2017

applied scientific research - egypt scholars...civil engineering (intelligent transportation...

Documents