1

Forest Fire Monitoring using Smart Unmanned

Aerial Systems

Hugh H.T. Liu University of Toronto

Institute for Aerospace Studies

1

Canada’s Wildfires Key Facts about Canada’s Forests * !  348 million hectares of forest !  153 million hectares: ‘sustainably managed’ !  8.8 million hectares: damaged by insects (2012) !  4.2 million hectares: burned by forest fires (2013)

2 * Natural Resources Canada (nrcan.gc.ca) and various resources therein

2

Fire Monitoring using Satellite Sensors !  NOAA’s Advanced Very High Resolution Radiometer

(AVHRR): daily snapshot of Canada at a 1-km nominal resolution

!  Vegetation (VGT) sensor: provides daily coverage of Canada at a 1-km resolution (no thermal channels)

!  Landsat’s Thematic Mapper (TM) sensor: a 30-m high resolution observing a given location only once every 16 days (detailed maps of burned areas)

!  NASA’s Moderate Resolution Imaging Spectroradiometer (MODIS)

3

Challenges !  “false alarms” by fire detection/filtering

algorithms (14%) !  The algorithm cannot detect fires through

thick cloud or smoke (satellite hotspots represents 65% of actual)

!  The time lapse !  The actual size of the actively burning area

is difficult to determine from satellite imagery.

4

3

Potential Improvement by ‘Smart’ UAVs (UAS)

!  Aero Solutions – "  Thermal infrared scanners flown on board aircraft are often used to map

fire hotspots and fire intensity over individual fires or small regions. This information allows fire management agencies to effectively target fire suppression efforts by water bombers and ground attack crews.

!  UAVs - "  Hot Spot Detection "  Fire Behaviour (Boundary)

!  Smart Solutions "  fully autonomous "  Integrated systems

5

Hotspot Detection Flight Demonstration

!  Burwash Training Site on 23-24 May 2013 !  to demonstrate a fully functional proof-of-

concept system for monitoring of forest fires using UAVs.

6

4

7

Flight Demo

8

5

Smart Solutions !  to detect all fire targets and hot spots placed in the

area by the MNR personnel; !  to transmit the location of each hot spot to a ground

station in flight; !  to store detected hot spot images of the fire (thermal

and visual) for post processing and verification; and !  to perform post-flight test analysis.

9

Smarter Solutions !  The wildfire boundary can be

represented by a one dimensional line propagating in the two dimensional plane;

!  The propagating behaviour model is developed;

!  The fire frontline is estimated by a network of UAVs in formation flight

10

6

UAS Forest Fire Management: Product and

Solutions Hugh H.T. Liu

Founder of Arrowonics and Professor, University of Toronto: liu@utias.utoronto.ca

UAS Forest Fire Monitoring

!  Unmanned Aerial Systems (UAS) in autonomous formation flight to detect and estimate fire front lines and detect hotspots

!  Collected imagery and data is continually relayed to ground station/command centre

!  Fire front estimation and prediction is used to make critical decisions

12

7

UAS Forest Fire Monitoring Software on-board of the UAS responsible for: "  Collecting radiometric data from thermal camera "  Converting radiometric data into thermal images easily

readable by fire crew members "  Processing and classifying data to detect hot spots based

on temperature threshold "  Stamping images with GPS data "  Interfacing with radio transmitters to send key data to

ground station Ground mapping software responsible for: "  Receiving GPS location of hot spots identified by the UAV "  Mapping of hot spot locations

13

Our Vision: Network of UAS Integrated Platform Development:

#  Fully autonomous from vehicle deployment to retrieval

#  Connected network of UAS for optimized applications #  formation, coordination, cooperation #  multi-UAS payload sharing, time sensitive missions #  redundancy

#  Integrated H/W and S/W system for all UAS in the network

14

8

Our Product: ArrowCloud Stations

Highly Autonomous Deployment Platform:

15

Remote Command

Centre

Activation and Autonomous

Takeoff

Perform Mission/Data

Collection

Autonomous Landing and

Recharge

Mission Instructions

Autonomous Flight to Desired Location

# User Input # as Needed

# Data Relayed # in Real Time

Autonomous Flight to Ground Station

Ready for Next Mission

Centre for Aerial Robotics Research and

Education (CARRE)

Director: Hugh Liu University of Toronto

Applied Science and Engineering

16

9

Centre for Aerial Robotics Research and Education (CARRE)

•  an initiative to expand and unify research and teaching activities related to the burgeoning field of aerial robotics

•  to promote collaborative research within the Faculty in aerial robotics, to enhance our Faculty’s national and international reputation as a centre of excellence in aerial robotics

A new NSERC CREATE program in unmanned aerial vehicles (UAVs) was established.

17

NSERC CREATE-UAV Program Under the NSERC Collaborative Research and Training Experience (CREATE) Program, the U of T led CREATE-UAV program is to develop a training program that will increase the number of professionals and nurture future leaders in the developing UAV industry.

May 20, 2015 NSERC CREATE National Announcement, UTIAS 18

10

Program

!  Unmanned aerial vehicles (UAVs) hold great promise for applications as diverse as natural resource monitoring, infrastructure inspection (pipeline, railways), agriculture, mineral exploration, journalism, and search & rescue operations.

!  Given Canada's vast geographical landscape (e.g. the Arctic) and natural resources, the Canadian commercial or civil UAV market is expected to grow rapidly.

!  This program provides a unique research and training opportunity focused on UAVs to give graduate students the interdisciplinary research, entrepreneurial, and leadership skills needed to propel Canadian aerospace companies forward into a prosperous future in this field. 19

Highlight of Research and Education

•  Comprehensive courses: Fundamentals of UAVs and Integrated UAS Design

•  Collaborative Research Projects •  UAV Design Challenge and Field Camp - Technical development and

demonstration •  Industrial Internship (4-month, 8-month) •  Outreach program: TEDxUAV – public awareness •  Student research forum •  International symposium on future UAVs and Applications •  Entrepreneurship and Leadership workshop

20

11

This is an Industry Stream Program

a vital research and training opportunity focused on UAVs to give graduate students the interdisciplinary research, entrepreneurial, and leadership skills needed to propel Canadian aerospace companies forward into a prosperous future in this field. •  Total Program funding over $2.7M, where NSERC support $1.65M,

plus various university and industry commitments •  11 faculty members (UofT, York, Waterloo) •  5 (initial) industry partners and many more •  10 international collaborators •  over 150 trainees, over 6 years

21

Research Objectives and Topics jointly proposed by researchers and industry partners, based on expertise, needs, and potential applications - including but not limited to the following topics •  Control, Autonomy and Navigation

vision-based localization and mapping; visual inspection; robust control and machine learning; sensory calibration; fault detection and fault tolerant control ...

•  ︎ Airframe Optimisation aerodynamic optimization; maneuverability performance; light-weight structures and materials; flow control ...

•  ︎ Sensory Feedback for Improved UAV Pilot Performance pilot’s training; flight simulation; performance assessment, pilot training ...

22

12

Team Members •  H.H.T. Liu - PI, systems and control •  D.W. Zingg - CRC, aerodynamic shape optimization •  T.D. Barfoot - CRC, mobile robotics •  M. Daly (York) - instrument scientist •  P. Grant - aircraft flight and simulation •  P. Lavoie - flow control and experimental aerodynamics •  S.L. Waslander (Waterloo) - autonomous vehicles •  J. Kelly - vision based and inertial navigation •  A. P. Schoellig - autonomous control •  C. Steeves / G. Hibbbard – structures •  R. Urtasun - machine learning and computer vision

23

National and International Collaborators

•  N.Roy - MIT •  C. Tropea - Tech U. Darmstadt

(Germany) •  C. Bettstetter - U of Glagenfurt, Austria •  A.M. Naguib - Michigan State U •  E. Frew - U of Colorado, Boulder •  J. Langelaan - Penn. State U

•  I. Mantegh – NRC •  M. Peasgood – Aeryon •  H. Bloandhemmat – Micropilot •  H. lacheray – Quanser •  B. McLuckie - Brican •  E. Earon - PrecisionHawk

24

13

Please Join Us! We welcome new research collaborators and industry partners •  active participation in research, design and

development projects •  active interactions in workshops, forums, panels •  active involvement in design challenge, field

camp, and comprehensive courses

•  ︎ internship: 2-3 internships per year

25

Further Information Director of CARRE

•  Hugh Liu, liu@utias.utoronto.ca

Associate Directors •  Jonathan Kelly, jkelly@utias.utoronto.ca •  Angela Schoellig, schoellig@utias.utoronto.ca

http://www.utias.utoronto.ca/aerial-robotics

26