themis/gbo engineering peer review 1 ucb, oct. 17, 2003 themis preliminary design review november...

THEMIS/GBO Engineering Peer Review 1 UCB, Oct. 17, 2003

THEMISPreliminary Design Review

November 13, 2003

THEMIS ASI Array

Eric Donovan on behalf of the NORSTAR TeamUniversity of Calgary & Athabasca University

THEMIS/GBO Engineering Peer Review 2 UCB, Oct. 17, 2003

THEMIS - Canada

• Motivation for Canadian Involvement

• Prototyping the ASI

• Data Retrieval and Distribution

• Canadian Involvement

• Data Management• Summary • Searching, Sophisticated Analysis• Real-time Website

• Building on Existing Infrastructure

THEMIS/GBO Engineering Peer Review 3 UCB, Oct. 17, 2003

CANOPUS ASI images (one CANOPUS ASI images (one minute cadence; 557 nm filter) minute cadence; 557 nm filter) for the late growth phase, early for the late growth phase, early expansive phase of a substorm. expansive phase of a substorm. The bottom panel is a keogram The bottom panel is a keogram showing high resolution MSP showing high resolution MSP data from the CANOPUS MSP, data from the CANOPUS MSP, also located at Gillam. also located at Gillam.

Note the spiral at 0332 UT, and Note the spiral at 0332 UT, and well defined north-south well defined north-south structure at 0337 UT. Events such structure at 0337 UT. Events such as this illustrate both the as this illustrate both the limitations and advantages of limitations and advantages of ground-based optical ground-based optical observations. The advantage is observations. The advantage is resolution (spatial here, and resolution (spatial here, and temporal not here, but easily temporal not here, but easily obtainable). The disadvantage is obtainable). The disadvantage is limitation to spatial scales smaller limitation to spatial scales smaller than ALL relevant substorm than ALL relevant substorm phenomena.phenomena.

THEMIS/GBO Engineering Peer Review 4 UCB, Oct. 17, 2003

THEMIS & Canada

THEMIS/GBO Engineering Peer Review 5 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 6 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 7 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 8 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 9 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 10 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 11 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 12 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 13 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 14 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 15 UCB, Oct. 17, 2003

THEMIS - Canada

• Motivation for Canadian Involvement

• Prototyping the ASI

• Data Retrieval and Distribution

• Canadian Involvement

• Data Management• Summary • Searching, Sophisticated Analysis• Real-time Website

• Building on Existing Infrastructure

THEMIS/GBO Engineering Peer Review 16 UCB, Oct. 17, 2003

THEMIS/GBO Engineering Peer Review 17 UCB, Oct. 17, 2003

Deployment

THEMIS/GBO Engineering Peer Review 18 UCB, Oct. 17, 2003

Existing “backbone” internet (BEST) ATHA, CONT (Ekati), CHUR, INUV, FSMI, ….ATHA, CONT (Ekati), CHUR, INUV, FSMI, ….

Local ISP & RF Link

[A]DSL Telus, Manitoba Telecom, Aliant, …..

Hardcopy via traditional mail

3k 1k/yr

5k 9k/yr

0.1k 1k/yr

8k 3k/yr

FSIM, INUV, FSMI, RANK, PBQ (2005+), …FSIM, INUV, FSMI, RANK, PBQ (2005+), …

PGEO, ATHA, FLIN, PINA, KAPU, HEBR, CART, ….PGEO, ATHA, FLIN, PINA, KAPU, HEBR, CART, ….

Telesat HSI Canadian Satellite

NOT INUVNOT INUV

“VSAT” 45k 1k/yr All sitesAll sites

Glentel, Nanometrics, PolarSat, Stratos, ….

Never fails (well almost never); Not at all sexy!Never fails (well almost never); Not at all sexy!

Sta

ndar

d In

tern

et

Dial up to local ISP

All sites; Not real time; Not very sexy.All sites; Not real time; Not very sexy.

0.1k 1k/yr

1k 1k/yr

Telus Cell Phone (Really)

Many sites in BC and Alberta (MacMurray, P. George, High Level ….)Many sites in BC and Alberta (MacMurray, P. George, High Level ….)

0.1k 1k/yr

Data Recovery

THEMIS/GBO Engineering Peer Review 19 UCB, Oct. 17, 2003

• 5 imagers 2003-2004 • 5 imagers 2004-2005• 6 imagers 2005-2006

• Mike Greffen (Team Leader)• Trond Trondsen• Mikko Syrjäsuo• Brian Jackel• Hercules Olivier• Eric Donovan• Igor Voronkov• P. T. Jayachandran

Canadian Personnel Involved in Deployment

Deployment Milestones for Canadian ASIs

DeploymentDeployment

THEMIS/GBO Engineering Peer Review 20 UCB, Oct. 17, 2003

U. Calgary Heritage & Team

• ISIS II Scanning Photometer• Other ground-based instruments (MSPs; FP; Spect.; First ASI, POCA)• CANOPUS (ASI; MSPs; Communications Infrastructure)• Viking UV• Freja UV• Interball II UVAI• Nozomi TPA• Numerous rocket optical and particle instruments• NORSTAR• MIRACLE ASI array (experience via Mikko Syrjäsuo)• Portable Auroral Imager; High Speed Photometer; Induction Coil Mags

Brief History of Relevant U. Calgary Space Group Experience

THEMIS/GBO Engineering Peer Review 21 UCB, Oct. 17, 2003

Canadian THEMIS ASI Team

• Eric Donovan (Team Leader).• Brian Jackel, Trond Trondsen (ASI specialists; Space Scientists)• Mikko Syrjäsuo (ASI specialist; Machine Vision)• Noora Partamies (PDF – as of 2004)• Leroy Cogger (senior advisor)• Mike Greffen (Deployment, Site Manager, Operations, Public Outreach)• Emma Spanswick (riometers, NORSTAR/THEMIS Web Site Project Manager)• Hercules Olivier (Electrical Engineering; Communications; Operations)• Cliff Marcellus, Greg Enno, Peter King (Technical/Engineering support as needed)

U. Calgary

Athabasca U.• Martin Connors (Athabasca Geophysical Observatory - PI).• TBD (Technician and Site Manager)

Contractors• Fokke Creutzberg (Keometrics – MSP Specialist)• Don Wallis (Magnametrics – Riometer Specialist)

THEMIS/GBO Engineering Peer Review 22 UCB, Oct. 17, 2003

THEMIS - Canada

• Motivation for Canadian Involvement

• Prototyping the ASI

• Data Retrieval and Distribution

• Canadian Involvement

• Data Management• Summary • Searching, Sophisticated Analysis• Real-time Website

• Building on Existing Infrastructure

THEMIS/GBO Engineering Peer Review 23 UCB, Oct. 17, 2003

Starlight Xpress MX716 w/all-sky optics

Linux Red Hat 9 w/MX716 driver

Image interval 5 sec; 1-sec exposures

NORSTAR/MIRACLE imaging code

Web server provides real-time access

Network time protocol

Instrument & Housing

Deployed at Athabasca Summer ‘03

Assessment by Calgary Team Fall ‘03

THEMIS/GBO Engineering Peer Review 24 UCB, Oct. 17, 2003

Athabasca Alberta

Deployment

THEMIS/GBO Engineering Peer Review 25 UCB, Oct. 17, 2003

http://www.phys.ucalgary.ca/NORSTAR/themis/data/Movies/

athabasca_2003_10_04.mov

THEMIS/GBO Engineering Peer Review 26 UCB, Oct. 17, 2003

THEMIS - Canada

• Motivation for Canadian Involvement

• Prototyping the ASI

• Data Retrieval and Distribution

• Canadian Involvement

• Data Management• Summary • Searching, Sophisticated Analysis• Real-time Website

• Building on Existing Infrastructure

THEMIS/GBO Engineering Peer Review 27 UCB, Oct. 17, 2003

Existing “backbone” internet (BEST) ATHA, CONT (Ekati), CHUR, INUV, FSMI, ….ATHA, CONT (Ekati), CHUR, INUV, FSMI, ….

Local ISP & RF Link

[A]DSL Telus, Manitoba Telecom, Aliant, …..

Hardcopy via traditional mail

3k 1k/yr

5k 9k/yr

0.1k 1k/yr

5k 3k/yr

FSIM, INUV, FSMI, RANK, PBQ (2005+), …FSIM, INUV, FSMI, RANK, PBQ (2005+), …

PGEO, ATHA, FLIN, PINA, KAPU, HEBR, CART, ….PGEO, ATHA, FLIN, PINA, KAPU, HEBR, CART, ….

Telesat HSI Canadian Satellite

NOT INUVNOT INUV

“VSAT” 45k 1k/yr All sitesAll sites

Glentel, Nanometrics, PolarSat, Stratos, ….

Never fails (well almost never); Not at all sexy!Never fails (well almost never); Not at all sexy!

Sta

ndar

d In

tern

et

Dial up to local ISP

All sites; Not real time; Not very sexy.All sites; Not real time; Not very sexy.

0.1k 1k/yr

1k 1k/yr

Telus Cell Phone (Really)

Many sites in BC and Alberta (MacMurray, P. George, High Level ….)Many sites in BC and Alberta (MacMurray, P. George, High Level ….)

0.1k 1k/yr

Data Recovery

THEMIS/GBO Engineering Peer Review 28 UCB, Oct. 17, 2003

Data Recovery

THEMIS/GBO Engineering Peer Review 29 UCB, Oct. 17, 2003

THEMIS - Canada

• Motivation for Canadian Involvement

• Prototyping the ASI

• Data Retrieval and Distribution

• Canadian Involvement

• Data Management• Summary • Searching, Sophisticated Analysis• Real-time Website

• Building on Existing Infrastructure

THEMIS/GBO Engineering Peer Review 30 UCB, Oct. 17, 2003

CGSM - NowCGSM - Now

THEMIS/GBO Engineering Peer Review 31 UCB, Oct. 17, 2003

CGSM - EnhancedCGSM - Enhanced

THEMIS/GBO Engineering Peer Review 32 UCB, Oct. 17, 2003

THEMIS - Canada

• Motivation for Canadian Involvement

• Prototyping the ASI

• Data Retrieval and Distribution

• Canadian Involvement

• Data Management• Summary • Searching, Sophisticated Analysis• Real-time Website• CDF files & FTP tree

• Building on Existing Infrastructure

THEMIS/GBO Engineering Peer Review 33 UCB, Oct. 17, 2003

Station

Imager

Filters

Summary Images

Calibration Data

Classification

Top 60

Entire Dataset

Name, location, mag midnight, history of operation, …

Rel

atio

nal D

atab

ase

--

130

GB

Online publicly accessible storage of complete high-res data for subset of nights of greatest interest for mission scientific objectives (ie., the Top 60 Substorms).

Great big pile of data online & in hardcopy at UCalgary and at UCB. This will not be culled.

THEMIS ASI UCalgary Data Management

THEMIS/GBO Engineering Peer Review 34 UCB, Oct. 17, 2003

Mapping

THEMIS/GBO Engineering Peer Review 35 UCB, Oct. 17, 2003

Thumbnails

THEMIS/GBO Engineering Peer Review 36 UCB, Oct. 17, 2003

Merging in Real Time

THEMIS/GBO Engineering Peer Review 37 UCB, Oct. 17, 2003

Data Summary

THEMIS/GBO Engineering Peer Review 38 UCB, Oct. 17, 2003

Data Searching

Machine Vision – Content Based Image Retrieval

We are exploring automatic classification of images, rapid retrieval of images from large datasets, and determination of motion of auroral forms, all using sophisticated machine vision techniques.

This classification and CBIR exercise reduces each image to a 48 element feature vector.

THEMIS/GBO Engineering Peer Review 39 UCB, Oct. 17, 2003

A classification of 300000 CANOPUS Gillam ASI images yielded these magnetic local time occurrence distributions for arcs, patches and omega bands. This took about 6 hours on a 1.5 GHz pentium. An equivalent manual survey, which yielded the same results, took 4 months of dedicated effort by a student (who miraculously stayed in the field!). This figure from Syrjäsuo and Donovan [Ann. Geophys., in press].

Data Searching

THEMIS/GBO Engineering Peer Review 40 UCB, Oct. 17, 2003

Optical Flow – a technique in which two-dimensional velocity distributions are obtained from image sequences [Partamies, Syrjäsuo, Janhunen (FMI), Hiebert].

Other techniques under development include…

Tomography – sophisticated technique to infer heights from multiple overlapping fields of view [Jackel, Donovan, Partamies].

Energy Flux Determination – modification Janhunen [2001] method for THEMIS imagers to estimate energy flux [Partamies, Donovan, Janhunen (FMI)].

Data Analysis

THEMIS/GBO Engineering Peer Review 41 UCB, Oct. 17, 2003

Data Distribution

Our contribution to distribution of THEMIS data will be carried out through the THEMIS webpage on NORSTAR-WEB.

We will display real time data products as well as housekeeping information vital to keeping downtime to a minimum.

All summary data, summary images, merged thumbnails, as well as tools for accessing the relational database will be publicly accessible (CDF files on our server FTP tree mirrored at UCB).

The “top 60’’ full time and space resolution image data base will be freely accessible to all THEMIS team members.

The complete full time and space resolution image data base will be publicly accessible via temporary passwords that will be granted in response to specific data requests.

THEMIS/GBO Engineering Peer Review 42 UCB, Oct. 17, 2003

THEMIS CANADA WEB

http://www.phys.ucalgary.ca/NORSTAR/themis/themis_main.html

THEMIS/GBO Engineering Peer Review 43 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 44 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 45 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 46 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 47 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 48 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 49 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 50 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 51 UCB, Oct. 17, 2003

THEMIS CANADA WEB

THEMIS/GBO Engineering Peer Review 52 UCB, Oct. 17, 2003

• Real-time Merged Webpage 2004• Archive structure 2004• Summary database 2005• Relational database 2006

• Eric Donovan• Trond Trondsen• Mikko Syrjäsuo• Brian Jackel

Canadian Personnel Involved in ASI Data Management

Development Milestones for Data Management