data mining and visualization in radio astronomybkent/talks/bkent_vis2010.pdf · visualization...

Data mining and Visualization in Radio Astronomy

Brian R. KentNational Radio

Astronomy Observatory

http://www.cv.nrao.edu/~bkent/

Visualize 2010

STScI and NASA

STScI and NASA

Katie Chenoweth et al.

Springel et al.

Abazajian et al 2009 and the SDSS Survey

New Astronomy Across the Spectrum

More pixels Faster computing More data

Sensitive “cameras” Larger surveys Higher resolution

Jorden 2008 (Astrophysics Detector Workshops)

Visualization challenges

• Sheer file and database size!• Variable reduction and Analysis efficiency• Viewing modes for 3D and 4D data• Cross correlation of cataloged data• Efficient database mining• Mosaicking and projection algorithms

Center for Cosmo Physics

Use of Montage for large mosaicking…

Montage allows:

Online combination of imaging surveys

Limited online combination of user images

Unlimited combination programmatic interface

Berriman et al. 2006

Brogan et al. 2009 arXiv:0909.5256v1

The Arecibo Radio Telescope andArecibo L-band Feed Array

The new surveys of Radio Astronomy

• Meridian transit survey strategy.• 7000 square degrees of high galactic

latitude sky with the 7-element Arecibo L-band Feed Array.

• Sensitivity down to 2 x 107 Msun at DVirgo= 16.7 Mpc

• Primary sky targets include Virgo, Leo, and void regions.

• Objectives include public release catalogs and statistics, HI mass function determinations for various environments, and cross-correlation with other wavelength regimes.

Generating ~50 TB of processed datacubesKent et al. 2008, 2009

1225-1525 MHz

The new surveys of Radio Astronomy

• Meridian transit survey strategy.• 7000 square degrees of high galactic

latitude sky with the 7-element Arecibo L-band Feed Array.

• Sensitivity down to 2 x 107 Msun at DVirgo= 16.7 Mpc

• Primary sky targets include Virgo, Leo, and void regions.

• Objectives include public release catalogs and statistics, HI mass function determinations for various environments, and cross-correlation with other wavelength regimes.

Generating ~50 TB of processed datacubesKent et al. 2008, 2009

1225-1525 MHz

Right Ascension=12 hr

Arecibo Large Sky Survey

(Equatorial Coordinates)

FITS to IDL

Bandpass Calibration

Interferenceflagging

2D signal extraction

Data cube gridding

Baseline Flatfield

Data Cube Visualization

3D signal extraction

Flux Measurement

ALFALFA Data Processing Pipeline

Export

Frequency (100 MHz wide)

Tim

e (s

econ

ds)

Distant GalaxyGPSLong range radar

Milky Way

Each Data Element: • 4096 Freq. Channels• 2 polarizations• 7 receivers• 1 second sampling• 600 records• 20736 pixels

GRIDview: Spectroscopy & Visualization analysis tool

• Flexibility for – Native IDL structures– FITS data with astronomy headers

• Suite of OOPs include:– Visualization and imaging procedures– Spectral line measurement and fitting tools– Catalog and database programs

• Extensibility:– Import other user catalogs (ASCII, FITS, XML)– Add other imaging databases (WSDL, XML)

Map display window

Spectral normalization display window

Grid header and coordinate information

Cube manipulation controls

“Slide” to a lower Doppler velocity range

Galaxies are detected…

Nice spiral UGC 8475

Overlay catalogs from various sources…

Examine continuum maps…

Fetch radio images…

“Slide” to a lower Doppler velocity range

Galaxies are detected…

…including this nice spiral!

Back to Spectral mode…

Grab spectrum from a single pixel in the data cube.

Fetch images from online databases

Fetch data from NASA Extragalactic database

Obtain a larger image and position

Query archival databases

Summary• The GRIDview suite is a fast, OOP suite written in IDL

for 3D spectroscopic visualization.• The software is used on several large astrophysical

projects across 40 professional and academic institutions.

• Information: http://www.cv.nrao.edu/~bkent/computing