meerkat science processing

MeerKAT Science Processing

Jasper Horrell, SKA SA Postgrad Bursary Conference, Stellenbosch – Dec 2010

Its 2014, imagine ….

2014 Vision

Back to the present:

Renewed 2010 focus on “science processing” with a dedicated team – fairly early days. 6 people currently plus some part-time inputs.

Two main focus areas as reflected in the approved large survey projects: a) imaging; b) non-imaging. Is there anything else?

Imaging: SP focus on data processing and archive post correlation. Define, design and build the facility systems to support this over next few years.

Non-imaging: Current SP focus on raw data capture, offline correlation, data analysis, pulsars, VLBI capture, SETI.

Interface to the approved large survey project teams re data products, formats, pipeline development, etc. (Meeting earlier today!)

SIMULATIONS, prototypes, involvement in early science projects, etc.

A bit more to this than “using CASA for data reduction”

Science Processing (SP)

MeerKATData Flow

KAT-7 – our playground

Observation Planning/Tracking

Getting there - not quite

Getting there – remote ops!

New Cape Town Office (cont.)

KAT-7 Early Fringes (2009)

KAT-7 Horizon Masks

KAT-7 Horizon Masks

KAT-7 Images (2010)

Moonsize

4-dishes

Started archiving visibility data from KAT-7 in Dec 2009

To date, 1230 data captures saved to archive

Approx 490 GB visibility data so far

On average 40 data captures per week (March to Oct, excluding RFI campaign)

Trivia: Fornax A and Pic A are most observed targets

KAT-7: Archive To Date

Archive will need more and more attention as data requirements evolve

Already seeing the need to support multiple data sources and formats – Fringe Finder, KAT-7 and holography

Currently reviewing and testing Apache OODT components to assess viability for MeerKAT data catalog and archiving

ASTRON and NRAO also have systems of interest….

Early days….

The threat of raw data!!!

KAT-7: Archive Future

KAT-7 Data Archive

First site test of raw data capture (full band 400 MHz for a few secs plus GPU-based digital down-converter for “indefinite” period)

First VLBI data capture run (HartRAO fringe test)

First SETI data capture for Project Dorothy

300 GB recorded

Also, “Picture A”

KAT-7: Last week on site

2 x 10 GbE NICs

2 x GTX 460 2GB GPUs

2 X 6 Core 3.0 GHz Xeons

16 x 2 TB HDD

24 GB RAM

The Beast - “Big Gulp?”

Pic A: Hot Off the Press!

Pic A: Hot Off the Press!

Pic A: Hot Off the Press!

Pic A: Hot Off the Press!

Pic A: Hot Off the Press!

Pic A: Hot Off the Press!

Exploring Polarimetric Cal

Exploring Polarimetric Cal !!

where (even) Oleg fears to tread…!!

Exploring Polarimetric Cal

Exploring Polarimetric Cal

Calibrated visibilities

Certain standard pipelines (spectral line image cubes, continuum images)

Storage for visibility data (project disk quotas?)

A 10 PB archive on site plus 3 PB in CT, plus likely European mirror

Archive access and facilities to reprocess from archive

Flexible architecture for “black belt” users plus space in the Array Processor Data Centre for additional user-supplied equipment

Support for some re-use of existing mature packages where possible (support CASA, MeqTrees).

Proposal management etc.

SP: MeerKAT to provide (?) :

Basic data capture framework to HDF5 file archive format and MS export writing capability (currently being extended for full KAT-7 correlator)

Simple archive for commissioning purposes

SCAPE commissioning package

First imaging software

Simulation and data reduction tests in Meqtrees

Prototype online system based on ROACH sampling, followed by GPUs pre-processing and disk storage. Allows for raw data recording for commissioning, VLBI, software correlation, SETI

Emerging NRAO and ASTRON collaborations

Some great early results on the KAT-7 system

A lot of work to be done over the next few years (we won’t be bored).

SP: In place now

Tests of streaming framework (online system) prototype for KAT-7 with new 16-input KAT-7 correlator (KAT-8 ready!) – now in lab testing phase, site deployment soon’ish

More mature imaging on KAT-7 using both CASA and MeqTrees. Aim for 7-antenna hand-crafted test image using both packages and compare. Also try “pipeline mode” on new data (expect poor initial results).

Coarse grain parallelization investigations (for CASA)

GPU gridding (for CASA)

Simulations for MeerKAT using primary beams (sky de-rotation effects etc). CASA and Meqtrees. Aim for SKA memo.

Beam polarization characterization on ATA and KAT-7

Continue RFI characterization and mitigation work (RFI database plus GPU flagger)

Ongoing HDF5 scalability testing

Start of some pulsar tools on KAT-7 (port GUPPI to ROACH2 plus GPU?)

More detailed design for MeerKAT (archive, data model etc)

Small VLBI and SETI projects on KAT-7

Engagement with large survey project teams!!

SP: Next few months

• Major joint effort needed over coming years to ensure that the science teams hit the ground running

• Also need to support commissioning and smaller PI proposals

• Technology within reach for MeerKAT scale imaging

• Fine tuning (e.g. imaging pipeline) will take time

• Technology challenges in pulsar search area

• KAT-7/8 will continue to be an excellent engineering and science test bed

• Plan re-use software where possible, but expect some custom developments

• Lots to do. Technical team will need to grow a bit. Project teams can help (a lot).

Conclusions

http://www.ska.ac.za

Questions…

jasper @ ska.ac.za

Credits (full SP Team):

Thomas BennettMattieu de Villiers

Jasper HorrellTshaks NamaliliSimon Ratcliffe

Ludwig Schwardt