rfi mitigation for the parkes galactic all sky survey (gass)
DESCRIPTION
RFI mitigation for the Parkes Galactic All Sky Survey (GASS). Peter M.W. Kalberla Argelander-Institut für Astronomie Bonn. Galactic All Sky Survey (GASS). N. M. McClure-Griffiths, D. J. Pisano, M. R. Calabretta, H. Alyson Ford, Felix J. Lockman, L. Staveley-Smith, - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/1.jpg)
Peter M.W. Kalberla Argelander-Institut für Astronomie Bonn
RFI mitigation for the Parkes Galactic All Sky Survey
(GASS)
![Page 2: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/2.jpg)
2
Galactic All Sky Survey (GASS)
McClure-Griffiths et al. (2009)Kalberla et al. (2010)
N. M. McClure-Griffiths, D. J. Pisano, M. R. Calabretta, H. Alyson Ford, Felix J. Lockman, L. Staveley-Smith, P. M. W. Kalberla, J. Bailin, L. Dedes, S. Janowiecki, B. K. Gibson, T. Murphy, H. Nakanishi, K. Newton-McGee, J. Kerp, B. Winkel
![Page 3: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/3.jpg)
3
GASS data final version (-0.12 to 50 K, log scale)
![Page 4: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/4.jpg)
4
GASS: survey parameters• 13 beam receiver• 21-cm line survey of the Galactic HI emission
– Declinations δ < 1 deg – (-500) < -468 < vLSR < +468 < (+500) km/s– Δv = 1 km/s– In-band frequency switching, Δv = 660 km/s
• Beam FWHM 14.4 arcmin • OTF mapping in RA and DEC, two coverages • 2.8·107 spectra, 5 sec dumps, noise ~0.4 K• 10 observing sessions between 2005 and 2006
• FITS maps: noise at full resolution (15.6 arcmin): 60 mK
![Page 5: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/5.jpg)
5
Every Thing You Always Wanted to Know About..
![Page 6: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/6.jpg)
6
Problems
• RFI at fixed frequency without significant variation in time– Causing in many cases negative signals (ghosts)
• Broad lines (Δv ~ 15 km/s) in March 2006
• Bandpass ghosts from HVC gas due to folding
• Footprints: strong RFI signals for short time intervals
• Ringing (Gibbs phenomenon) from correlator
![Page 7: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/7.jpg)
7
First step: Use livedata flags
• LAB data are used for fitting the instrumental baseline
• At that stage it is easy to replace channels flagged by livedata during first stage of reduction with LAB data
• Alternatively flagged data can be interpolated from neighboring channels of Parkes data
• The replacement using LAB is far better!!
• 0.1% of all data affected
![Page 8: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/8.jpg)
8
Remaining RFI: „footprints“
![Page 9: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/9.jpg)
9
Clean
![Page 10: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/10.jpg)
10
Median filter (at any observed position)
• Determine median, mean and rms fluctuations within a radius of 6 arcmin (consistent with HIPASS)
• Find channels that have– High rms scatter (> 3 σrms) and
– Large differences between median and mean (>σm)
• Replace data that deviate > 2 σm from median by median– Do not filter for T > 0.5 K (T > 2 K at b > 10 deg)– Do not filter at positions with continuum > 200 mJy
• 0.07% of all data affected
![Page 11: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/11.jpg)
11
Clean data
![Page 12: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/12.jpg)
12
Observed, flagged RFI replaced
![Page 13: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/13.jpg)
13
Peirce criterion (1852) AJ 2, 161
• Criterion for the rejection of doubtful observations
• Cutoff limit for exclusion of outliers depends on number of available data points
• For 40 profiles (typically) a 2 σrms limit is adequate if about 10% of the data are suspect
• A 1.6 σrms limit would be adequate if about 20% of the data are suspect
• We use a fixed 2 σrms limit with deviations from the median
![Page 14: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/14.jpg)
14
Extra treatment:• Eliminate spectra with high noise (>3 times average)
and with more than 30 flagged channels (0.3% affected)
• Bandpass ghosts can be minimized by median filtering
• RFI in March 2006 (broad Gaussian lines)– Fit parameters– Flag data accordingly – Median filtering as usual RFI
• Emission lines > 2 K– No automatic filtering– Inspect data and filter only those regions that are
affected
![Page 15: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/15.jpg)
15
Reorganize database for computational reasons
• 300 GB sdfits files with 2.8·107 spectra are hard to handle
• Generate compressed random access database– 135 GB in single file, pointer information – fast access of individual profiles
• Benefit of new data format:– Allows fast filtering
– Very fast on-the-fly processing of FITS cubes
![Page 16: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/16.jpg)
16
Stray radiation (the reason for the second data
release)
![Page 17: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/17.jpg)
17
21cm line work and Darwinism• Correction for stray radiation suffers from detailed
observations of the antenna diagram • Antenna parameters:• Model parameters need to be self-consistent• ~60 different runs• Baseline correction:• Code and parameters need to survive • ~50 different versions necessary• RFI mitigation • Comparison of all profiles at any position within 6 arcmin (109 cases)• >2 CPU years in total Hornet magazine, 1871
?
How are the Profiles today?
Does the solution survive?
![Page 18: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/18.jpg)
18
Summary
• RFI post-processing needs redundancy – Typically no more that 25% of the data are bad – Limit: 50%
• Fast data access necessary for filtering– New data format needed (random access)– Advantages: generation of FITS cubes very fast
• Replace bad data by LAB data or by median – Surprisingly simple recipe to use other data
![Page 19: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/19.jpg)
19
This all was about…
RFI in the protected band
But <0.5% of data affected
![Page 20: RFI mitigation for the Parkes Galactic All Sky Survey (GASS)](https://reader035.vdocuments.mx/reader035/viewer/2022062500/5681502e550346895dbe23c0/html5/thumbnails/20.jpg)
20
Dirty stuff you don’t want to see….