design issues and implementation challenges

Paul Alexander, Peter HallDesign Issues and Implementation ChallengesAAVP 2010

Design issues and implementation Design issues and implementation challengeschallenges

Paul Alexander and Peter Hall


Aim and scopeAim and scope

•Concentrate on the design issues for SKA1

SKA1 AA-low is a (transformational) world leading instrument Essential to design for SKA1

Consider how to transition to SKA2

•Identify issues which are independent of detailed design Then consider issues which drive detailed design

•Aim is to pose questions that we can aim to make progress on during the course of this meeting

Some questions should be answered

•General point: Transition from a research programme to an instrument project

means we need to retire questions with an accountable path of how and why the decision was reached.


SKA-lowSKA-low• Excellent learning platforms in pathfinders

– LOFAR, MWA, ...– Science, engineering, project management, operational lessons

• Why optimize SKA-low?– Evolving science case

• Possible new specification optimization

– Pathfinders not scalable to SKA-1• e.g. LOFAR x10 > SKA-1 budget

– Rapid technology changes• Verify or change long-standing assumptions

– Cost optimization funds new capabilities• More independent FoVs, increased time domain processing, ...

– Actual SKA site conditions impact SKA-low design significantly


SKA-low designSKA-low design• SKA-low is part of bigger SKA system

– Specifications flow from (updated) SKA Design Reference Mission

– Performance/cost analysis must be done in SKA design environment

– Cost must reflect “total cost of ownership”– SKA environment must capture key AA-lo issues

• SKA operational model is critical to costing, e.g. – Simultaneity of SKA-low & SKA-mid operations

• Data transmission, signal + post-processing– Data archiving– Site infrastructure constraints and costs

• Including energy availability and cost– Support model, and lifetime costs (“maintenance”)


Top level issuesTop level issues

Insensitive to detailed designInsensitive to detailed design


Basic specificationsBasic specifications

•A low-frequency sparse aperture array with A/Tsys of up to 2000 m2/K At what frequency is this optimised (100MHz?) ?

•Operating at frequencies between 70 and 450 MHz At what range of frequencies is this optimised How tight are the constraints both scientifically and technically?

•Array will be centrally condensed but some of the collecting area will be in stations located out to a maximum baseline length of 100 km from the core

What fraction of the collector is on longer baselines? How large is the core?


Possible trade-offs Possible trade-offs (cost constrained design)(cost constrained design)

• Built area vs FoV– More area, or more accessible and/or processed FoV?

• Accessible bandwidth vs sensitivity– Fewer compromises in a narrower band array

• Accessible bandwidth vs polarization capability, polarimetry performance

• Processed FoV, bandwidth vs other parameters– Optimum investment in data transmission, DSP, computing– Investment level as a function of time

• U-V coverage vs other parameters– More stations are costly (e.g. infrastructure, correlation)– Station numbers and size related to calibration strategy (esp.

ionospheric)


Frequency range 6.5:1Frequency range 6.5:1

What frequency range must the array elements be designed/optimised for?

• Approach 1: Observatory

• Aim for best “average” or “uniform” response across the frequency range

• Approach 2: Observatory, but prioritising EoR

• Design antenna for good performance in EoR frequency range

• What is the EoR frequency range 70 – 200 MHz? What about foregrounds?

• Approach 3: EoR instrument with observatory function

• Optimise design for EoR frequency range

• Approach 4: Identify the technical difficulties and relax frequency range

• 100-450 MHz is only 4.5:1


Sky coverageSky coverage

ALMA

45 degree scan

30 degree scan

• Critical design driver for element• Observatory requirement – large sky coverage lower gain antenna larger

scan angle of 45 degrees. What is largest scan angle we would like?• Dedicated EoR experiment perhaps require smaller scan angle higher

gain antenna possible

Circumpolar limit

GC SKA1


AsideAside

SKA1 specification is for an amazing instrument

~ 1 order of magnitude in sensitivity

~ 2-3 orders of magnitude in survey speed


Sensitivity requirementSensitivity requirement

Design specification: 2000 m2/K

f Tsky (K) Aeff (km2)

100 MHz 988 2.1

150 MHz 350 0.70

• We will be building approximately a square kilometre of collecting area

• What sensitivity do we require across the band?

• Very dependent on the frequency at which the array becomes sparse

• Major impact on element design


Tailoring the AA systemTailoring the AA system

100

10

1

100

1000

Frequency (MHz)

Sky

Brig

htne

ss T

empe

ratu

re (K

)

Aeff

Aeff/Tsys

Fully sampled AA-hi

Sparse AA-lo

TskyBecoming sparse

Aeff / T

sys (m2 / K

)

AA frequency overlap

Dishoperation

f AA f max

10000


SKASKA11 sensitivity model sensitivity model

2000 m2/K at 100 MHzTrec = 60 KAA sparse above 150 MHz


TTsyssys across the band across the band

Matching

f Tsky (K)

100 MHz 988

150 MHz 350

180 MHz 221

210 MHz 150

240 MHz 106

400 MHz 29 • Trec important even at 200 MHz

• Dominant at upper end of band

• True low-noise LNAs still important

Challenges: “Matching” across the band to ensure Trec dominated at upper end and Tsky at lower


SKASKA11 survey speed survey speed2000 m2/K at 100 MHzTrec = 60 KAA sparse above 150 MHzNB gives 100 sq degrees across band


Survey speedSurvey speed• What survey speed do we require at fixed Aeff/Tsys?

• Direct implication for cost of correlator and post-correlator processing

• See next section for possible trade off

• Upgrade path

• Increasing survey speed is perhaps easiest designed in upgrade path for AA-low


Data rateData rate

• Data rate and survey speed intimately linked

• Review basic design equations

Re-write in terms of FoV and total collecting area

B

D

Ns Stations


SKASKA11 data rates and configuration data rates and configuration• AA Line experiment 50 AA-low stations

• 100 sq degrees, 10000 channels over 380 MHz bandwidth

3.3 GS/s

• Issues

• What data rate can we process?

• Trade UV coverage (Ns) for FoV and hence survey speed ()

• Line vs continuum requirements

• What is the longest baseline

• What temperature sensitivity do we need and on what scales

Defines filling factor in the core


SKASKA11 configuration configuration

Ideally – do not design in these trade-offs

Need to consider evolution of processing capability in designing configuration

Or even repositionable antenna positions?!


Station and element designStation and element design• One or two elements?

• Many aspects to this – see later

• How sparse can the station be?

• Side lobes even for a random configuration when very sparse

• Complicates imaging, and increases Tsys

• Station size?

• Increasing D reduced UV coverage, reducedprocessing load, less complicated ionospheric model, move DSP from correlator to station B/F


Station designStation design

regular triangular sparse

thinned circular random

Embedded element pattern Random

minimum / 2Random

minimum 2

R1

R2

R3

Nima and Eloy


Configuration, station design Configuration, station design and SKAand SKA22

• Is SKA1 a subset of SKA2

• Should we compromise the design (and hence science return) of SKA1

to ease implementation of SKA2?

• Optimum SKA1 AA-low core may have f ~ 0.5 Dcore ~ 1km.

SKA2 AA-low core is larger with f ~ 1

Almost certainly need to reposition elements on SKA1 SKA2

Do not compromise design of SKA1 maximise science return for SKA1 & accept additional cost in SKA2

Inner (20%)

Core (50%)

Mid (30%)

500 m

2500 m

180 km

Not to Scale

SKA Phase 1 Array Distribution


SKA information and data systemSKA information and data system

Imaging processor

Visibility processors

Science product archive

Local science

reduction

Science proposal

Data product distribution

Data routing

Col

lect

ors

Grid science reduction and visualisation

Monitor and Control system

M&C database

Global and local sky model

Calibration loop

Observation definition

Ae

Ae

Ae

TileProcessor

- hi

TH_0

TH_1

TH_n

TileProcessor

- lo

TL_0

TL_1

TL_m

StationProcessor

0e/o

e/o

e/o

e/o

…..

…..

o/eo/eo/eo/e

o/eo/eo/eo/e

……

.

e/o

e/o

e/o

e/o

Station Processor n

……

.

Lon

g d

istance drivers

…..

o/eo/eo/eo/eo/eo/e

e/oe/oe/oe/o

e/oe/oe/oe/o

Lon

g d

istance drivers

…..

Lon

g d

istance drivers

…..

....

…..

1.0-1.4GHzanalogue

1.0 GHzanalogue

12 f ibre lanes @10Gb/s each

……

…...

12 f ibre lanes @10Gb/s each

10Gb/s f ibre

…..

Max 4 Station Processors

Local Processinge.g. Cal; pulsars

To Correlator

Inputs #: 1296Channel rate:120Gb/s

(raw)Total i/p rate: 1.5 Pb/s

Typical:AA-hi tiles: 300AA-lo tiles: 45Total: 345I/p data rate: 42Tb/s

Notes:1. No control network shown2. Up to 4 station processor systems can

be implemented in parallel3. Data shown are raw, typ. get 80% data

Hierarchical station beam

former


Processing – how much Processing – how much and where?and where?

• For a given sensitivity and survey speed we can decide where and how to

do the processing

Beam forming vs correlation survey speed vs imaging fidelity?

• Physical location of processing

Physically distribute processing only if it leads to a reduction in data

rate


Processing – how much Processing – how much and where?and where?

• For a given sensitivity and survey speed we can decide where and how to

do the processing

Beam forming vs correlation survey speed vs imaging fidelity?

• Physical location of processing

Physically distribute processing only if it leads to a reduction in data

rate – e.g. Station beamformer


Specific Design and Specific Design and Implementation IssuesImplementation Issues


Element and communicationsElement and communications• Can we cover band with a single element?

Where are the compromises?

Can we afford two elements?

• Where do we digitise

Link, power consumption, lightning protection ...

• What is the communication link?

Cost, calibratability and lightning protection

• How is the element powered?

Cost, sustainabilty, manufacturability and deployability

• What is the element assembly and how are they deployed?

Cost, sustainabilty, manufacturability and deployability


Station B/F and correlatorStation B/F and correlator• Station B/F

What is, and can we meet the power budget with an all digital

design?

Do we deploy ASICs in the SKA1 design? If so what are the

timescales for development cycle.

Note cost of Station B/F dominated by number of elements not

how they are deployed (e.g. Station size)

Internal station correlation for calibration?

• Correlator

Is a software correlator possible or desirable for SKA1 or

commissioning?


Post-correlator processingPost-correlator processingWhat is our system concept for SKA1 processing?

o Is the post correlator processing a single Peta-scale

machine or machine designed for our data flow?

Our problem is highly parallel in places and we could

deploy a “UV-processor”

Need to be sure of processing model to

go down this route, but can deliver more

Flops cheaply

Single-pass algorithms will reduce cost do we want

to restrict ourselves in this way?

Subtract current sky model from visibilities using current calibration model

Grid UV data to form e.g. W-projection

Major cycle

Image gridded data

Deconvolve imaged data (minor cycle)

Solve for telescope and image-plane calibration model

Update current sky model

Update calibration model

Astronomicalquality data

UV data store

UV processors

Imaging processors


Cost controlCost control

Item Advantage ChallengeOne element Single core, one RF

chainAdequate performance across band; sparcity at high f

Larger station size Reduce cost of correlator, infrastructure and post-processor

Loss of UV coverage

ASICs deployed in DSP

Power reduction saves on operating budget

Time to deployment, commissioning harder? Loss of flexibility

Custom processing path

Maximise Flops for cost

Loses flexibility


SKA-low implementation SKA-low implementation challengeschallenges

• Low capital cost– N x 100,000 active antennas integrated, reproducible– Strong incentive to incorporate Design for Manufacture early in

development cycle• Low operating cost

– Easily dominates capital cost over life of SKA– Reliability and maintainability are crucial

• Probably dominant aspect of designing “outdoor” portion SKA-low– Robust system is essential

• Damage limitation strategies (lightning etc), intelligent and resilient processing

• Low deployment cost (next slide)• Data processing and archiving prominent in SKA Observatory plan• EMC

– SKA-low is especially vulnerable to poor EMC practices, or poor site management with respect to RFI


Deployment challengeDeployment challenge

• 300,000 elements (or tiles) deployed over 2 years – 1 element/tile every minute!

• Connectivity and commissioning need to keep pace with deployment

• Parallel, industrialized deployment needed– … and during pre-construction

• Substantial site specific and environmental issues• “Design for deployment” essential

– Results in highly modular, maintainable design

design issues and implementation challenges

Documents