october 8, 2010 - session 2refag2010 paris1 distributed processing systems for large geodetic...
TRANSCRIPT
October 8, 2010 - session 2 REFAG2010 Paris 1
Distributed processing systemsfor large geodetic solutions
IAG WG 1.1.1“Comparison and combination of precise orbits
derived from different space geodetic techniques”
Henno Boomkamp
October 8, 2010 - session 2 REFAG2010 Paris 2
Introduction• Objectives of IAG WG 1.1.1
1. Study systematic errors between orbits based on different data2. Improve models and solution strategies3. Develop the means to allow the above
• Key solution mechanism: simultaneous analysis– simultaneous (re-)processing of geodetic datasets is either free of
biases, or allows estimation & analysis of inter-system biases
• Concrete targets– Analysis of all GPS / GNSS data in a single solution Dancer– Analysis of all geodetic data in a single solution Digger
October 8, 2010 - session 2 REFAG2010 Paris 3
POD m
odule
Digger (… reprocessing)- data types: all- long arc (years)- long latency (1 week)- key technology: BOINC
Digger (… reprocessing)- data types: all- long arc (years)- long latency (1 week)- key technology: BOINC
Consistency among techniques
Dart (Dancer-RTK)- data types: GNSS only- very short arc (RT)- zero latency- key technology: BURST
Dart (Dancer-RTK)- data types: GNSS only- very short arc (RT)- zero latency- key technology: BURST
Real-time access to accurate ITRF
ITRF
Dancer- data types: GNSS & VLBI- short arc (48 hrs) - short latency (30 min)- key technology: JXTA
Dancer- data types: GNSS & VLBI- short arc (48 hrs) - short latency (30 min)- key technology: JXTA
Large scale, direct access ITRF
Three WG projectsEffort: Dancer 90 %
Digger 9 % Dart 1 %
October 8, 2010 - session 2 REFAG2010 Paris 4
The trouble with GPS• Limited processing capacity of AC
– Current IGS approach would require hundreds of Analysis Centres …
• Most data not available at short latency– Reduces statistical quality ITRF– Reduces relevance of ITRF
20,000 permanent GPS receivers
5,000 public
400ITRF
• If we want to run a conventional batch LSQ solution for all receivers– Distribution over many computers is inevitable– Geographical separation of the computer cluster is inevitable
• If we want to run a conventional batch LSQ solution for all receivers– Distribution over many computers is inevitable– Geographical separation of the computer cluster is inevitable
October 8, 2010 - session 2 REFAG2010 Paris 5
10 GPS sites = 10 PC = 10 AC
October 8, 2010 - session 2 REFAG2010 Paris 6
Dancer overviewDancer brings the analysis to the data rather than vice versa • LSQ solution implemented as a peer-to-peer process on the internet• Based on existing JXTA P2P software (SUN Microsystems)
Natural separation of analysis is by receiver, not by AC• Geographical distribution of data is at the level of receivers• Solution becomes scalable in the number of stations• 99% of estimated parameters can be pre-eliminated at receiver level
… the required computers are readily available!• Every permanent receiver is connected to a local or remote computer• Most of these computers do not do anything apart from RINEX ftp• Processing capacity is perfectly collocated with the data owners
October 8, 2010 - session 2 REFAG2010 Paris 7
Separation of LSQ in tasks per receiver (1)Local parameters are pre-eliminated at the receiver:
N
A
N
A
NtN
t
tN
y
y
yy
x
x
xx
M
M
M
B
B
BBBBA
2
1
2
1
2
1
2
1
21
.
00
0
0
00Ax
global (orbits, sat clocks, pole)
jx
local j = 1 … N
N
jjAA
1
N
jAjA yy
1
AtjjjjjjjA
tj xByMxyxMxB
1
N
jjjjAjA
N
j
tjjjj yMByxBMBA
1
1
1
1
N
jjA
N
jjjA
N
jj yxBxA
111
With 10,000 receivers:Total solution• 30 million parameters• 170 million observationsAfter pre-elimination• 90,000 global parametersSingle process• 15,000 non-zero eq.
With 10,000 receivers:Total solution• 30 million parameters• 170 million observationsAfter pre-elimination• 90,000 global parametersSingle process• 15,000 non-zero eq.
October 8, 2010 - session 2 REFAG2010 Paris 8
Global normal equation represents the average equation of all receivers
Dancer averages diagonal D before solution, and the vector afterwards
Same solution , thanks to distributive property of multiplication with
Separation of LSQ in tasks per receiver (2)
N
jj
N
jj bxM
11
N
1
N
1bxM~~
N
jj
N
jj FD
NFDMM
N 11
~1~~~1
jjjj xbxFD
~xxx
N
N
jj
~1
1
xFbDxbxFD ~~~~~~ 1
x
jjjjjjj xFbDxbxFD
1~~
x 1~ D
October 8, 2010 - session 2 REFAG2010 Paris 9
Averaging N vectors without a central serversquare dance algorithm
• N/2 pairs can be formed by toggling one bit of each number 1...N
011010110 111010110• Pair-wise exchange of vectors: both computers find the same sum
x11010110 first bit has now become irrelevant…
• New exchange pairs are formed by toggling the second bit
011010110 101010110
xx1010110 first two bits are now irrelevant; etc…
• Some additional operations are necessary:– folding nodes are introduced to make N an exact power of two
– N splits into 50% core nodes and 50% spare nodes for contingencies
After log2N exchange cycles, all N computers have the same vectorAfter log2N exchange cycles, all N computers have the same vector
October 8, 2010 - session 2 REFAG2010 Paris 10
core node data volume (1-way)
0.0
50.0
100.0
150.0
200.0
250.0
1 10 100 1000 10000 100000
48h / 30s
24h / 30s
48h / 300s
24h / 300s
arc length/ epoch rate
MB
N
October 8, 2010 - session 2 REFAG2010 Paris 11
Sched beta
Dancer project status
October 8, 2010 - session 2 REFAG2010 Paris 12
Summary• Rigorous LSQ solutions for all GPS receivers are possible
– workload can be distributed over (some) existing hardware• Dancer has no data centres, analysis centres, combination centres, product centres, central bureau…
– anonymous participation avoids political issues of data access– differences regional vs. global reference frames disappear
• GNSS receivers become smart receivers– Dancer process can be embedded on future receiver hardware– Smart receiver generates products, not (just) observations
• Other distributed processes to follow• DIGGER simultaneos reprocessing of all geodetic techniques• DART RTK layer on top of Dancer for global access
October 8, 2010 - session 2 REFAG2010 Paris 13
www.GPSdancer.com
.. .
..
. . . .
... .
..
. ..
.
..
.
..
.
.
..
.
..
.
.
.
.
.
.
.
.
.
for • further information• more details on solution mathematics• download latest version of the software• check project status• e-mail contacts and web links
for • further information• more details on solution mathematics• download latest version of the software• check project status• e-mail contacts and web links
Dancer screenshot