transitioning low earth orbit satellite archive data from informix (geodetic datablade) to...
TRANSCRIPT
![Page 1: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/1.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS)
Churngwei Chu [email: [email protected]], Fenny Wang, Walt Baskin, and Kay RoweScience Systems and Applications, Inc. Hampton, VA
and
Juliet PaoNASA Langley Research CenterAtmospheric Science Data CenterHampton, VA
September 26, 2007
National Aeronautics and Space Administration
www.nasa.gov
![Page 2: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/2.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS)National Aeronautics and Space Administration
Outline• Introduction
• Geodetic Model vs Geospatial Model (PostGIS)
• Steps to Project a Polygon in Geodetic Model to a 2-D map
• Results
• Summary & Future Work
![Page 3: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/3.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 3National Aeronautics and Space Administration
Atmospheric Science Data Center
Radiation Budget
Clouds
Aerosols
Tropospheric Chemistry
• Providing data services for over 42 Projects• In 2006, 251 Terabytes of data distributed to 20,500 customers in 176 countries• >2 Petabytes of data in the archive• Increasing rate of ~28 Terabytes/month
Processing, Archiving, and Distributing Earth Science Data at NASA Langley Research Center
http://eosweb.larc.nasa.gov
![Page 4: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/4.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS)National Aeronautics and Space Administration
NASA's Earth Observation System (1/3)
EOS Spacecraft
EOS Ground Stations
![Page 5: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/5.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS)National Aeronautics and Space Administration
NASA's Earth Observation System (2/3)
Data Processing
&MissionControl
Data Transport to Data Centers
![Page 6: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/6.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS)National Aeronautics and Space Administration
NASA's Earth Observation System (3/3)
Data Centers
InstrumentTeams/SIPS
Research Users
Education Users
Value-Added Providers
International Partners & Data Centers
Interagency Data Centers
Internet
![Page 7: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/7.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 7National Aeronautics and Space Administration
ANGē (Archive Next Generation)• Distributed, multi-tier, service-based architecture enabling new methods for managing science
data
• Goals
Merge two systems into one.
Improve efficiency and automation.
Use commodity hardware (PCs) to reduce hardware and maintenance cost.
Use Open Source Software such as PostgreSQL/PostGIS, JBoss and GNU compilers.
![Page 8: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/8.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 8National Aeronautics and Space Administration
Sample Data File
![Page 9: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/9.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 9National Aeronautics and Space Administration
Data Flow for Informix
Data
GeoBox((-60,80),(60,-160),ANY,(1993-12-31 22:30:00,1994-01-01 01:30:00))
Informix Geodetic Datablade
GRing
![Page 10: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/10.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 10National Aeronautics and Space Administration
Geo-object Types– Polygons
– Points
– Bounding boxes
– Line strings
Operations
Intersection between a query bounding box and above data types.
![Page 11: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/11.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 11National Aeronautics and Space Administration
Challenges of Moving Data to PostGIS• Informix Geodetic Datablade uses geodetic model – spherical coordinate.
• Geospatial Model (PostGIS) uses projections – planar coordinate.
• Invalid polygons created by GRing stored in Informix.
• GRing still generates some invalid polygons.
![Page 12: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/12.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 12National Aeronautics and Space Administration
Geodetic Model - Polygon• “A line segment is represented as a geodesic. On an ellipsoidal datum, a geodesic is the
shortest path between two points.” -- Informix Geodetic DataBlade Module User's Guide
• “You must specify the vertices in a ring that encloses a region so that the interior of the polygon is on your left as you traverse each ring from vertex to vertex.”
-- Informix Geodetic DataBlade Module User's Guide
![Page 13: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/13.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 13National Aeronautics and Space Administration
Sample Data Across Anti-meridian in Geodetic Model
![Page 14: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/14.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 14National Aeronautics and Space Administration
Cylindrical Projection in Geospatial Model
![Page 15: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/15.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 15National Aeronautics and Space Administration
Projected Geoobj from GRing on 2-D map
![Page 16: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/16.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 16National Aeronautics and Space Administration
Polygons in 2-D Map
![Page 17: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/17.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 17National Aeronautics and Space Administration
Sample Data Covering South Pole in Geodetic Model
![Page 18: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/18.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 18National Aeronautics and Space Administration
Cylindrical Projection in Geospatial Model
![Page 19: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/19.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 19National Aeronautics and Space Administration
Projected Polygon on 2-D map
![Page 20: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/20.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 20National Aeronautics and Space Administration
Data Flow for PostGIS
Data GRing
Polygon(.......)
ASDC ConversionProcess
A set of counter-clockwise sub-polygonsA set of clockwise sub-polygonsType of the original projected polygon
PostGIS
![Page 21: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/21.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 21National Aeronautics and Space Administration
Steps to project a polygon into PostGIS Model
1) Project a polygon to 2-D map by adding points on anti-meridian.
2) Determine type of projected polygon:
a line string
a clockwise polygon
a counter-clockwise polygon
3) Partition the mapped polygon into two groups of sub-polygons, clockwise and counter-
clockwise.
4) If the polygon is projected into a line, more edges are added to enclose north pole or south pole which becomes a counter-clockwise polygon.
![Page 22: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/22.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 22National Aeronautics and Space Administration
Polygon Covers South Pole Generated by GRing
![Page 23: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/23.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 23National Aeronautics and Space Administration
Polygon in 2-D Map
![Page 24: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/24.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 24National Aeronautics and Space Administration
Covered Region in 2-D Map
![Page 25: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/25.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 25National Aeronautics and Space Administration
Data Covers both poles in Geodetic Model
![Page 26: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/26.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 26National Aeronautics and Space Administration
Cylindrical Projection in Geospatial Model
![Page 27: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/27.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 27National Aeronautics and Space Administration
Original Polygon From GRing on a 2-D map
![Page 28: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/28.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 28National Aeronautics and Space Administration
After Adding Points on Anti-meridian
![Page 29: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/29.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 29National Aeronautics and Space Administration
Two Clockwise Polygons
![Page 30: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/30.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 30National Aeronautics and Space Administration
Intersection Query• In our PostGIS DB table, we have four columns to keep geo-spatial data, a polygon type,
counter-clockwise sub-polygons, clockwise sub-polygons, and a polygon (text format) in the geodetic model.
• The geometries, points or line strings are stored in the column with counter-clockwise sub-polygons.
• The following is a sample intersection query with bounding box (-40, -40) and (40, 40).
select granule_id from ctest where (((polygon_type > 1) and intersects(geoobj_ccw, GeomFromText('POLYGON((-40 -40, -40 40, 40 40, 40 -40, -40 -40))', -1)) and (geoobj_cw is null or not contains(geoobj_cw, GeomFromText('POLYGON((-40 -40, -40 40, 40 40, 40 -40, -40 -40))', -1)))) or (polygon_type = 1 and ((intersects(geoobj_ccw, GeomFromText('POLYGON((-40 -40, -40 40, 40 40, 40 -40, -40 -40))', -1)) or (geoobj_cw is null or not contains(geoobj_cw, GeomFromText('POLYGON((-40 -40, -40 40, 40 40, 40 -40, -40 -40))', -1))))))) order by granule_id;
![Page 31: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/31.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 31National Aeronautics and Space Administration
Results
139913990(60, 160)(89.9, -160)5
145314530(-89.9, -120)(-60, -60)4
141814171(60, 60)(89.9, 120)3
1413139815(-40, -40)(40, 40)2
1807179413(-40, 120)(40, -120)1
No. of returned records from PostgreSQl/PostGIS
No. of returned records from
Informix
No. of Different Records
Bounding Box
![Page 32: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/32.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 32National Aeronautics and Space Administration
Summary• Improved geospatial queries on a global scale.
Future Work• Enhance Performance of the algorithm and the queries.• Move our geo-spatial data in Sybase Space Query Server to Postgres/PostGIS.• Integrate with other open source GIS utilities and applications to expand ASDC user community.
![Page 33: Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) Churngwei Chu [email: Churgwei.Chu-1@NASA.GOV],](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649f035503460f94c172ae/html5/thumbnails/33.jpg)
Transitioning Low Earth Orbit Satellite Archive Data from Informix (Geodetic DataBlade) to PostgreSQL (PostGIS) 33National Aeronautics and Space Administration