speed up your gis server - run gis software on solid-state drives (ssd)
DESCRIPTION
Solid-State Drives (SSDs) are recently becoming popular storage mediums in laptops, and are praised for their speed and shock resistance. However, there has been some debate over whether or not SSDs are ready for the data center. GIS software is known for its high demands on server hardware. Especially hard-drives became a bottleneck when serving map tiles, rendering map images or running spatial computations. SSDs seem to be one possibility to increase IO performance, but optimized server hardware is rare and still very expensive. We contacted a Korean maker specialising in high-performance SSD servers, mainly used to provide video-on-demand services, and we received two of their enterprise servers for testing and benchmarking with FOSS4G software. In this presentation we present our benchmarking results and explain the strong and weak points by comparing traditional hard-drives and solid-state disks.TRANSCRIPT
![Page 1: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/1.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #1
Speed up your GIS serverGIS software on solid-state drives (SSD)
![Page 2: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/2.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #2
Speed up your GIS serverGIS software on solid-state drives (SSD)
Seongbong Kim
Daniel Kastl
![Page 3: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/3.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #3
Motivation
● Involved in pgRouting project● Lots of questions about
performance● pgRouting's bottleneck is
loading data from hard disks into memory
● How to be as fast as Google?
There is no simple answer ...
![Page 4: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/4.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #4
… what about running
PostgreSQL/pgRouting on SSD?
![Page 5: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/5.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #5
Who is using SSD with a laptop?
![Page 6: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/6.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #6
Who is using SSD on a server?
![Page 7: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/7.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #7
What is a solid-state drive?
A solid-state drive (SSD) is a data storage device that uses solid-state memory to store persistent data with the intention of providing access in the same manner of a traditional block i/o hard disk drive.
SSDs are distinguished from traditional hard disk drives (HDDs), which are electromechanical devices containing spinning disks and movable read/write heads.
In contrast, SSDs use microchips which retain data in non-volatile memory chips and contain no moving parts.
http://en.wikipedia.org/wiki/Solid-state_drive
![Page 8: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/8.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #8
SSD vs. HDD (1)
Characteristic Solid-state drive Hard disk drive
Spin-up time short long
Random access time ~ 0.1 ms 5-10 ms
Read latency time short long
Read performance consistent varying
Moving parts no yes
Weight/size smaller larger
Write longevity type dependent unlimited
![Page 9: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/9.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #9
SSD vs. HDD (2)
Characteristic Solid-state drive Hard disk drive
Parallel operation possible no
Shock, altitude, vibration
resistant not resistant
Magnetic susceptibility no possible
Software encryption limited yes
Costs (energy) lower high
Costs (capacity) high “low”
![Page 10: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/10.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #10
MLC vs. SLC
Multi-Level-Cell
● shorter lifespan
(10.000 write cycles)
● higher data density
● slower throughput
● less efficient writing
● less expensive
Single-Level-Cell
● longer lifespan
(100.000 write cycles)
● lower data density
● higher throughput
● more efficient writing
● more expensive
![Page 11: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/11.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #11
Test Server Specification
Model Name Intel(R) Xeon(R) CPU - E5620 @ 2.40GHz
Processors 6
CPU cores 4
Cache Size 12.288 KB
Total Memory 24.675.980 KB (24 GB)
RAID 0
Total SSD 750 GB (SLC)
OS CentOS 5.6
![Page 12: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/12.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #12
Benchmark
![Page 13: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/13.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #13
Data & Tools
● Geodata● Raster: Landsat imagery North America● Vector: OpenStreetMap Planet
● Processing● PostgreSQL import with Imposm● Image processing with GDAL● Routing topology with osm2po● Image rendering with Mapserver (Fast CGI)
![Page 14: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/14.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #14
Ambitious Plans
● File system benchmarks
● WMS map rendering from raster data
● WMS map rendering from vector data (DB)
● Tile seeding from raster data
● Tile seeding from vector data (DB)
● Serving from tile cache
![Page 15: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/15.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #15
Reality
● File system benchmarks
● WMS map rendering from raster data
● WMS map rendering from vector data (DB)
● Tile seeding from raster data
● Tile seeding from vector data (DB)
● Serving from tile cache
![Page 16: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/16.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #16
Benchmarking Tools
● Apache JMeter for WMS
● pgBench for PostgreSQL
● FIO for file system
=> WMS Shootout
![Page 17: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/17.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #17
File I/O Benchmark
Sequential Read Sequential Write Random Read Random Write0
500
1000
1500
2000
2500
3000
3500
4000
16k32k64k128k256k512k1024k
![Page 18: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/18.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #18
![Page 19: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/19.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #19
![Page 20: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/20.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #20
![Page 21: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/21.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #21
![Page 22: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/22.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #22
Use Case - VOD or FTP
Requirements● Mainly reading access● Fast read rate● Serving data
Example● Tile server● Data server
http://www.maptiler.org/
![Page 23: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/23.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #23
Use Case - DB / FS only
Requirements● Combined writing
and reading access● Fast read rate
Examples● Geo-database● Map rendering● Data processing
![Page 24: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/24.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #24
Lessons Learned
● Different field of expertize● Relatively small amount of community
resources● Standards & best practices● Comparability● PostgreSQL/PostGIS – how to tune it right?
… well, everyone asks for benchmarks
![Page 25: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/25.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #25
![Page 26: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/26.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #26
Want to benchmark your Open Source software?
![Page 27: Speed up your GIS server - run GIS software on solid-state drives (SSD)](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5459010caf79594f558b544e/html5/thumbnails/27.jpg)
GIS software on solid-state drives (SSD)FOSS4G2011 - #27
Thank you!
LSD Tech Korea