making elephants fly - openpower foundation · 2019-03-01 · power database files x86. cold...
TRANSCRIPT
![Page 1: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/1.jpg)
Join the Conversation #OpenPOWERSummit
Jez Wain, Escala Competency Centre Atos, Grenoble
Making Elephants Fly
![Page 2: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/2.jpg)
Not so long ago
Power
x86
image: www.thegreenhead.com
![Page 3: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/3.jpg)
RISC vs. CISC
![Page 4: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/4.jpg)
![Page 5: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/5.jpg)
https://media-cdn.tripadvisor.com/media/photo-s/04/12/37/ed/gatlinburg-photography.jpg
Data lock-in
![Page 6: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/6.jpg)
image: http://pinterest.com/pin/121878733635791126
![Page 7: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/7.jpg)
Power
![Page 8: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/8.jpg)
Power x86
Common data format
Linux-el
![Page 9: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/9.jpg)
Power x86
Common application data format
Linux-el
![Page 10: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/10.jpg)
Common application data format
Power Database Files x86
![Page 11: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/11.jpg)
Cold Standby
Power Database Files
![Page 12: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/12.jpg)
Cold Standby
Power Database Files
![Page 13: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/13.jpg)
Cold Standby
Power Database Files x86
![Page 14: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/14.jpg)
Cold Standby - Fail-back
Database Files x86Power
![Page 15: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/15.jpg)
Cold Standby
Database FilesPower
![Page 16: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/16.jpg)
Improving the recovery time and recovery point
![Page 17: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/17.jpg)
PostgreSQL Operation
Database Files
WALWALWAL
![Page 18: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/18.jpg)
Ubuntu/x86Ubuntu/Power
Hot/Warm Standby
WALWALWAL Archived WAL
WALWALWAL
Transaction
Stream
Log
Shipping
WAL Directory
![Page 19: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/19.jpg)
x86
Power
Web Server
Read + Write
Read Only
Log ship & TX stream
192.160.160.170
10.197.160.58
![Page 20: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/20.jpg)
INSERT INTO rtdata (ts, ip, value)
VALUES(now(), inet_server_addr(), 0)
![Page 21: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/21.jpg)
ID Timestamp DB IP Value
1631 2018-05-14 19:37:22 192.160.160.170 0
1630 2018-05-14 19:37:16 192.160.160.170 0
1629 2018-05-14 19:37:11 192.160.160.170 0
1628 2018-05-14 19:37:01 192.160.160.170 0
ID Timestamp DB IP Value
1631 2018-05-14 19:37:22 192.160.160.170 0
1630 2018-05-14 19:37:16 192.160.160.170 0
1629 2018-05-14 19:37:11 192.160.160.170 0
1628 2018-05-14 19:37:01 192.160.160.170 0
Both servers running
- The web-server is writing on main database
- The main database is replicated on the hot standby server.
- The web-server can read from both databases.
![Page 22: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/22.jpg)
ID Timestamp DB IP Value
1713 2018-05-14 19:44:12 192.160.160.170 0
1712 2018-05-14 19:44:07 192.160.160.170 0
1711 2018-05-14 19:44:02 192.160.160.170 0
1710 2018-05-14 19:43:57 192.160.160.170 0
Primary DB halted
- Data still available on secondary (read-only) - Server cannot write until secondary promoted to primary
2018-05-14 19:44:17.390369+02 Cannot execute query INSERT INTO rtdata (ts, ip, value) VALUES(now(), inet_server_addr(), 0)
![Page 23: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/23.jpg)
ID Timestamp DB IP Value
1723 2018-05-14 19:46:19 10.197.160.58 0
1722 2018-05-14 19:46:14 10.197.160.58 0
1721 2018-05-14 19:46:09 10.197.160.58 0
1720 2018-05-14 19:46:04 10.197.160.58 0
Standby promoted to primary
- After a few seconds the hot-standby is promoted as main db server. - Data are now written through the hot standby server (10.197.160.58)
![Page 24: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/24.jpg)
![Page 25: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/25.jpg)
Postgres-XL: Sharded PosgreSQL
- Scale-out, shared-nothing cluster
X86
Postgres-XL
X86 X86 X86
Coordinator
Global transaction manager
Data nodes
![Page 26: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/26.jpg)
Postgres-XL: Sharded PosgreSQL
- Scale-out, shared-nothing cluster
X86
Postgres-XL
X86 X86 X86 Power Power
![Page 27: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/27.jpg)
Postgres-XL: Sharded PosgreSQL
- Scale-out, shared-nothing cluster
Postgres-XL
Power Power
![Page 28: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/28.jpg)
PostgreSQL on Power9 Performance Benchmark
![Page 29: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/29.jpg)
- TPC-B style transactional workload
- Five queries: SELECT, UPDATE, INSERTRead-only (SELECT) option
- Simulates multiple clients
pgbench
![Page 30: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/30.jpg)
pgbench scaling factor
Table # rows
pgbench_branches 1
pgbench_tellers 10
pgbench_accounts 100,000
pgbench_history 0
Scale = 1
Scale Rows in accounts DB size
100 10,000,000 1.5 GB
300 30,000,000 4.5 GB
1,000 100,000,000 15.0 GB
3,000 300,000,000 45.0 GB
![Page 31: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/31.jpg)
Power9 14 cores, 64GBRHEL 7.5 4.14
PG Bench Injector
PG Bench Injector
PG Bench Injector
EMC Unity 300 (Flash)
NVMe
16GB/s FC
Postgres 10.5 & 11 beta4
pg_bench: scale 1000 = ~15 GB Select (read-only)
![Page 32: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/32.jpg)
~700MB/s
8Gb FC Saturated
![Page 33: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/33.jpg)
Tran
sact
ions
per
sec
ond
120K
220K
320K
420K
520K
620K
# Clients16 32 48 64 80 96 112 128 144
SCALE 1000 SCALE 1500 SCALE 2000 SCALE 2500
Power9: 14 cores 40 GB RAM
![Page 34: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/34.jpg)
https://www.enterprisedb.com/blog/enterprisedb-postgres-plus-and-lenovo-3850-x6-equals-performance-and-scalability
![Page 35: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/35.jpg)
Splendid Data – PostgresPURE on IBM Power System S822L benchmark report
16
7 Results
7.1 PostgreSQL 9.4.4 results
PostgreSQL 9.4.4 Scale 100 Scale 300 Scale 1,000 Scale 3,000
1:1 2:1 1:1 2:1 1:1 2:1 1:1 2:1
incl.
conn
Clients/Threads 26/26 48/24 28/28 52/26 30/30 52/26 28/28 48/24
TPS /Latency 423,148/0.123
379,866/0.126
406,589/0.138
377,233/0.138
379,414/0.158
368,224/0.141
365,974/0.153
355,882/0.135
excl.
conn
Clients /Threads 124/124 260/130 90/90 292/146 30/30 244/122 28/28 48/24
TPS /Latency 726,022/0.749
709,271/0.819
435,311/0.589
566,129/1.017
380,335/0.158
446,477/0.859
366,383/0.153
356,229/0.135
0"
50000"
100000"
150000"
200000"
250000"
300000"
350000"
400000"
450000"
0" 20" 40" 60" 80" 100" 120" 140"
TPS$includ
ing$conn
ec.o
n$establishing$
Clients$
TPS$5$PostgreSQL$9.4.4$
scale"100"
scale"300"
scale"1000"
scale"3000"
0"
0,2"
0,4"
0,6"
0,8"
1"
1,2"
0" 20" 40" 60" 80" 100" 120" 140"
Latency(in(m
illise
cond
s(
Clients(
Latency(in(ms(0(PostgreSQL(9.4.4(
scale"100"
scale"300"
scale"1000"
scale"3000"
Sébastien Chabrolles IBM Montpellier + Splendid Data Power8: 20 cores 256 GB RAM
![Page 36: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/36.jpg)
0
12500
25000
37500
50000
INTEL E7-4890 2.8GHz (60c)
PostgreSQL 9.6
POWER8 3.42GHz (20c)
PostgreSQL 9.5
POWER9 3.3GHz (14c)
PostgreSQL 11beta4
X4
TPS/per core
![Page 37: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/37.jpg)
Tran
sact
ions
per
sec
ond
26K
30K
34K
38K
42K
46K
50K
54K
# Clients16 32 48 64 80 96 112 128 144 160
EXT4 nobarrier pgbench -j8
Read-Write - scale 1,00014 cores [email protected] SMT8, 40GB RAM, shared_buffers=24GB
![Page 38: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/38.jpg)
E4-900 : 14 cores [email protected] 40GB RAM PostgreSQL11beta4 pgbench -S with shared_buffers=8GB
Tran
sact
ions
per
sec
ond
200K
300K
400K
500K
600K
700K
# Clients
16 32 48 64 80 96 112 128
SMT4SMT8
SMT4 vs SMT8 on POWER9
![Page 39: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/39.jpg)
Leading performance and cost optimized for Big Data and Analytics workloads
Standardize on LinuxEscala supports industry standard Linux distributions from RedHat, Suse and Ubuntu along with applications optimized for emerging business challenges such as Big Data & analytics, mobile and Java platforms. The support for little endian Linux ensures easy portability for thousands of Open Source applications. Leading ISV’s such as SAP also support the Linux on Power platform.
OpenStack Cloud enabling / automationThe PowerVC management solution turns Escala VMs, storage and virtual networks into a OpenStack based IAAS cloud solution. PowerVC groups multiple physical servers into managed pools, greatly simplifying the creation and deployment of AIX and Linux VMs. Features like Snapshots and cloning reduce the average deployment time of applications from several hours to just a few minutes. Automation can be integrated or triggered by popular Open Source tools such as Chef or Puppet.
The constant economic pressure to reduce IT infrastructure costs while increasing flexibility through dynamic, on-demand provisioning has driven many organizations towards standardization onto x86 hardware and the Linux operating system.As price/performance has become the key measure for infrastructure purchases, TCO calculations are mainly conducted in isolated domains such as hardware, OS/virtualization and support costs. However, bigger picture aspects such as the cost impact on ISV licenses, platform reliability, OS vulnerability and VM isolation are often not factored in.Escala delivers key advantages which can have a significant overall TCO impact such as a leading performance per core, the best reliability in the industry, superior security isolation and virtualization efficiency.
Escala Mission Critical Linux serversEscala Mission Crit ical Linux servers are designed for commercial workloads requiring performance, reliability and scalability. The Escala L1-800 and L3-800 servers use the latest Power8 processor technology delivering unprecedented performance, scalability, reliability, and manageability, for demanding commercial workloads. They are optimized to help deliver new solutions and services faster, and with higher quality. They are also optimized for big data and analytics, and provide the ideal foundation for scale-out data and cloud environments in a compact 2U package. They deliver superior throughput compared to x86-based offerings for similar workloads.
Escala OpenPower-based Linux servers Escala OpenPower-based Linux servers are optimized for Big Data and cloud workloads. The Escala L1-OP80 and L3-OP80 Power8 servers are built for most OpenSource workloads, as well as OpenSource databases, due to their cost efficiency and expandability. They are designed to run either Hadoop environments thanks to larger internal storage capacity and powerful SMT8 cores, or in-memory analytics such as Spark, as well as innovative solutions from the OpenPower partner ecosystem such as CAPI. They are also ideal for hyperscale ISP environements where high-efficiency and cost-effectiveness are the most important features.
Atos and Linux advantages:• Market leading memory bandwidth
• Slim packaging with high internal disk expandability
• Ideal for Big Data as a Hadoop node thanks to internal disk expandability and powerful SMT8 cores
• Ideal for Spark, OpenSource databases (PostgresSQL, MariaDB..) and Business applications (SAP)
discover the open alternative: Escala linux solutions
escala mission critical servers
![Page 40: Making Elephants Fly - OpenPOWER Foundation · 2019-03-01 · Power Database Files x86. Cold Standby - Fail-back Database Power Files x86. Cold Standby Database ... PostgresPURE on](https://reader033.vdocuments.mx/reader033/viewer/2022060401/5f0e266f7e708231d43dd9b6/html5/thumbnails/40.jpg)
https://disneymovieyear.files.wordpress.com/2013/01/photo20.jpg