which database is better for zabbix ? postgresql vs mysql
TRANSCRIPT
![Page 1: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/1.jpg)
Which Database is Better for Zabbix ? PostgreSQL vs MySQL
Yoshiharu Mori
SRA OSS Inc. Japan
![Page 2: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/2.jpg)
About Myself Yoshiharu Mori
Belongs To: SRA OSS,Inc.Japan
Division: OSS technical Group
Providing support and consulting services
for 30 or more kind of OSS
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 2
![Page 3: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/3.jpg)
Zabbix experience • We started Zabbix support service from 2011
• Last year, we joined Zabbix patnership program.
• Our company systems is monitored by Zabbix.
• We design the monitoring system of our customer’s. o Over 100 devices
o Including virtulization system (vCenter /ESXi)
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 3
![Page 4: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/4.jpg)
Agenda
• Introduction
• Simple test
large number of items
• Partitioning test
large number of items with Partitioning
4 Copyright © 2013 SRA OSS, Inc. Japan All rights reserved.
![Page 5: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/5.jpg)
Which database engine
are you using for Zabbix?
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 5
![Page 6: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/6.jpg)
PostgreSQL is Slow?
• The Performance of PG
was improved from 9.2!
Compared with PG9.1
o Read Query 4x Faster
o Write Query 5x Faster
Reference:
PostgreSQL 9.2 release news
http://www.postgresql.org/about/news/1415/
Robert Haas@PGcon2012
Performance Improvement in PostgreSQL 9.2
http://www.pgcon.org/2012/schedule/events/416.en.html
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 6
![Page 7: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/7.jpg)
MySQL vs PostgreSQL
• Zabbix benchmark with MySQL and PostgreSQL
• Environment
o Amazon EC2 M1 medium instance
o 3.75 Gib RAM
o 2ECU (1 core)
o 410 GB instance storage
o OS: Amazon Linux AMI 2013.03 (64bit)
o 3 servers ( 1 for Zabbix server and DB ,2 for Zabbix Agent)
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 7
![Page 8: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/8.jpg)
Software • Zabbix 2.1.1 ( alpha release)
• Use latest release version for the components
o MySQL 5.6.12
o PostgreSQL 9.2.4
o Apache httpd 2.4.6
o PHP 5.4.17
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 8
![Page 9: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/9.jpg)
Test Target • Simulate 600 hosts ( 2 agent servers )
• 26400 items
( 44 items/1 host,5 sec interval, keep history 1day)
• 10200 triggers ( 17 triggers / 1 host )
• Starting with empty history
• Monitoring for:
4 hours (short run)
36 hours (long run)
• Zabbix:
modified CacheSize=64M in zabbix_server.conf
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 9
![Page 10: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/10.jpg)
Simple Test • Test 1
o Default DB setting
• Test 2 o Same buffer size and Transaction log size
• Each Test is o Monitoring 4 hours
o comparing CPU utilization
and Zabbix performance
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 10
![Page 11: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/11.jpg)
Test 1: default DB Config • MySQL
o character-set-server=utf8
o skip-character-set-client-handshake
o innodb_file_per_table
• PostgreSQL
o ALL default
PostgreSQL uses files per table
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 11
![Page 12: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/12.jpg)
Test 1: CPU utilization MySQL PostgreSQL
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 12
PG: Heavy io wait
User System iowait
![Page 13: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/13.jpg)
Test 1: Zabbix Performance MySQL PostgreSQL
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 13
zabbix[wcache,values] :Values processed by Zabbix server per second
zabbix[queue]:Zabbix queue
PG cannot keep up
with processing
![Page 14: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/14.jpg)
Test 2: Tuning • Same Buffer and Transaction log size
• MySQL
o Innodb_buffer_pool_size = 512MB
o Innodb_log_file_size = 256MB
x innodb_log_files_in_group=2(default)
= log size 512MB
• PostgreSQL
o shared_buffers = 512MB
o checkpoint_segments = 32
16MB/each segmnet →log size 512MB
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 14
![Page 15: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/15.jpg)
Test 2: CPU utilization MySQL PostgreSQL
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 15
Almost same load (PG is slightly stable and low io)
User System iowait
![Page 16: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/16.jpg)
Test 2: Zabbix Performance MySQL PostgreSQL
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 16
zabbix[wcache,values] :Values processed by Zabbix server per second
zabbix[queue]:Zabbix queue
![Page 17: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/17.jpg)
Compare DB Size
•Test 2 Case
•Exclude Transaction log
•MySQL : 3.3 GB
•PostgreSQL : 3.2 GB
•Almost Same Size
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 17
![Page 18: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/18.jpg)
Study of Simple Test
• MySQL default setting is better than PostgreSQL.
• By Increasing Buffer and Transaction log size,
MySQL and PostgreSQL are
almost same performance.
• The size of DB is same in this Simple Test.
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 18
![Page 19: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/19.jpg)
Characteristic of Zabbix DB
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 19
Large write and small read
![Page 20: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/20.jpg)
Tuning Point • Checkpoints tuning is very important in write case.
o Incorrect Checkpoints increase write load
o The opportunity of checkpoints are
• full of Buffers
• full of Transaction log
o To Improve Checkpoint,
it’s important of Buffer and Transaction log tuning
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 20
![Page 21: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/21.jpg)
Zabbix DB Tuning • Zabbix DB continue write operation by same rate.
• Estimate the growth rate of DB size
In this test case 3.2GB/4h ≒ 70MB/5min
• Set a plenty of buffer and transaction log size from a
growth rate of DB size.
In this test case
512MB buffers and 512MB transaction log size
Copyright © 2013 SRA OSS, Inc.
Japan All rights reserved. 21
![Page 22: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/22.jpg)
Partitioning Test • Same DB Tuning
o Buffer and Total Transaction log size 512MB
• Test 3
o Non-Partitioning and housekeeper is enabled.
o Monitoring 36 hours
• Test 4
o Partitioning daily and housekeeper is disabled.
o Monitoring 36 hours
• Comparing CPU utilization and Zabbix performance
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 22
![Page 23: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/23.jpg)
Test 3: CPU utilization MySQL PostgreSQL
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 23
Housekeeper’s Large io wait
User System iowait
![Page 24: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/24.jpg)
Test 3: Zabbix Performance MySQL PostgreSQL
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 24
zabbix[wcache,values] :Values processed by Zabbix server per second
zabbix[queue]:Zabbix queue
PostgreSQL is more stable in heavy io situation.
![Page 25: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/25.jpg)
Test 4 Partitioning • Disable Housekeeper
• Daily partitioning at history* tables.
• PostgreSQL supports partitioning o via table inheritance
o define a trigger or rule.
for redirecting data inserted into the parent table
to the appropriate child partition.
o The performance is also depend on the trigger or rule.
o In this test case, we define the trigger
• Using pl/pgsql
• Using C language (heap_insert method)
https://github.com/matheusoliveira/pg_partitioning_tests
Developed by Matheus de Oliveira
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 25
![Page 26: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/26.jpg)
Test 4: CPU utilization MySQL PG(pl/pgsql)
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 26
PG(pl/pgsql) need more 3x CPU(user) power
User System iowait
![Page 27: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/27.jpg)
Test 4: Zabbix Performance MySQL PG(pl/pgsql)
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 27
zabbix[wcache,values] :Values processed by Zabbix server per second
zabbix[queue]:Zabbix queue
In spite of using more CPU, PG is still stable.
![Page 28: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/28.jpg)
Test 4: CPU utilization MySQL PG(C lang)
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 28
The cpu usage of PG(C lang) is better than PG(pl/pgsql),
But it is worse than MySQL.
User System iowait
![Page 29: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/29.jpg)
Test 4: Zabbix Performance MySQL PG(C lang)
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 29
zabbix[wcache,values] :Values processed by Zabbix server per second
zabbix[queue]:Zabbix queue
Almost same performance
![Page 30: Which Database is Better for Zabbix ? PostgreSQL vs MySQL](https://reader030.vdocuments.mx/reader030/viewer/2022012017/615b9b70d21e582daf121e7b/html5/thumbnails/30.jpg)
Conclusion • In our test
o PostgreSQL and MySQL is almost
same performance as Zabbix DB.
o Tuning Checkpoint(buffer and transaction log)
is important.
o MySQL’s partition is easy and better cpu
usage than PostgreSQL.
(PostgreSQL’s partition supports foreign key)
o PostgreSQL is more stable in heavy io situation.
Copyright © 2013 SRA OSS, Inc. Japan All rights reserved. 30