elkingyour postgresql database infrastructure€¦ · >what’s need to be done on the postgres...

Infrastructure at your Service.

Elking your PostgreSQL Database Infrastructure


30.06.17Elking your PostgreSQL Database Infrastructure

About me

Arnaud BerbierSenior Consultant

+41 79 128 91 45

[email protected]

Agenda

1.Playground Infrastructure

2.Elastic Search Stack

3.PostgreSQL Monitoring

4.Conclusion

30.06.17


Playground Infrastructure


> Big Picture

> Use case and deployment process

Big Picture

30.06.17



Elastic Stack 5.4.1 Postgres Clusters 10.0 PGPool II

PG Standby Node

PG Primary Node PGPool Node 0

PGPool Node 1

Hea

rtbe

at

SSH192.168.56.0/24

Version 5.1.18 r114002 (Qt5.6.2)

HealthCheck

Version 2.3.0

Stre

amin

g R

eplic

atio

n

vmtestpgsql02192.168.56.210/32

vmtestpgpool02192.168.56.230/32

vmtestpgpool01192.168.56.220/32

Kibana

Elastic Node 0

Elastic Search

Beats

Beats

Beats

vmtestpgsql01192.168.56.200/32

vmtestelastic01192.168.56.100/32

Ship

Ship

Inge

st

Visualize

Use case and deployment process

30.06.17



Database Clusters

Elastic Stack Server

Dashboard import

Metric Agent Deployment

PGPool Metric Agent Deployment

Elastic Search Stack


> What does ELK mean ?

> What’s a Beat ?

> Is all the stack Open Source ?

> How much costs the cloud offering ?

> ELK is a complete monitoring platform - Elastic Search Stack

> At the beginning Elastic Stack was commonly known as ELK

> E - Elastic Search> Store, Search and Analyze

> L – Logstach> Collect, Enrich and Transport

> K – Kibana> Explore, Visualize and Share

What does ELK means ?

30.06.17



DATA DATADATA DATA

INFORMATIONDATA

DATATRANS

INFORMATION

What does ELK means ?

30.06.17



Kibana

Elastic Search

Logstash Beats

Elastic Cloud

Graph

Alerting

Monitoring

Reporting

Security

User InterfaceVisualization

> It evolved to be a full Monitoring Stack

Store, Index & Analyzse

Ingest

X-Pack

> Lightweight Data Shippers to ingest data into Elastic

What’s a Beat ?

30.06.17



Is all the stack open source ?

30.06.17



> Components support over subscriptions


30.06.17



Open Source Basic Gold Platinum

Elas

ticSe

arch Elasticsearch

Kibana

Beats

Logstash

X-P

ack Monitoring

Multicluster

Configurable Data retention

X-Se

curi

ty Authentication + Encryption

Role-Based access ctrl

Audit

Encryption at rest support

> Components support over subscriptions

> Details : https://www.elastic.co/subscriptions


30.06.17



Open Source Basic Gold Platinum

X-P

ack

Alerting via Watcher

Reporting

Graph Analytics & Visualization

Dev Tools – Search Profiler

Supp

ort Support Coverage BH 24x7

Response TimesCrit. 4hL2: 1dL3: 2d

Crit. 1hL2: 4hL3: 1d

Unlimited # of Incidents

> Elastic Cloud Offering – Host Elastic Search and Kibana

How much cost the cloud offering ?

30.06.17



PostgreSQL Monitoring


> Postgres Cluster Settings

> What’s need to be done on the Postgres Clusters> Filebeat log harvesting

> The elastic stack process owner needs access to the Postgres log files> log_file_mode in pg_settings parameters

Postgres Cluster Settings

30.06.17



postgres=# select name, setting, unit from pg_settings where name like '%log_file%' or name like '%log_dir%';

name | setting | unit

----------+---------------------------------------------+------

log_directory | /app/postgres/admin/dbixchange01/pg_log |

log_file_mode | 0776 |

log_filename | postgresql-%a.log |(3 rows)

> This correspond to file mode 666

> What’s need to be done on the Postgres Clusters> Metric Beat postgresql

> A dedicated user for DB connection > Allowed access to the database

> Elastic servers need to be accessible by the deployed beat agents> Firewall> Network

Postgres Cluster Settings

30.06.17



> Easy to deploy> Ingest lots of data of any log type> Already have predefined metric such as

Pro & contra

30.06.17

Conclusion

> Most of the required features needs subscription



30.06.17

We look forward to working with you!


Arnaud BerbierSenior Consultant

+41 79 128 91 [email protected]

Any questions? Please do ask

elkingyour postgresql database infrastructure€¦ · >what’s need to be done on the postgres...

Documents