background image: 960x540 pixels - send to back of slide and set to 80% transparency

CFLOGS bosh release (LT)Etourneau Gwenn@The_Shinji62

Me ….

Etourneau Gwenn

Senior Solution Architect @Pivotal (Cloud Foundry Superman)

https://twitter.com/the_shinji62

L,,, https://github.com/shinji62

Cloud Foundry logging stack

3

4

Almost no bufferIn memory buffer If not consume fast enough you lose logs.

You lose LOGS

Cloud Foundry logging stack

Syslog

5

KAFKA

Syslog

Cloud Foundry logging stack

Why Kafka ?

6

● Kafka ‒ Can ingest million of logs by second‒ Persistency‒ More easy to consume‒ A lot of community around Kafka

Losing logs is still possible

No real 100% no loss solution but this should help

CF logs

7

● One command deploy bosh deploy● Release including

‒ KAFKA‒ Firehose kafka nozzle (fork from the Rakuten one)‒ Kafka-manager‒ CF syslog forwarder

CF logs

8

● Create a kafka topic for every events type‒ LogMessage, ValueMetrics and so on ..

● Forwarder to syslog is included ‒ Can forward to any syslog endpoint‒ Include Apps info. Org etc ..

CF logs Bosh release

9

https://github.com/shinji62/cflogs-boshrelease