| Log management as a service Simplify Log Management

How to Make Logging Even More Feature Rich By Using a Queueing Server

Matthew Setter

| Log management as a service Simplify Log Management

• Do you want some form of local persistence?

• Do you need additional processing capabilities?

Is Your Logging Good Enough?

image copyright i k o http://goo.gl/vEWrxW

Logging is essential. But sometimes you can make life easier,

perhaps better, by extracting the logging logic

| Log management as a service Simplify Log Management

Queueing Servers Extract Your Logging Logic?

| Log management as a service Simplify Log Management

1. They allow apps to respond quicker

2. They introduce fully asynchronous functionality to backend systems

3. They increase reliability and uptime

Why Use a Queueing Server?

image copyright Oberazzi http://goo.gl/ZRGP47

| Log management as a service Simplify Log Management

•  Applications can focus more on what they’re designed for

•  Applications can be more responsive user needs

•  Logging can be managed by a separate application

•  Logging can be scaled up and down as needed

Other Benefits of Queueing Servers

image copyright Ken Fager http://goo.gl/kQ0q2E

| Log management as a service Simplify Log Management

What is a Queueing Server?

A queueing server has multiple queues which can hold different types of information, or messages. Messages are added to the queue by processes often referred to as producers and removed from the queue by processes known as consumers.

| Log management as a service Simplify Log Management

How Does a Queueing Server Work?

image copyright Surendra Pulagam http://goo.gl/go5rJ3

A process, called a producer, adds a message to a queue

A process, called a consumer, retrieves the message from the queue, then processes the information contained in the message

| Log management as a service Simplify Log Management

Item Description

Message The message to be logged. e.g., { operation: “read-user-account”, status: “OK” }

Priority The message priority level

Params Any extra information, such as formatting, memory observation and so on.

What is a Message Object

| Log management as a service Simplify Log Management

<?php require('vendor/autoload.php'); use Pheanstalk\Pheanstalk; $pheanstalk = new Pheanstalk('127.0.0.1'); $queueName = "log-messages-queue"; $pheanstalk->useTube($queueName)->put( json_encode( [ 'message' => 'Unable to connect to MySQL database', 'priority' => 'debug', 'params' => [ 'logMemoryUsage' => true, 'format' => '%datetime% - %channel% - %level_name% - %message%\n', ] ] ) );

Code Example Adding a Message To a Queue

| Log management as a service Simplify Log Management

Now There’s a Message In The Queue

| Log management as a service Simplify Log Management

<?php require 'vendor/autoload.php'; use Pheanstalk\Pheanstalk; $pheanstalk = new Pheanstalk('127.0.0.1'); $queueName = "log-messages-queue"; $pheanstalk->watch($queueName); while ($job = $pheanstalk->reserve()) { $messageData = json_decode($job->getData()); $log = getLoggerService($messageData); $logLevel = "add" . ucfirst(strtolower($messageData->priority)); $log->$logLevel($messageData->message); $pheanstalk->delete($job); }

Code Example Extract, Log & Delete The Message

| Log management as a service Simplify Log Management

•  Check Your Benchmarks - speed may not be related to logging

•  Don’t use a queueing server before you need to •  Analyze & debug your applications •  Check if your language or 3rd party library

natively supports queueing or queueing-like functionality

Caveats

| Log management as a service Simplify Log Management

Queuing Servers Can: •  Allow for more efficient logging •  Help reduce application overhead •  Help increase responsiveness •  Make applications more maintainable

In Conclusion

| Log management as a service Simplify Log Management

Want Your Logging Approach to Result in More Insights?

About Us:

Loggly is the world’s most popular cloud-based log management solution, used by more

than 5,000 happy customers to effortlessly spot problems in real-time, easily pinpoint

root causes and resolve issues faster to ensure application success.

Let Loggly help you with the heavy lifting!

Visit us at loggly.com or follow @loggly on Twitter.

Try Loggly FREE for 30 days

| Log management as a service Simplify Log Management

Did you like this presentation?

Head over to our blog for

more great content!

Take me to the Loggly Blog