getting modern with logging via log4perl

Getting Modern with Logging via Log4perl

Dean HamsteadSydney PMJuly [email protected]

Logging

Despite how much time we spend greping and tailing logs (or both), logging is:● Always overlooked● Rarely done well

○ Inconsistent message formats (“started foo”, “foo ended”)

○ Unhelpful messages (“Error 0x1234 occurred”)○ Stacktraces???○ Organic growth

● Reinvented over and over again

Why are we acting against our own

self-interest?

Perl Logging 101.

● warn() and die(),● collected via cron, apache etc.

● Similarly, print STDERR $foo

Better than nothing? Chaos in apache as everything goes to the fallback error.log

Perl Logging 202.

● open(my $log,’>>’, ‘foo.log’)● print $log “my message”● $debug && print $log “debugging message”

For extra marks:● sub log { print $log localtime, @_,”\n” }● log(“my message”)● sub debug {} etc...

Perl Logging 203

● package MyApp::Log;● use MyApp::Log qw/ log /;● log(“my message”)

or● $log = MyApp::Log->new();● $log->message(“my message”)(Then horrible stuff starts happening like passing the $log object around)

Decisions, Decisions

Log::Log4perl● Inspired by Log4j● Modular● Its own config file● Optionally in-line

configured● Available via CPAN

or your distro packaging

Log::Dispatch● It’s own design● Module● In-line configured● Available via CPAN

or your distro packaging

Also check out...

● Log::Any - which tries to let you not care as much

● Log::Contextual● Log::Agent - looks perfectly suitable too● Sys::Syslog - it may be enough for you● Apache2::Log - may also be just fine

Let’s just stick with Log4perl...

Installation# The cpanm way

cpanm Log::Log4perl

# The Debian/Ubuntu/Mint way, where we say log and perl twice

# as a sort of incantation. Don’t say it in the mirror though.

apt-get install liblog-log4perl-perl

# The Fedora/RHEL/CentOS way

yum install ‘perl(Log::Log4perl)’

# The FreeBSD way

pkg install p5-log-log4perl

Basic Usageuse Log::Log4perl;

Log::Log4perl->init('my.log4perl.conf');

# defaults to pwd

$logger = Log::Log4perl->get_logger('My::Instance');

# Call the above over and over, you’ll always get the

# same object. i.e. it’s a Singleton

$logger->info('Foo');

$logger->fatal('Bar');

etc...

Basic Usage ++

Have Log4perl watch the config, reload and reconfigure automagically on the fly:

Log::Log4perl->init_and_watch()

Or wait for HUP signal with:

Log::Log4perl->init_and_watch($conf_file, 'HUP');

Logging LevelsLifted from Log::Log4perl pod:There are six predefined log levels: FATAL, ERROR, WARN, INFO, DEBUG, and TRACE (in descending priority). Your configured logging level has to at least match (>=) the priority of the logging message.If your configured logging level is WARN, then messages logged with info(), debug(), and trace() will be suppressed. fatal(), error() and warn() will make their way through, because their priority is higher or equal than the configured setting.

Logging Levels (cont.)

The 6 basic logging levels have corresponding methods:

$logger->trace("..."); # Log a trace message $logger->debug("..."); # Log a debug message $logger->info("..."); # Log a info message $logger->warn("..."); # Log a warn message $logger->error("..."); # Log a error message $logger->fatal("..."); # Log a fatal message

Logging Levels (cont.)Because, why have just one way?

use Log::Log4perl::Level;

$logger->log($TRACE, "..."); $logger->log($DEBUG, "..."); $logger->log($INFO, "..."); $logger->log($WARN, "..."); $logger->log($ERROR, "..."); $logger->log($FATAL, "...");

This is actually useful for cleanly & concisely varying the log level based on some logic appropriate to your program.

Austerity is a good thing!This is expensive and useless if errors aren’t logged: $logger->error("Erroneous array: @super_long_array");

So we can check before leaping into the cold murky waters: if($logger->is_error()) { $logger->error("Erroneous array: @super_long_array"); }

Here is the whole family: $logger->is_trace() # True if trace messages would go through $logger->is_debug() # True if debug messages would go through $logger->is_info() # True if info messages would go through $logger->is_warn() # True if warn messages would go through $logger->is_error() # True if error messages would go through $logger->is_fatal() # True if fatal messages would go through

This example lifted from Log::Log4perl pod

Not true in Log4perl!

Forget: $logger->fatal($err) && die($err);

Use: $logger->logdie($err);

Instead of: warn($message); $logger->warn($message);

Try: $logger->logwarn($message);

Shortcuts!

Also you get for free:

● Log event time and date● System hostname● Pid of process● Line number where called● Package/Class caller● Chomping, multi-line alignment● OS-independant newline● Milliseconds since program started● Milliseconds since last log event● Much much more!

Log4perl <3 Carp

Functions that, in addition to logging, also pass the stringified message to their companions in the Carp package:

$logger->logcarp(); # warn w/ 1-level stack trace $logger->logcluck(); # warn w/ full stack trace $logger->logcroak(); # die w/ 1-level stack trace $logger->logconfess(); # die w/ full stack trace

Configuration: log4perl.conf

● By default, use a standalone config file● Softens the observer problem● Handy with config. management systems

and development. Install the file appropriate to the environment

● Keep your own personal library of log4perl configs. Copy them in place as needed.

A Bit of Theory

● As we have seen, Log4perl provides standard methods

● In the config file, select the minimum log level then direct it to an Appender

● An Appender is basically an output● An Appender can optionally apply a Filter● Most Appenders allow you to specify a

layout for your log messages

Basic File Output############################################################

# A simple root logger with a Log::Log4perl::Appender::File

# file appender in Perl.

############################################################

log4perl.rootLogger=DEBUG, LOGFILE

log4perl.appender.LOGFILE=Log::Log4perl::Appender::File

log4perl.appender.LOGFILE.filename=/var/log/myapp/myapp.log

log4perl.appender.LOGFILE.mode=append

log4perl.appender.LOGFILE.layout=PatternLayout

log4perl.appender.LOGFILE.layout.ConversionPattern=[%d] %F %L %c - %m%n

Basic Screen Output############################################################

# A simple root logger with a Log::Log4perl::Appender::File

# file appender in Perl.

############################################################

log4perl.rootLogger=DEBUG, LOGFILE

log4perl.appender.LOGFILE=Log::Log4perl::Appender::Screen

log4perl.appender.LOGFILE.layout=PatternLayout

log4perl.appender.LOGFILE.layout.ConversionPattern=[%d] %F %L %c - %m%n

Other Appenders

● DBI - make your logs the DBA’s problem● String - because perl● RRDs - graphs impress management● ScreenColoredLevels - like Screen but with colors● Socket - why not?3rd Party...● SMTP - Why not flood your inbox?● Gearman, RabbitMQ - flood your SOA● Chunk::Store::S3 - because AWS solves everything● ElasticSearch(::Bulk) - because no one ever got fired for

using ELK● Journald - shove it into Lennart Poettering’s monster

When to log?

● Program starts, stops, awakens or sleeps● Opening files, sockets etc.● Before and after retrieving URL’s● Done reading or calculating an important

value● Before and after decisions● When exceptions are caught● Log more details when something bad

happens (insomuch as we can anticipate it)

My Rules of Thumb. Log when...

How much logging in Production?What’s happening when no one’s watching...

Do not log when...● You’re a discrete general purpose module ● Instead, “throw exceptions” (i.e. die() or carp()) and make the caller play

catch. Let them log if needed.

my $obj = Foo->new();

eval { $obj->action() };

if ($@) { # handle $@

● Or, return null and provide an error inspection method or variable

$obj->action() or die(‘Bad thing: ‘, $obj->errormsg);$obj->action() or die(‘Bad thing: ‘, $Foo::errormsg);

● Keep it simple. Write shy modules and avoid side effects.

Using Log4perl in a larger program

● Frameworks tend to have it inbuilt or via plugin: Catalyst, Net::Server for example. Profit!

● Don’t pass the $logger object around, remember that Log4perl is a Singleton!

Example 1

package MyApp::UtilityMethods;

use Log::Log4perl;

# Log::Log4perl::init() called in main::

my $log =

Log::Log4perl->get_logger(__PACKAGE__);

sub action {

$log->debug(q/Running Action/);

}

Example 2package MyApp::Some::Base;

use Log::Log4perl; # Log::Log4perl::init() is called in main::

sub log { return $self->{_log} }

sub new {

my ( $p, @a ) = @_;

my $c = ref($p) || $p;

my $self = bless {}, $c;

$self->{_log} = Log::Log4perl->get_logger(ref $self);

$self->log->debug( q|I'm here| );

return $self

}

Example 2 (cont.)package MyApp::Some::Thing;

use parent qw/ MyApp::Some::Base /;

sub action {

my $self = shift;

$self->log->debug(q| Running Action |);

}

Planning

● Have clear guidelines for severity levels

Planning (cont.)

● Let Log4perl take care of metadata line the timestamp, hostname, pid, package name etc.

$log->debug(‘Program: starting up at ’

. localtime()

$log->debug(‘starting up’);

Questions?Thanks for listening

getting modern with logging via log4perl

Software