the essential perl hackers toolkit

The Essential Perl Hacker's Toolkit

Stephen R. Scaffidistephen@scaffidi.net

YAPC::NA 2012

Thanks for coming!

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

Contents

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

This is a talk about the essentials

essential to...

getting my job done

getting my job done better

getting my job done faster

so I can go out and drink beer

so I can go out and drink beerspend time posting on reddit

so I can go out and drink beerspend time posting on reddit

watching youtube

so I can go out and drink beerspend time posting on reddit

watching youtubeDANCING WITH THE STARS

the editor of the previous three slides has been sacked

I <3 Task::Kensho

This talk != Task::Kensho

But it is essential

So look it up:Task::Kensho

my @essentials

Modules and tools I depend on

make my job easier

make my code better

easier to write

easier to read

easier to maintain

fewer bugs

because...

this is the code I don't write

I'm not that good.

Neither are you*

Neither are you*

*with several exceptions, half of them probablyleaving this room right now...

Previous iterations of this talk...

touchy-feely

waxed poetic about community

gushed about the CPAN

but...

there's simply no time

too many modules

too many modules(on the dance floor)

I prefer Perl because of this variety

No language is perfect

but...

Perl has the CPAN

And the CPAN gives us many, many ways to improve Perl

mst once said...

Perl is my VMCPAN is my language

so...

let's learn about some of the best parts of this language we call the

CPAN

Allow me some hubris...for the next 40 minutes or so.

Bring out the modules!

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

strict

warnings

really?

experience tells me the answer is yes.

English

Get English versions of Perl's “punctuation vars”

use English qw( -no_match_vars );

$#@^ English$\ $OUTPUT_RECORD_SEPARATOR

$$ $PID

$^O $OSNAME

$+ $LAST_PAREN_MATCH

$! $OS_ERROR

$? $CHILD_ERROR

$< $REAL_USER_ID

Examples

perldoc perlvar

autodie

Does what it says on the tin...

# overrides various built-ins...use autodie;

# Now, they all succeed or die!open my $foo, '>', 'file.txt';chdir 'foo/bar';pipe my($baz), my($buh);readline $foo;print “Do you ever check for print errors???”;

autodie is indispensable.

Why write an “or die” clause after every open() when you no

longer have to???

Data::Dumper

Better than print().

Data::Dump::Streamer

Better than Data::Dumper!

List::Util

List::UtilList::MoreUtils

List::UtilList::MoreUtils

Scalar::Util

For those go to Steve Lembark's talk on Util modules tomorrow

morning.

parent

Stop messing with @ISA

# sets up @ISA for youuse parent qw(Some::Base Other::Base);

Clear and declarative!

Try::Tiny

Stop messing with eval {} for exception handling.

use Try::Tiny;

try { something_that_might_die();}catch { handle_error($_); # note: not $@}finally { clean_up();};

IO::All

Maybe not essential

But damn handy!

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

Not just modules

App::cpanminus

A light-weight cpan client that understands local::lib

App::Ack

ack is better than grep

Devel::NYTProf

Hands-down, the best profiler for Perl

Devel::Cover

Tells you how well your tests cover your code

Speaking of tests...

prove

use it to run your tests

Lots of things under Test::

Test::Moreand/or

Test::Most

Many, many others...

Perl::Critic

Make sure your code confirms to certain quality standards.

You do use coding standards, right??

Mooseand/or

Mouse / Moo / Any::Moose

Stop writing your own OO layers

Stop doing it inconsistently

Stop messing up inheritance, destructors, accessors and all

those little things that you have to write over and over and over

again...

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

File::Spec

my $path = join '/', qw(foo bar baz)

What if it runs on Windows?Or VMS?

Or early MacOS?AcornOS?

QNX?

use File::Spec;

my $correct_path =File::Spec->catdir qw(foo bar

baz);

use File::Spec::Functions qw(catdir);

my $path = catdir qw(foo bar baz);

Declarative and correct

Lots of other functions, too:

● canonpath● catfile● rootdir● updir● devnull● tmpdir● curdir

● splitpath● splitdir● catpath● abs2rel● rel2abs● file_name_is_absolute● case_tolerant

Path::Class

use Path::Class qw(dir file); # create dir and file objects...my $dir = dir('foo', 'bar');my $file = file('bob', 'file.txt'); # Stringifies with proper separator on# each supported platformprint "dir: $dir\n";print "file: $file\n";

# methods do what you expect... my $subdir = $dir->subdir('baz');my $parent = $subdir->parent;my $dir2 = $file->dir;

File::Find

Traversing directories in a filesystem is tricky.

You're not that good.

File::Find::Rules

If you don't like File::Find, File::Find::Rules has some very

nice functionality. Try it.

File::Touch

For when you need to touch a file. Simple, and works cross-

platform.

File::Basename

same as the system's basename and dirname commands.

File::Slurp

Simple thing... but which is more readable?

my $text =do { local(@ARGV, $/) = "file.txt"; <> };

or

my $text = read_file 'file.txt';

File::ReadBackwards

When you need it... you DON'T want to do it yourself!

File::Temp

Let this module handle temp files and directories for you and stop

doing it wrong.

File::HomeDir

Don't try to find a user's home directory on your own.

File::Which

Why are you shelling out to which?

STOP IT!

File::Copy

Don't shell out to the system.Don't write your own copy

routine.

Please, PLEASE, just use File::Copy.

File::LibMagicand/or

File::MimeInfo

If you need to recognize a common file type...

STOP DOING IT YOURSELF!!!

If one doesn't work for you, try the other.

File::Path

recursively create and remove directories

File::stat

Can you remember this?

my ($dev, $ino, $mode, $nlink, $uid, $gid, $rdev, $size, $atime, $mtime, $ctime, $blksize, $blocks) = stat $file;

Maybe this?

my $dev = (stat $file)[0];my $ino = (stat $file)[1];my $mode = (stat $file)[2];my $nlink = (stat $file)[3];my $uid = (stat $file)[4];my $gid = (stat $file)[5];my $rdev = (stat $file)[6];my $size = (stat $file)[7];# etc...

Try this, instead...

use File::stat;

my $s = stat $file;$s->dev;$s->ino;$s->size;

Much nicer, no?

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

Tie::IxHashor

Tie::StoredOrderHash

Building your own sucks

Try this:

use Tie::StoredOrderHash qw(ordered);

my $hash = ordered [one => 1, two => 2, three => 3

];

while (my($k, $v) = each %$hash) {print "$k: $v ";

} # one: 1 two: 2 three: 3

Hash::Merge::Simple

Recursively merge hash structures

use Hash::Merge::Simple qw(merge);

my $a = { a => 1, c => 3, d => { i => 2 }, r => {}};

my $b = { b => 2, a => 100, d => { l => 4 }};

my $c = merge $a, $b;

# $c is now {# a => 100, b => 2, c => 3,# d => { i => 2, l => 4 }, r => {} }

Simple, declarative and correct.

Params::Util

use Params::Util qw(_SCALAR _HASH _INSTANCE);

sub foo {my $object = _INSTANCE(shift, 'Foo') or die;my $image = _SCALAR(shift) or die;

my $opts = _HASHLIKE(shift) || {};}

Sure, it's ugly...

But focused on correctly doing something common and

(surprisingly) tricky

Params::Validate

The de-facto standard in parameter validation.

Algorithm::Combinatorics

Don't do combinatorics on your own. Your code will be wrong. It

will be slow. It will suck.

Set::CrossProductor

Set::CartesianProduct::Lazy

Just here for shameless self-promotion.

Data::Compare

Recursively and correctly compare data structures.

Test::Deep::NoTest

More functionality than Data::Compare.

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

DateTime

The most correct handling of dates and times in Perl. Use it.

DateTime::Format::Strptimeand/or

DateTime::Format::DateManip

Use these to transform various time formats from strings into

DateTime objects

Time::Piece

Make localtime() and gmtime() return objects...

use Time::Piece; my $t = localtime;print "Time is $t\n";print "Year is ", $t->year, "\n";

Time::Local

Gives you inverse functions for localtime() and gmtime()

use Time::Local qw(timelocal timegm);

my $localtime = timelocal($sec, $min, $hour, $mday, $mon, $year);

my $gmtime = timegm($sec, $min, $hour, $mday, $mon, $year);

DateTime::Tiny(and friends)

If DateTime is too “heavy” for you, at least use

DateTime::Tiny, or Date::Tiny or Time::Tiny

Time::ParseDate

Parsing dates and times is a PITA. Use a module...

use Time::ParseDate qw(parsedate);

$epoch_time =parsedate("12/11/94 2pm", %options)

Date::Extract

When you need to extract dates and times from arbitrary blocks of

text...

But it tries hard to avoid false-positives

Date::Extract::Surprise

For when you're willing to be surprised at the dates found!

Simply works harder to find dates and times in text, at the cost of

possibly false-positives.

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

YAML::Any

use YAML::Any qw(Dump Load);

$YAML::Indent = 3;

my $yaml = Dump(@objects);

my @objects = Load($yaml);

JSON

use JSON qw(to_json from_json);

my $json = to_json($perl_scalar);

my $perl_scalar = from_json($json);

Text::CSV

use Text::CSV;

my $csv = Text::CSV->new({ binary => 1 });open my $fh, "<:encoding(utf8)", "test.csv";while ( my $row = $csv->getline( $fh ) ) {

$row->[2] =~ m/pattern/ or next; push @rows, $row;}

URIURI::Split

URI::Escape

Spreadsheet::Read

Parse and read in a bunch of different spreadsheet formats,

including Excel.

XML::Simple

For simply reading in and parsing some XML into a data structure.

For non-simple stuff, look at Task::Kensho!

Check Task::Kensho for more XML essentials.

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

Module::Runtime

safely and properly load modules at run-time

IO::Interactive

Do you know how to properly test for an interactive terminal?

DOUBTFUL

IPC::Run

The grand-daddy of running external commands and

controlling the IO

Capture::Tiny

Capture stdout and stderr of any code

It Just Plain Works(tm)

Net::Serveror

Net::Daemon

Writing daemons properly is hard

String::ShellQuote

(my $unsafe_string =q{string you're passing to system}) =~ s/([\/;()\\])/\\$1/g

Unsafe, incorrect, and ugly.

use String::ShellQuote qw(shell_quote);

my $safe_string = shell_quoteq{something you might pass to

system};

Much safer and more correct

IPC::System::Simple

Make using system() and output capturing (a-la backticks) safer

use IPC::System::Simpleqw(system systemx);

# overrides system() built-in...

# Succeed or die, avoid shell if @argssystem("some_command",@args);

# NEVER invokes the shellsystemx("some_command",@args);

use IPC::System::Simpleqw(capture capturex);

# Use capture() instead of backticks: (also works in list context)

# Succeeds or dies,# avoids shell if @argsmy $output =

capture("some_command",@args);

# NEVER invokes the shellMy $output =

capturex("some_command",@args);

Introduction The Little Things Slightly Bigger Things Handling Files and Filesystems Wrangling Data Structures and Types Dealing with Dates and Times Reading and Writing File Formats Potpourri Conclusion

Don't take my word for it

Explore the CPAN for yourself

Create your own toolkit

Be consistent

be thoughtful

Share what you learn

Contribute and help this language grow!

Thanks again.I'm hercynium on IRC and

twitter.