07.04.2008 @ vienna.pm aldo “dada” calpini 1/22dada.perl.it/talks/tyfs.pdf · 07.04.2008 @...

07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 11/22/22


Thank you for smokingThank you for smoking

……wtf?wtf?



no, actually not.no, actually not.



but…but…

SMOKE TESTINGSMOKE TESTING

can be healthy!can be healthy!



what is…what is…


““...a collection of tests [..] sometimes performed by the ...a collection of tests [..] sometimes performed by the automated system that builds the final software”automated system that builds the final software”

““...a collection of written tests that are performed on a ...a collection of written tests that are performed on a system prior to being accepted for further testing. This system prior to being accepted for further testing. This is also known as a build verification test”is also known as a build verification test”

(Wikipedia)(Wikipedia)



...in Perl words:...in Perl words:

make testmake test



...in Perl words:...in Perl words:

make testmake test

(automated)(automated)



SMOKE TESTINGSMOKE TESTINGin software development cycle:

REPOSITORY

COMMIT

HACK HACK HACK

DEVELOPER




DEVELOPER

REPOSITORY

UPDATE

DEVELOPER

DEVELOPER

DEVELOPER

UPDATE

UPD

ATE UPD

ATE


SMOKE SERVERTEST R

EPOR

T

TEST REPORT

TEST REPO

RTDEVELOPER

DEVELOPER

DEVELOPER

DEVELOPER

TEST REPORT





who's using…who's using…

SMOKE TESTINGSMOKE TESTING PerlPerl

• 187.867 tests187.867 tests

• 17 architectures, 15 major platforms17 architectures, 15 major platforms

ParrotParrot• ~11.000 tests~11.000 tests

• 7 architectures, 8 major platforms7 architectures, 8 major platforms

Pugs Pugs (currently down)(currently down)

• ~16.000 tests~16.000 tests

• 6 architectures, 5 major platforms (eg. where GHC runs)6 architectures, 5 major platforms (eg. where GHC runs)

and then there's CPAN Testersand then there's CPAN Testers



the benefits ofthe benefits of


catches hard to spot errorscatches hard to spot errors• eg. feature X breaks Y on platform Zeg. feature X breaks Y on platform Z

• or simply, breaks something with a different setup than the developer's oneor simply, breaks something with a different setup than the developer's one

......before before going to releasegoing to release



the benefits ofthe benefits of


catches hard to spot errorscatches hard to spot errors• eg. feature X breaks Y on platform Zeg. feature X breaks Y on platform Z

• or simply, breaks something with a different setup than the developer's oneor simply, breaks something with a different setup than the developer's one

......before before going to releasegoing to release

and test reports are cool :-)and test reports are cool :-)



do do youyou need need


??



do do youyou need need


?? probably notprobably not ...but if you really do, try Smolder:...but if you really do, try Smolder:

• http://www.sourceforge.net/projects/smolderhttp://www.sourceforge.net/projects/smolder



but still...but still...

test reports are cooltest reports are cool



enterenter

TAPTAPTest Anything ProtocolTest Anything Protocol

example:example:1..61..6

ok 1ok 1

ok 2ok 2

ok 3 – this is a commentok 3 – this is a comment

ok 4ok 4

ok 5ok 5

not ok 6 damn!not ok 6 damn!

# Failed test 'damn!'# Failed test 'damn!'

# at t/foobar.t line 10.# at t/foobar.t line 10.

# got: 'foo'# got: 'foo'

# expected: 'bar'# expected: 'bar'



enterenter

Test::TAP::ModelTest::TAP::Model

based on Test::Harness::Strapsbased on Test::Harness::Straps• (actually old, replaced by TAP::Parser, but we'll have to live with it)(actually old, replaced by TAP::Parser, but we'll have to live with it)

can run tests (as can run tests (as make testmake test does) and collect results does) and collect results in a Perl data structurein a Perl data structure



enterenter

Test::TAP::HTMLMatrixTest::TAP::HTMLMatrix

uses Test::TAP::Model to produce super cool HTML test uses Test::TAP::Model to produce super cool HTML test reportsreports



how to use it:how to use it:

~/bin/smoke~/bin/smoke#!/usr/bin/perl

use strict;

use warnings;

use Test::TAP::Model;

use Test::TAP::HTMLMatrix;

my $title = $ARGV[0] ? "$ARGV[0] test report" : undef;

eval "use blib;"; # nevermind if we're not testing a distribution

my $tap = Test::TAP::Model::Visual>new_with_tests(glob('t/*.t'));

$tap>run_tests();

my $v = Test::TAP::HTMLMatrix>new($tap, $title);

print $v>html;



how to use it:how to use it:

chdir to a folder with testschdir to a folder with tests• usually your module/distribution folderusually your module/distribution folder

run: run: smoke > report.htmlsmoke > report.html

now you're ready to upload report.html to your website, now you're ready to upload report.html to your website, email it to friends, print it and use it to impress girls.email it to friends, print it and use it to impress girls.

automationautomation is left as an exercise to the reader :-) is left as an exercise to the reader :-)



that's allthat's all some reference:some reference:

• Perl smoke tests: Perl smoke tests: http://www.nntp.perl.org/group/perl.daily-build.reports/http://www.nntp.perl.org/group/perl.daily-build.reports/

• Parrot smoke tests: Parrot smoke tests: http://smoke.parrotcode.org/smoke/http://smoke.parrotcode.org/smoke/

• Pugs: Pugs: http://www.pugscode.orghttp://www.pugscode.org

• in particular: in particular: http://svn.pugscode.org/pugs/util/testgraph.plhttp://svn.pugscode.org/pugs/util/testgraph.pl

• TAP: TAP: http://testanything.orghttp://testanything.org

• Smolder: Smolder: http://www.sourceforge.net/projects/smolderhttp://www.sourceforge.net/projects/smolder

thank you (for smoking)thank you (for smoking)

http://www.nntp.perl.org/group/perl.daily-build.reports/

http://smoke.parrotcode.org/smoke/

http://www.pugscode.org/

http://svn.pugscode.org/pugs/util/testgraph.pl

http://testanything.org/

http://www.sourceforge.net/projects/smolder

07.04.2008 @ vienna.pm aldo “dada” calpini 1/22dada.perl.it/talks/tyfs.pdf · 07.04.2008 @...

Documents