perl in testing

Upload: pbbhaskar507

Post on 10-Apr-2018

219 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/8/2019 Perl in Testing

    1/89

    Automatic Testing with Perl

    Gabor Szabo

  • 8/8/2019 Perl in Testing

    2/89

    Automatic Testing with Perl

    by Gabor Szabo

    1.05 Edition

    Published Mon May 28 07:11:15 2007

    Copyright 2004, 2005, 2006, 2007 PTI Ltd. Perl Training Israelhttp://www.pti.co.il/

  • 8/8/2019 Perl in Testing

    3/89

    Table of Contents1. About Perl Training Israel .................................................................................................................... 1

    2. Introduction............................................................................................................................................ 2

    2.1. Self Introduction - Who am I ? ................................................................................................... 2

    2.2. Self Introduction - Who are you ?............................................................................................... 2

    3. Preface .................................................................................................................................................... 3

    3.1. Objectives.................................................................................................................................... 3

    3.2. Plan of the seminar...................................................................................................................... 3

    4. Manual testing........................................................................................................................................ 4

    4.1. Manual testing............................................................................................................................. 4

    4.2. Web site testing ........................................................................................................................... 4

    4.3. CLI testing................................................................................................................................... 4

    4.4. Database testing .......................................................................................................................... 4

    4.5. GUI testing .................................................................................................................................. 5

    5. Basic Testing Framework in Perl ......................................................................................................... 6

    5.1. Testing a simple command line tool............................................................................................ 65.2. Calculator test ............................................................................................................................. 6

    5.3. Error ! .......................................................................................................................................... 6

    5.4. Calculator test with expected results........................................................................................... 7

    5.5. More difficult output ................................................................................................................... 7

    5.6. Print only ok/not ok..................................................................................................................... 7

    5.7. Write the ok function .................................................................................................................. 8

    5.8. Introducing Test::Simple............................................................................................................. 9

    5.9. Add names to the tests .............................................................................................................. 10

    5.10. Enlarge our test suit ................................................................................................................. 10

    5.11. Load Test::Simple at run time ................................................................................................. 11

    5.12. Forget about your "plan", use "no_plan"................................................................................. 12

    5.13. Put the test cases in an external file......................................................................................... 135.14. Harness.................................................................................................................................... 14

    5.15. Move external call into function ............................................................................................. 15

    5.16. Exercises: MyCalc .................................................................................................................. 15

    5.17. Solution: MyCalc .................................................................................................................... 15

    5.18. Test::Simple............................................................................................................................. 17

    6. Test::More ............................................................................................................................................ 18

    6.1. Moving over to Test::More ....................................................................................................... 18

    6.2. Test::More ok( trueness, name);................................................................................................ 18

    6.3. Test::More is( value, expected_value, name); ........................................................................... 19

    6.4. diag(just_a_message ); .............................................................................................................. 19

    6.5. like(value, qr/expected regex/, name); ...................................................................................... 20

    6.6. cmp_ok( this, op, that, name);................................................................................................... 20

    6.7. is_deeply( complex_structure, expected_complex structure, name); ....................................... 21

    6.8. TODO........................................................................................................................................ 22

    6.9. TODO with Harness.................................................................................................................. 23

    6.10. Platform dependent tests ......................................................................................................... 23

    6.11. SKIP some tests ...................................................................................................................... 24

    iii

  • 8/8/2019 Perl in Testing

    4/89

    6.12. My own test functions ............................................................................................................. 24

    6.13. My own test functions - improved .......................................................................................... 25

    6.14. Create a test module ................................................................................................................ 25

    6.15. Test::Builder............................................................................................................................ 26

    6.16. Number of tests ....................................................................................................................... 27

    6.17. Early Abnormal Exit ............................................................................................................... 27

    6.18. Less than planned tests - Early Normal Exit ........................................................................... 286.19. More tests than planned .......................................................................................................... 29

    6.20. Multiply................................................................................................................................... 29

    6.21. Error in the test........................................................................................................................ 30

    6.22. Multiply - fixed ....................................................................................................................... 30

    6.23. Multiple expected values......................................................................................................... 30

    6.24. Exercises ................................................................................................................................. 31

    7. Command line application .................................................................................................................. 33

    7.1. bc - An arbitrary precision calculator language ........................................................................ 33

    7.2. Normal operation ...................................................................................................................... 33

    7.3. Expect.pm ................................................................................................................................. 33

    7.4. Simple computation - adding two values .................................................................................. 337.5. Results ....................................................................................................................................... 34

    7.6. Simple computation - separate send commands ....................................................................... 34

    7.7. Simple computation - is it really working ? .............................................................................. 35

    7.8. Results ....................................................................................................................................... 35

    7.9. Reduce output ........................................................................................................................... 36

    7.10. Output...................................................................................................................................... 36

    7.11. More than one test ................................................................................................................... 36

    7.12. Output...................................................................................................................................... 37

    7.13. External test file ...................................................................................................................... 37

    7.14. Random regression tests.......................................................................................................... 38

    7.15. Random and regression testing ............................................................................................... 39

    7.16. Random and regression testing - slight improvement ............................................................. 407.17. Results ..................................................................................................................................... 41

    7.18. Recording session ................................................................................................................... 41

    7.19. Capturing both STDOUT and STDERR ................................................................................. 41

    7.20. Capturing both STDOUT and STDERR manually ................................................................. 42

    7.21. Capturing both STDOUT and STDERR using IPC::Run3 ..................................................... 42

    8. Networking devices .............................................................................................................................. 44

    8.1. Introduction - pick the right abstraction level ........................................................................... 44

    8.2. Socket level programming using Socket.pm ............................................................................. 44

    8.3. Socket level programming using IO::Socket............................................................................. 45

    8.4. Newline ..................................................................................................................................... 46

    8.5. Net::Telnet................................................................................................................................. 468.6. Net::Telnet for HTTP ................................................................................................................ 46

    8.7. Net::Telnet configure VLAN..................................................................................................... 47

    8.8. ftp using Net::FTP..................................................................................................................... 49

    8.9. ssh using Net::SSH.................................................................................................................... 49

    8.10. LWP::Simple........................................................................................................................... 50

    8.11. LWP......................................................................................................................................... 51

    Copyright 2007, Gabor Szabo @ Perl Training Israel iv

  • 8/8/2019 Perl in Testing

    5/89

    8.12. WWW::Mechanize.................................................................................................................. 51

    8.13. WWW::GMail......................................................................................................................... 52

    8.14. Exercise: CNN Developing story ............................................................................................ 52

    8.15. Exercise: Search on Financial Times ...................................................................................... 52

    8.16. Exercise: Compare exchange rates.......................................................................................... 53

    8.17. Telnet to Unix machines.......................................................................................................... 53

    9. Servers .................................................................................................................................................. 54

    9.1. Net::Server ................................................................................................................................ 54

    9.2. Skeleton Server ......................................................................................................................... 54

    9.3. Simple Echo Server ................................................................................................................... 54

    9.4. Echo Server ............................................................................................................................... 55

    9.5. Complex network servers .......................................................................................................... 56

    10. Command Line Interface .................................................................................................................. 58

    10.1. Introduction ............................................................................................................................. 58

    10.2. Connect to the device .............................................................................................................. 58

    10.3. Reduce timeout ....................................................................................................................... 59

    10.4. Exercise: Telnet....................................................................................................................... 60

    10.5. Our test script .......................................................................................................................... 60

    11. Testing networking devices ............................................................................................................... 64

    11.1. Elements.................................................................................................................................. 64

    11.2. Hardware setup........................................................................................................................ 64

    11.3. Access the administrative interface......................................................................................... 64

    11.4. Configure devices on all sides of our box ............................................................................... 64

    11.5. Run tests .................................................................................................................................. 64

    11.6. Check results ........................................................................................................................... 64

    11.7. Expect.pm ............................................................................................................................... 65

    11.8. Other modules ......................................................................................................................... 65

    11.9. Networking.............................................................................................................................. 65

    11.10. Network devices .................................................................................................................... 6511.11. Devices connected to Serial or Parallel port ......................................................................... 66

    11.12. X10 protocol ......................................................................................................................... 66

    12. Web Applications............................................................................................................................... 67

    12.1. What can be tested ?................................................................................................................ 67

    12.2. Tools ........................................................................................................................................ 67

    12.3. Small test HTTP server ........................................................................................................... 67

    12.4. Fetching a static page .............................................................................................................. 67

    12.5. Fetching a not-existing static page .......................................................................................... 68

    12.6. Checking good HTML ............................................................................................................ 68

    12.7. Checking bad HTML .............................................................................................................. 69

    12.8. What is this bad HTML ? ........................................................................................................ 69

    12.9. HTML::Tidy and Test::HTML::Tidy ...................................................................................... 70

    12.10. Test using W3C validator ...................................................................................................... 70

    12.11. LWP::Simple and LWP ......................................................................................................... 71

    12.12. WWW::Mechanize................................................................................................................ 71

    12.13. Web based Calculator............................................................................................................ 71

    12.14. More things to test................................................................................................................. 73

    Copyright 2007, Gabor Szabo @ Perl Training Israel v

  • 8/8/2019 Perl in Testing

    6/89

    12.15. Test page with JavaScript ...................................................................................................... 73

    13. Database access using Perl DBI ........................................................................................................ 74

    13.1. Architecture of a DBI Application.......................................................................................... 74

    13.2. Create Sample Database.......................................................................................................... 74

    13.3. Connect to database ................................................................................................................ 74

    13.4. SELECT with one result ......................................................................................................... 7513.5. SELECT with more results ..................................................................................................... 75

    13.6. SELECT, prepare with placeholders ....................................................................................... 76

    13.7. SELECT, using hashref........................................................................................................... 76

    13.8. INSERT ................................................................................................................................... 76

    13.9. Sample database ...................................................................................................................... 77

    13.10. Other methods ....................................................................................................................... 78

    13.11. Attributes............................................................................................................................... 78

    13.12. Error handling ....................................................................................................................... 78

    13.13. Debugging (Trace levels) ...................................................................................................... 79

    14. Database access using Class::DBI .................................................................................................... 80

    14.1. Class::DBI ............................................................................................................................... 80

    14.2. INSERT using Class::DBI ...................................................................................................... 8014.3. SELECT using Class::DBI...................................................................................................... 80

    14.4. MyDBI and MyUsers .............................................................................................................. 80

    Copyright 2007, Gabor Szabo @ Perl Training Israel vi

  • 8/8/2019 Perl in Testing

    7/89

    List of Examples5-1. mycalc on the command line................................................................................................................ 6

    5-2. examples/intro/t01_calc.t ..................................................................................................................... 6

    5-3. examples/intro/t01_calc.out ................................................................................................................. 6

    5-4. examples/intro/t02_calc.t ..................................................................................................................... 7

    5-5. examples/intro/t02_calc.out ................................................................................................................. 75-6. examples/intro/t03_calc.t ..................................................................................................................... 8

    5-7. examples/intro/t03_calc.out ................................................................................................................. 8

    5-8. examples/intro/t04_calc.t ..................................................................................................................... 9

    5-9. examples/intro/t04_calc.out ................................................................................................................. 9

    5-10. examples/intro/t05_calc.t ................................................................................................................... 9

    5-11. examples/intro/t05_calc.out ............................................................................................................... 9

    5-12. examples/intro/t09_calc.t ................................................................................................................. 10

    5-13. examples/intro/t09_calc.out ............................................................................................................. 10

    5-14. examples/intro/t10_calc.t ................................................................................................................. 11

    5-15. examples/intro/t10_calc.out ............................................................................................................. 11

    5-16. examples/intro/t11_calc.t ................................................................................................................. 11

    5-17. examples/intro/t12_calc.t ................................................................................................................. 125-18. examples/intro/t12_calc.out ............................................................................................................. 12

    5-19. examples/intro/t13_calc.t ................................................................................................................. 13

    5-20. examples/intro/calc.txt ..................................................................................................................... 13

    5-21. examples/intro/t13_calc.out ............................................................................................................. 13

    5-22. examples/harness.pl.......................................................................................................................... 14

    5-23. examples/intro/t13_calc.harness.out ................................................................................................ 14

    5-24. examples/intro/t14_calc.t ................................................................................................................. 15

    5-25. examples/intro/mycalc_test.t............................................................................................................ 16

    6-1. examples/intro/t21_calc.t ................................................................................................................... 18

    6-2. examples/intro/t22_calc.t ................................................................................................................... 19

    6-3. examples/intro/like.t ........................................................................................................................... 20

    6-4. examples/intro/cmp_ok.t .................................................................................................................... 20

    6-5. examples/intro/is_deeply.t .................................................................................................................. 21

    6-6. examples/intro/is_deeply.out .............................................................................................................. 22

    6-7. examples/intro/t23_calc.t ................................................................................................................... 23

    6-8. examples/intro/without_skip.t ............................................................................................................ 24

    6-9. examples/intro/skip.t .......................................................................................................................... 24

    6-10. examples/intro/Test/MyTest.pm ....................................................................................................... 26

    6-11. examples/intro/early-abnormal-exit.t ............................................................................................... 27

    6-12. examples/intro/early-normal-exit.t ................................................................................................... 28

    6-13. examples/intro/too-many-tests.t ....................................................................................................... 29

    6-14. examples/intro/t24_calc.t ................................................................................................................. 29

    6-15. examples/intro/t25_calc.t ................................................................................................................. 306-16. examples/intro/multiple_choice.t ..................................................................................................... 31

    7-1. examples/bc/bc1.pl ............................................................................................................................. 34

    7-2. examples/bc/bc2.pl ............................................................................................................................. 34

    7-3. examples/bc/bc3.pl ............................................................................................................................. 35

    7-4. examples/bc/bc4.pl ............................................................................................................................. 36

    7-5. examples/bc/bc5.pl ............................................................................................................................. 36

    vii

  • 8/8/2019 Perl in Testing

    8/89

  • 8/8/2019 Perl in Testing

    9/89

    Chapter 1. About Perl Training Israel

    PTI has been providing training and development services since 2000. Our courses include the following:

    Perl Courses

    Fundamentals of Perl

    References, Modules and Objects (Advanced)

    Web Application Development with Perl (Advanced)

    Debugging Perl Scripts and Applications (Advanced)

    Perl Quick Start (Beginner)

    QA Automation using Perl (Advanced)

    Perl for QA Professionals (Beginner)

    Non-Perl Courses

    Regular Expressions for Java programmers

    Regular Expressions for .NET programmers

    Version Control with Subversion

    Details can be found on our web site http://www.pti.co.il

    1

  • 8/8/2019 Perl in Testing

    10/89

    Chapter 2. Introduction

    2.1. Self Introduction - Who am I ? Gabor Szabo http://www.pti.co.il

    First program in 1983

    In the field since 1993

    Perl trainer since 2000

    Perl developer

    CPAN author

    System Administrator

    Web application developer

    Testing Automation

    http://www.szabgab.com

    2.2. Self Introduction - Who are you ?

    Name

    Workplace

    Background in computers (e.g. Operating Systems)

    Background in programming (which languages? How many years?)

    Background in Perl?

    Why do you take this course ?

    What do you expect to accomplish ?

    2

  • 8/8/2019 Perl in Testing

    11/89

    Chapter 3. Preface

    3.1. Objectives

    All tests successful.

    Files=10, Tests=2078, 50 wallclock secs

    Why and when to test ?

    Learn about the tools Perl provides for automated testing

    Use the test framework provided by Perl

    3.2. Plan of the seminar1. Introduction

    2. Manual testing, What, when how to test ?

    3. Introduce the Perl modules for basic testing framework

    4. Command line applications

    5. Devices with CLI access

    6. Web applications

    7. Testing Databases

    8. X Windows

    9. Microsoft Windows GUI Applications

    10. Internet Explorer based applications

    11. Appendix

    3

  • 8/8/2019 Perl in Testing

    12/89

    Chapter 4. Manual testing

    4.1. Manual testing

    Before we dig in into the world of automated testing we should first see that we are speaking about the

    same thing.

    4.2. Web site testing

    You type in a URL

    see if the site comes up correctly,

    click on the link to the registration form,

    fill in the fields (youll have to play with this and fill in the fields with good values, bad values, find theedge cases etc.)

    Check if you get back the correct response page.

    Maybe check if the database contains the correct information

    4.3. CLI testing

    Here you have a device like a router, or some other box connected to the network. Normally you would

    telnet to it and then interactively test the various commands to see if they work. In addition you might be

    able to fetch the raw configuration information where you can validate if the configuration values were

    written correctly.

    Going even further after you configured the device somehow you can test it if the new behaviour of the

    device really can be observed: You connect other devices and ping this box or try to send packets and see

    if they get to the correct location.

    Telnet to device

    Use SNMP to monitor/configure the device

    Prepare external entities on 2 or more sides of the device

    Send packets

    Check if the packets were received correctly

    4

  • 8/8/2019 Perl in Testing

    13/89

    Chapter 4. Manual testing

    4.4. Database testing

    This can be considered as part of any application as there is some kind of a database used by every

    application. In the simple case the database might be a flat file but it can also be some csv file or xml

    file or an RDBMS that you can access using SQL. In this case you would like to test what are the

    consequences on the database of certain operations of the application ?

    Prepare a database

    Execute some code

    Check if the database was updated correctly

    4.5. GUI testing

    Launch the application

    Find various controls Type in values/check if they show up correctly

    Push buttons

    Check if results are correct

    Copyright 2007, Gabor Szabo @ Perl Training Israel 5

  • 8/8/2019 Perl in Testing

    14/89

    Chapter 5. Basic Testing Framework in Perl

    5.1. Testing a simple command line tool

    First well talk about a simple application with simple tests and from there youll be able to move on to

    test more complex applications. I have a new calculator program called "mycalc". It expects a

    mathematical expression on the command line and prints the result on the screen. e.g.:

    Example 5-1. mycalc on the command line

    > mycalc 3 + 4

    7

    Lets look at it first an then write a couple of tests for it.

    5.2. Calculator test

    Example 5-2. examples/intro/t01_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    system "./mycalc 1 + 1";

    print "\n";

    system "./mycalc 2 + 2";

    print "\n";

    system "./mycalc 2 + 2 + 2";

    print "\n";

    Output:

    Example 5-3. examples/intro/t01_calc.out

    2

    4

    4

    5.3. Error !

    Did you notice the bad answer ?

    6

  • 8/8/2019 Perl in Testing

    15/89

    Chapter 5. Basic Testing Framework in Perl

    The third line should have been 6.

    5.4. Calculator test with expected results

    So we should write the expected value next to the correct one:

    Example 5-4. examples/intro/t02_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    system "./mycalc 1 + 1";

    print " 2\n";

    system "./mycalc 2 + 2";

    print " 4\n";

    system "./mycalc 2 + 2 + 2";

    print " 6\n";

    Output:

    Example 5-5. examples/intro/t02_calc.out

    2 2

    4 4

    4 6

    Now it is better.

    5.5. More difficult output

    But what if the expected output is:

    This is the expected output of my program that should be checked.

    This is the expected output of my program that should be checked.

    Yes, in this case they are actually the same. What if you had 300 such line pairs ? And what if 3000 ?

    It would be probably much better if our testing program already compared the expected value with the

    actual results and would only print OK or NOT OK depending on success or failures.

    OK

    OK

    NOT OK

    Copyright 2007, Gabor Szabo @ Perl Training Israel 7

  • 8/8/2019 Perl in Testing

    16/89

    Chapter 5. Basic Testing Framework in Perl

    5.6. Print only ok/not ok

    Example 5-6. examples/intro/t03_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    my $result;

    $result = ./mycalc 1 + 1;

    if ( $result == 2 ) {

    print "ok\n";

    }

    else {

    print "not ok\n";

    }

    $result = ./mycalc 2 + 2;

    if ( $result == 4 ) {print "ok\n";

    }

    else {

    print "not ok\n";

    }

    $result = ./mycalc 2 + 2 + 2;

    if ( $result == 6 ) {

    print "ok\n";

    }

    else {

    print "not ok\n";

    }

    # We replaced the "system" calls with backtick in order to catch the STDOUT

    # It is extreamly verbose and we are repeating the same code a lot of times

    Output:

    Example 5-7. examples/intro/t03_calc.out

    ok

    ok

    not ok

    Copyright 2007, Gabor Szabo @ Perl Training Israel 8

  • 8/8/2019 Perl in Testing

    17/89

    Chapter 5. Basic Testing Framework in Perl

    5.7. Write the ok function

    Example 5-8. examples/intro/t04_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    ok(./mycalc 1 + 1 == 2);

    ok(./mycalc 2 + 2 == 4);

    ok(./mycalc 2 + 2 + 2 == 6);

    sub ok {

    my ($ok) = @_;

    print $ok ? "ok\n" : "not ok\n";

    }

    Output:

    Example 5-9. examples/intro/t04_calc.out

    ok

    ok

    not ok

    But why reinvent the wheel ?

    Besides, if there are lots of tests, we would need some way to easily recognise which test(s) fail. So we

    should put a counter on our tests.

    5.8. Introducing Test::Simple

    Example 5-10. examples/intro/t05_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    # tell how many tests you are going to write. This is our "plan"

    use Test::Simple tests => 3;

    # the ok function of Test::Simple prints "ok" or "not ok"

    ok ./mycalc 1 + 1 == 2;

    ok ./mycalc 2 + 2 == 4;

    ok ./mycalc 2 + 2 + 2 == 6;

    Output:

    Copyright 2007, Gabor Szabo @ Perl Training Israel 9

  • 8/8/2019 Perl in Testing

    18/89

    Chapter 5. Basic Testing Framework in Perl

    Example 5-11. examples/intro/t05_calc.out

    1..3

    ok 1

    ok 2

    not ok 3

    # Failed test in t05_calc.t at line 11.

    # Looks like you failed 1 test of 3.

    It is more verbose, it has a couple of additional useful piece of information: 1..3 says how many tests we

    were planning then we get the tests numbered and we even get a small explanation when the test fails.

    5.9. Add names to the tests

    So Test::Simple makes our life a bit more simple in that we dont have to write our testing expression. In

    addition this new "ok" function can actually do some more favour. It can get two arguments. The first one

    indicates success or failure of the test and the second one is the name of the test. When running a test

    with these additional names they get printed on the same line where the "ok" or "not ok" is printed. In

    case of lots of tests it will make it easier to locate the test and if the names were written carefully they

    can provide an immediate hint what went wrong. Sometimes you wont even need to look at the test

    script itself, right from this comment youll know where to look for the bug.

    Example 5-12. examples/intro/t09_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::Simple tests => 3;

    ok ./mycalc 1 + 1 == 2, small sum: 1+1;

    ok ./mycalc 2 + 2 == 4, small sum: 2+2;

    ok ./mycalc 2 + 2 + 2 == 6, two operators: 2+2+2;

    Output:

    Example 5-13. examples/intro/t09_calc.out

    1..3

    ok 1 - small sum: 1+1

    ok 2 - small sum: 2+2

    not ok 3 - two operators: 2+2+2

    # Failed test two operators: 2+2+2

    # in t09_calc.t at line 9.

    # Looks like you failed 1 test of 3.

    Copyright 2007, Gabor Szabo @ Perl Training Israel 10

  • 8/8/2019 Perl in Testing

    19/89

    Chapter 5. Basic Testing Framework in Perl

    5.10. Enlarge our test suit

    Example 5-14. examples/intro/t10_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    my %tests = (

    1 + 1 => 2,

    2 + 2 => 4,

    2 + 2 + 2 = > 6 ,

    1+1 => 2,

    0+ -1 => -1,

    0-1 => -1,

    -1+1 => 0,

    );

    use Test::Simple tests => 7;

    foreach my $t ( keys %tests ) {

    ok( ./mycalc $t == $tests{$t}, $t );

    }

    Example 5-15. examples/intro/t10_calc.out

    1..7

    o k 1 - 0 - 1

    ok 2 - 1 + 1

    ok 3 - -1+1

    o k 4 - 0 + - 1

    o k 5 - 1 + 1

    n o t o k 6 - 2 + 2 + 2

    # Failed test 2 + 2 + 2

    # in t10_calc.t at line 18.

    ok 7 - 2 + 2

    # Looks like you failed 1 test of 7.

    There is a small problem though. When you add a new test to the hash, you also have to remember to

    update the tests => 7 line.

    There are a number of solution to this problem

    5.11. Load Test::Simple at run time

    Example 5-16. examples/intro/t11_calc.t

    #!/usr/bin/perl

    Copyright 2007, Gabor Szabo @ Perl Training Israel 11

  • 8/8/2019 Perl in Testing

    20/89

    Chapter 5. Basic Testing Framework in Perl

    use strict;

    use warnings;

    my %tests = (

    1 + 1 => 2,

    2 + 2 => 4,

    2 + 2 + 2 = > 6 ,

    1+1 => 2,

    0+ -1 => -1,

    0-1 => -1,

    -1+1 => 0,

    );

    require Test::Simple;

    import Test::Simple tests => scalar keys %tests;

    foreach my $t ( keys %tests ) {

    ok( ./mycalc $t == $tests{$t}, $t );

    }

    5.12. Forget about your "plan", use "no_plan"

    Example 5-17. examples/intro/t12_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    my %tests = (

    1 + 1 => 2,

    2 + 2 => 4,

    2 + 2 + 2 = > 6 ,

    1+1 => 2,

    0+ -1 => -1,

    0-1 => -1,

    -1+1 => 0,

    );

    use Test::Simple "no_plan";

    foreach my $t ( keys %tests ) {

    ok( ./mycalc $t == $tests{$t}, $t );

    }

    Example 5-18. examples/intro/t12_calc.out

    o k 1 - 0 - 1

    ok 2 - 1 + 1

    ok 3 - -1+1

    Copyright 2007, Gabor Szabo @ Perl Training Israel 12

  • 8/8/2019 Perl in Testing

    21/89

    Chapter 5. Basic Testing Framework in Perl

    o k 4 - 0 + - 1

    o k 5 - 1 + 1

    n o t o k 6 - 2 + 2 + 2

    # Failed test 2 + 2 + 2

    # in t12_calc.t at line 18.

    ok 7 - 2 + 2

    1..7

    # Looks like you failed 1 test of 7.

    The 1..7 is now at the end.

    5.13. Put the test cases in an external file

    Example 5-19. examples/intro/t13_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::Simple "no_plan";

    open my $fh, "

  • 8/8/2019 Perl in Testing

    22/89

    Chapter 5. Basic Testing Framework in Perl

    Example 5-21. examples/intro/t13_calc.out

    ok 1 - 1 + 1

    ok 2 - 2 + 2

    n o t o k 3 - 2 + 2 + 2

    # Failed test 2 + 2 + 2

    # in t13_calc.t at line 16.

    o k 4 - 1 + 1

    o k 5 - 0 + - 1

    o k 6 - 0 - 1

    ok 7 - -1+1

    1..7

    # Looks like you failed 1 test of 7.

    5.14. Harness

    This is a module that can analyse the ok / not ok printouts with the numbers. In particular it can analysethe output of Test::Simple, Test::More and all the Test::Builder based modules.

    Example 5-22. examples/harness.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::Harness qw(runtests);

    runtests @ARGV;

    Run the previous test file using Test::Harness

    $ perl ../harness.pl t13_calc.t

    Example 5-23. examples/intro/t13_calc.harness.out

    t13_calc....

    # Failed test 2 + 2 + 2

    # in t13_calc.t at line 16.

    # Looks like you failed 1 test of 7.

    dubious

    Test returned status 1 (wstat 256, 0x100)

    DIED. FAILED test 3

    Failed 1/7 tests, 85.71% okay

    Failed Test Stat Wstat Total Fail List of Failed

    -------------------------------------------------------------------------------

    t13_calc.t 1 256 7 1 3

    Failed 1/1 test scripts. 1/7 subtests failed.

    Files=1, Tests=7, 1 wallclock secs ( 0.12 cusr + 0.05 csys = 0.17 CPU)

    Copyright 2007, Gabor Szabo @ Perl Training Israel 14

  • 8/8/2019 Perl in Testing

    23/89

    Chapter 5. Basic Testing Framework in Perl

    Failed 1/1 test programs. 1/7 subtests failed.

    5.15. Move external call into function

    Example 5-24. examples/intro/t14_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::Simple "no_plan";

    open my $fh, "

  • 8/8/2019 Perl in Testing

    24/89

    Chapter 5. Basic Testing Framework in Perl

    5.17. Solution: MyCalc

    Example 5-25. examples/intro/mycalc_test.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::Simple tests => 5+2*3;

    use MyCalc;

    # tests, one by one

    ok(add(1, 1) == 2);

    ok(add(1, -1) == 0);

    ok(sum(1) == 1);

    ok(sum() == 0);

    ok(sum(1, 1, 1, 1) == 4);

    # tests listed in an array

    my @tests = (

    {

    func => add,

    in => [2, 3],

    out => 5,

    },

    {

    func => sum,

    in => [1, 2, 3],

    out => 6,

    },

    );

    foreach my $t (@tests) {

    if ($t->{func} eq add) {

    ok(add( @{ $t->{in} } ) == $t->{out}, "add @{ $t->{in} }");

    }

    if ($t->{func} eq sum) {

    ok(sum( @{ $t->{in} } ) == $t->{out}, "sum @{ $t->{in} }");

    }

    }

    # The same but Danger! Danger! Danger!

    # Using symbolic references here!foreach my $t (@tests) {

    no strict refs;

    ok(&{ $t->{func} }( @{ $t->{in} } ) == $t->{out}, "$t->{func} @{ $t->{in} }");

    # the same with helper variables:

    my $func = $t->{func};

    Copyright 2007, Gabor Szabo @ Perl Training Israel 16

  • 8/8/2019 Perl in Testing

    25/89

    Chapter 5. Basic Testing Framework in Perl

    my @in = @{ $t->{in} };

    ok(&$func(@in) == $t->{out}, "$func @in");

    }

    5.18. Test::Simple

    This is all very nice and Simple.

    What if you want More ?

    Copyright 2007, Gabor Szabo @ Perl Training Israel 17

  • 8/8/2019 Perl in Testing

    26/89

    Chapter 6. Test::More

    6.1. Moving over to Test::More

    Test::Simple is really a very simple module. Its sole exported function is the "ok" function.

    Test::More has the same "ok" function - so it is a drop-in replacement - but it also has lots of

    other functions and tools:

    ok

    is

    isnt

    diag

    like

    cmp_ok

    is_deeply

    SKIP

    TODO

    6.2. Test::More ok( trueness, name);

    A drop-in replacement of Test::Simple.

    Example 6-1. examples/intro/t21_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::More tests => 3;

    ok ./mycalc 1 + 1 == 2, 1+1;

    ok ./mycalc 2 + 2 == 4, 2+2;

    ok ./mycalc 2 + 2 + 2 == 6, 2+2+2;

    Result

    $ perl t6_calc.t

    1..3

    o k 1 - 1 + 1

    o k 2 - 2 + 2

    not ok 3 - 2+2+2

    18

  • 8/8/2019 Perl in Testing

    27/89

    Chapter 6. Test::More

    # Failed test (t6_calc.t at line 7)

    # Looks like you failed 1 tests of 3.

    6.3. Test::More is( value, expected_value, name);

    It would be much better to see the expected value and the actually received value. This usually helps in

    locating the problem.

    Example 6-2. examples/intro/t22_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::More tests => 3;

    is ./mycalc 1 + 1, 2, 1+1;

    is ./mycalc 2 + 2, 4, 2+2;

    is ./mycalc 2 + 2 + 2, 6, 2+2+2;

    Result

    $ perl t7_calc.t

    1..3

    o k 1 - 1 + 1

    o k 2 - 2 + 2

    not ok 3 - 2+2+2

    # Failed test (t7_calc.t at line 7)

    # got: 4

    # expected: 6# Looks like you failed 1 tests of 3.

    See, in this case we can already guess that it cannot add 3 values.

    compares using eq

    6.4. diag(just_a_message );

    diag prints out a message along with the rest of the output.

    Use it for whatever extra output in order to ensure that your printouts will not interfere with future

    changes in the test environment modules (such as Test::Harness).

    diag "We are going to test the Foo-Bar device now";

    Copyright 2007, Gabor Szabo @ Perl Training Israel 19

  • 8/8/2019 Perl in Testing

    28/89

    Chapter 6. Test::More

    # We are going to test the Foo-Bar device now

    6.5. like(value, qr/expected regex/, name);

    It is especially important when you dont want or cant realisticly expect an exact match with the result.

    compares with =~

    Example 6-3. examples/intro/like.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::More tests => 2;

    like( foo(), qr/\d+/, "there are some digits in the result" );

    like( bar(), qr/\d+/, "there are some digits in the result" );

    sub foo {

    return "This is a long text with a number 42 in it";

    }

    sub bar {

    return "This is another string with no number in it";

    }

    $ perl like.t

    1..2

    ok 1 - there are some digits in the resultnot ok 2 - there are some digits in the result

    # Failed test there are some digits in the result

    # in like.t at line 7.

    # This is another string with no number in it

    # doesnt match (?-xism:\d+)

    # Looks like you failed 1 test of 2.

    6.6. cmp_ok( this, op, that, name);

    compares with anything

    Copyright 2007, Gabor Szabo @ Perl Training Israel 20

  • 8/8/2019 Perl in Testing

    29/89

    Chapter 6. Test::More

    Example 6-4. examples/intro/cmp_ok.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::More tests => 2;

    my $start = time;

    wait_for_input_with_timeout(3);

    my $end = time;

    cmp_ok $end - $start, ">=", 2, "process was waiting at least 2 secs";

    cmp_ok $end - $start, "

  • 8/8/2019 Perl in Testing

    30/89

    Chapter 6. Test::More

    my %a = fetch_data_from_bug_tracking_system(0);

    is_deeply( \%a, \%expected, "Query 0" );

    my %b = fetch_data_from_bug_tracking_system(1);

    is_deeply( \%b, \%expected, "Query 1" );

    my %c = fetch_data_from_bug_tracking_system(2);

    is_deeply( \%c, \%expected, "Query 2" );

    sub fetch_data_from_bug_tracking_system {

    my @sets = (

    { bugs => 3,

    errors => 6,

    failures => 8,

    warnings => 1,

    },

    { bugs => 3,

    errors => 9,

    failures => 8,

    warnings => 1,

    },

    { bogs => 3,

    erors => 9,

    failures => 8,

    warnings => 1,

    },

    );

    my $h = $sets[shift];

    return %$h;

    }

    Example 6-6. examples/intro/is_deeply.out

    1..3

    ok 1 - Query 0

    not ok 2 - Query 1

    # Failed test Query 1

    # in is_deeply.t at line 19.

    # Structures begin differing at:

    # $got->{errors} = 9

    # $expected->{errors} = 6

    not ok 3 - Query 2

    # Failed test Query 2

    # in is_deeply.t at line 22.

    # Structures begin differing at:

    # $got->{errors} = Does not exist

    # $expected->{errors} = 6

    # Looks like you failed 2 tests of 3.

    Copyright 2007, Gabor Szabo @ Perl Training Israel 22

  • 8/8/2019 Perl in Testing

    31/89

  • 8/8/2019 Perl in Testing

    32/89

    Chapter 6. Test::More

    6.10. Platform dependent tests

    Example 6-8. examples/intro/without_skip.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::More "no_plan";

    like( /sbin/ifconfig, qr/eth0/ );

    like( ipconfig, qr/Windows IP Configuration/ );

    $ perl without_skip.t

    ok 1

    not ok 2

    # Failed test (without_skip.t at line 5)

    # undef

    # doesnt match (?-xism:Windows IP Configuration)

    1..2# Looks like you failed 1 tests of 2.

    6.11. SKIP some tests

    Example 6-9. examples/intro/skip.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::More tests => 2;

    like( /sbin/ifconfig, qr/eth0/ );

    SKIP: {

    skip "Windows related tests", 1 if $^O !~ /Win/i;

    like( ipconfig, qr/Windows IP Configuration/ );

    }

    $ perl skip.t

    ok 1

    ok 2 # skip Windows related tests

    1..2

    Copyright 2007, Gabor Szabo @ Perl Training Israel 24

  • 8/8/2019 Perl in Testing

    33/89

    Chapter 6. Test::More

    6.12. My own test functions

    After writing lots of tests, youll start to re-factor your code and create functions for specific parts of the

    test:

    You will have some code like this:

    use Test::More tests => 2;

    is(my_test(2, +, 3), 5);

    is(my_test(4, +, 4), 10);

    sub my_test {

    my ($x, $op, $z) = @_;

    # do stuff

    return $result;

    }

    6.13. My own test functions - improved

    It would be better if we could eliminate the multiple occurance of is

    use Test::More tests => 2;

    my_test(2, +, 3, 5);

    my_test(4, +, 4, 10);

    sub my_test {

    my ($x, $op, $z, $expected) = @_;

    # do stuff

    is($result, $expected);

    }

    6.14. Create a test module

    You need the above my_test subroutine in several of your test files. So you create a module for this.

    use Test::More tests => 2;

    use Test::MyTest "my_test";

    my_test(2, +, 3, 5);

    Copyright 2007, Gabor Szabo @ Perl Training Israel 25

  • 8/8/2019 Perl in Testing

    34/89

    Chapter 6. Test::More

    my_test(4, +, 4, 10);

    package Test::MyTest;

    require Exporter;

    our @ISA = qw(Exporter);

    our @EXPORT_OK = qw(my_test);

    sub my_test {

    my ($x, $op, $z, $expected) = @_;

    # do stuff

    is($result, $expected);

    }

    Not good !

    We need the is function from the Test::More package. so well have to use it in our module as well but

    Test::More needs a "plan".

    6.15. Test::Builder

    Example 6-10. examples/intro/Test/MyTest.pm

    package Test::MyTest;

    use strict;

    use warnings;

    use base Exporter;

    our @EXPORT_OK = qw(my_test);

    use Test::Builder;

    my $Test = Test::Builder->new;

    sub my_test {

    my ($x, $op, $y, $expected) = @_;

    my $result;

    if ($op eq +) {

    $result = $x + $y;

    } else {

    die "Not yet implemented";

    }

    $Test->is_num($result, $expected);

    Copyright 2007, Gabor Szabo @ Perl Training Israel 26

  • 8/8/2019 Perl in Testing

    35/89

    Chapter 6. Test::More

    }

    1;

    Test modules created using Test::Builder all work nicely together. Among other things, they dont get

    confused with the counting of the tests.

    Test::Simple, Test::More, Test::Exception, Test::Differences all use Test::Builder as a back-end.

    6.16. Number of tests

    It is always good to have a plan when you start working even if you are going to change your plans

    within a short time. If for nothing else if you use some version control youll be able to see your

    progress. (What do you mean you dont have version control ?) Checking if the right number of tests

    were successfully executed is a test in itself. Some things that can go wrong if you dont check it:

    Test script dies before running all the tests.

    Test script dies after running all the tests.

    You run too many tests - in itself it is not a problem but later when for some reason your test script

    stops after the correct number of test you wont notice the missing tests.

    It is always good to have some plan.

    6.17. Early Abnormal Exit

    Example 6-11. examples/intro/early-abnormal-exit.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    # example of tests where the script dies before running all the tests

    use Test::More tests => 4;

    is ./mycalc 1 + 1, 2, 1+1;

    is ./mycalc 2 + 2, 4, 2+2;

    exit;

    is ./mycalc 3 + 3, 6, 3+3;

    is ./mycalc 4 + 4, 8, 4+4;

    Directly:

    Copyright 2007, Gabor Szabo @ Perl Training Israel 27

  • 8/8/2019 Perl in Testing

    36/89

    Chapter 6. Test::More

    1..4

    o k 1 - 1 + 1

    o k 2 - 2 + 2

    # Looks like you planned 4 tests but only ran 2.

    With Harness:

    early-abnormal-exit....ok 2/4# Looks like you planned 4 tests but only ran 2.

    early-abnormal-exit....dubious

    Test returned status 2 (wstat 512, 0x200)

    DIED. FAILED tests 3-4

    Failed 2/4 tests, 50.00% okay

    Failed Test Stat Wstat Total Fail Failed List of Failed

    ----------------------------

    early-abnormal-exit.t 2 512 4 4 100.00% 3-4

    Failed 1/1 test scripts, 0.00% okay. 2/4 subtests failed, 50.00% okay.

    6.18. Less than planned tests - Early Normal Exit

    Example 6-12. examples/intro/early-normal-exit.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    # example of too few tests (less than planned)

    use Test::More tests => 4;

    is ./mycalc 1 + 1, 2, 1+1;is ./mycalc 2 + 2, 4, 2+2;

    is ./mycalc 3 + 3, 6, 3+3;

    Directly:

    1..4

    o k 1 - 1 + 1

    o k 2 - 2 + 2

    o k 3 - 3 + 3

    # Looks like you planned 4 tests but only ran 3.

    With Harness:

    early-normal-exit....ok 2/4# Looks like you planned 4 tests but only ran 3.

    early-normal-exit....dubious

    Test returned status 1 (wstat 256, 0x100)

    DIED. FAILED test 4

    Failed 1/4 tests, 75.00% okay

    Copyright 2007, Gabor Szabo @ Perl Training Israel 28

  • 8/8/2019 Perl in Testing

    37/89

    Chapter 6. Test::More

    Failed Test Stat Wstat Total Fail Failed List of Failed

    -----------------------------

    early-normal-exit.t 1 256 4 2 50.00% 4

    Failed 1/1 test scripts, 0.00% okay. 1/4 subtests failed, 75.00% okay.

    6.19. More tests than planned

    Example 6-13. examples/intro/too-many-tests.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    # example of too many tests (more than planned)

    use Test::More tests => 2;

    is( ./mycalc 1 + 1, 2, 1+1 );

    is( ./mycalc 2 + 2, 4, 2+2 );

    is( ./mycalc 3 + 3, 6, 3+3 );

    Directly:

    1..2

    o k 1 - 1 + 1

    o k 2 - 2 + 2

    o k 3 - 3 + 3

    # Looks like you planned 2 tests but ran 1 extra.

    With Harness:

    too-many-tests....ok 3/2# Looks like you planned 2 tests but ran 1 extra.

    too-many-tests....dubious

    Test returned status 1 (wstat 256, 0x100)

    DIED. FAILED test 3

    Failed 1/2 tests, 50.00% okay

    Failed Test Stat Wstat Total Fail Failed List of Failed

    -------------------------------

    too-many-tests.t 1 256 2 1 50.00% 3

    Failed 1/1 test scripts, 0.00% okay. -1/2 subtests failed, 150.00% okay.

    6.20. Multiply

    Example 6-14. examples/intro/t24_calc.t

    #!/usr/bin/perl

    Copyright 2007, Gabor Szabo @ Perl Training Israel 29

  • 8/8/2019 Perl in Testing

    38/89

    Chapter 6. Test::More

    use strict;

    use warnings;

    use Test::More tests => 1;

    is ./mycalc 2 * 2, 4, 2*2;

    Results

    $ perl t10_calc.t

    1..1

    Bareword found where operator expected at (eval 1) line 1, near "2 IE"

    (Missing operator before IE?)

    n o t o k 1 - 2*2

    # Failed test (t10_calc.t at line 5)

    # got:

    # expected: 4

    # Looks like you failed 1 tests of 1.

    6.21. Error in the test

    User error

    When you find a problem it is not always the problem in the application you are testing.

    If it was a human who did the work we would call it a "user error" but in our case the

    user is a program itself. As the developer of the application can make errors the

    person who writes the tests can also make errors.

    6.22. Multiply - fixed

    Example 6-15. examples/intro/t25_calc.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Test::More tests => 1;

    is ./mycalc "2 * 2", 4, 2*2;

    Results

    $ perl t25_calc.t

    1..1

    ok 1 - 2*2

    Copyright 2007, Gabor Szabo @ Perl Training Israel 30

  • 8/8/2019 Perl in Testing

    39/89

    Chapter 6. Test::More

    6.23. Multiple expected values

    Someone asked me this question:

    Q: Is it possible instead ofis(foo(), 42) to have a fixed set of expected values ?

    A: With is there is no such option but you have a number of solutions anyway.

    Example 6-16. examples/intro/multiple_choice.t

    #!/usr/bin/perl

    use strict;

    use warnings;

    # run this script several times to see the (random) error message

    use Test::More tests => 2;

    like( foo(), qr/^(23|42|68)$/ );

    ok( grep { foo() eq $_ } ( 23, 42, 68 ), "name" );

    sub foo {

    return ( ( 23, 42, 68, 100, 200 )[ rand(5) ] );

    }

    $ perl multiple_choice.t

    not ok 1

    # Failed test (multiple_choice.t at line 4)

    # 100

    # doesnt match (?-xism:^(23|42|68)$)

    not ok 2# Failed test (multiple_choice.t at line 5)

    1..2

    # Looks like you failed 2 tests of 2.

    $ perl multiple_choice.t

    ok 1

    ok 2

    1..2

    6.24. Exercises* take the ifconfig/ipconfig test script and fix it so

    there will be a skip block on the ifconfig part as well.

    * create a test function is_any that will get

    a scalar and and a reference to an array and

    Copyright 2007, Gabor Szabo @ Perl Training Israel 31

  • 8/8/2019 Perl in Testing

    40/89

    Chapter 6. Test::More

    check if the scalar is one of the values in

    the array.

    write test script that uses this function.

    * create a test module that exports the is_any function.

    Write test script that uses this module.

    * Change the test script you wrote for MyCalc.pm

    to use the is function of Test::More.

    Copyright 2007, Gabor Szabo @ Perl Training Israel 32

  • 8/8/2019 Perl in Testing

    41/89

    Chapter 7. Command line application

    7.1. bc - An arbitrary precision calculator language

    It is much more than a calculator, it is a language. Luckily we dont need to learn the whole language in

    order to to do simple calculations. Normally you execute bc from the command line and then you type

    in your calculations. Pressing ENTER will do the calculation.

    7.2. Normal operation

    > bc

    bc 1.06

    Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.

    This is free software with ABSOLUTELY NO WARRANTY.

    For details type warranty.

    23+7

    30

    quit

    >

    Try it ....

    7.3. Expect.pm

    Provides a way to describe user behaviour in a command line environment

    Can send information as if it was typed in the keyboard

    Can wait for some Expect-ed value and based on this value do something

    Originally an extension of Tcl

    Ported to Perl

    Can be used in environments such as:

    Command line application like bc

    Telnet to another box and type in things

    Anything usually a person would do on the command line.

    33

  • 8/8/2019 Perl in Testing

    42/89

    Chapter 7. Command line application

    7.4. Simple computation - adding two values

    Example 7-1. examples/bc/bc1.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Expect;

    my $e = Expect->new;

    $e->raw_pty(1);

    $e->spawn("bc") or die "Cannot run bc\n";

    $e->expect(1, "warranty") or die "no warranty\n";

    $e->send("23+7\n");

    $e->expect(1, 30) or die "no sum\n";

    print "Success\n";

    raw_pty turns off echo

    spawn starts the external program

    expect(timeout, regex) return undef if failed

    timeout is in seconds, 0 means check once, undef means wait forever

    send - as if the user was typing at the keyboard

    7.5. Results

    >perl examples/bc/bc1.pl

    bc 1.06

    Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.

    This is free software with ABSOLUTELY NO WARRANTY.

    For details type warranty.

    30

    Success

    7.6. Simple computation - separate send commands

    Instead of calling send once, we will call it now 4 times

    Copyright 2007, Gabor Szabo @ Perl Training Israel 34

  • 8/8/2019 Perl in Testing

    43/89

    Chapter 7. Command line application

    Example 7-2. examples/bc/bc2.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Expect;

    my $e = Expect->new;

    $e->raw_pty(1);

    $e->spawn("bc") or die "Cannot run bc\n";

    $e->expect(1, "warranty") or die "no warranty\n";

    $e->send("23");

    $e->send("+");

    $e->send("7");

    $e->send("\n");

    $e->expect(1, 30) or die "no sum\n";

    print "Success\n";

    7.7. Simple computation - is it really working ?

    The same computation but we expect the wrong value ...

    This is not that clever but I could not find a real bug in bc to show here as an example and Id like to

    show what happens when the computation fails.

    Example 7-3. examples/bc/bc3.pl

    #!/usr/bin/perluse strict;

    use warnings;

    use Expect;

    my $e = Expect->new;

    $e->raw_pty(1);

    $e->spawn("bc") or die "Cannot run bc\n";

    $e->expect(1, "warranty") or die "no warranty\n";

    $e->send("23+7\n");

    $e->expect(1, 29) or die "no sum\n";

    print "Success\n";

    7.8. Results

    The same computation but we expect the wrong value ... and fail

    Copyright 2007, Gabor Szabo @ Perl Training Israel 35

  • 8/8/2019 Perl in Testing

    44/89

    Chapter 7. Command line application

    >perl examples/bc/bc3.pl

    bc 1.06

    Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.

    This is free software with ABSOLUTELY NO WARRANTY.

    For details type warranty.

    30

    no sum

    7.9. Reduce output

    We dont want to see all the output bc generates and then try to look for the correct responses or the error

    messages. Wed prefer just see ok or not ok

    Example 7-4. examples/bc/bc4.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Expect;

    use Test::More qw(no_plan);

    $Expect::Log_Stdout = 0;

    my $e = Expect->new;

    $e->raw_pty(1);

    $e->spawn("bc") or die "Cannot run bc\n";

    $e->expect(1, "warranty") or die "no warranty\n";

    $e->send("23+7\n");

    ok($e->expect(1, 30));

    $Expect::Log_Stdout = 0; - turn off the printing to the screen

    7.10. Output

    >perl examples/bc/bc4.pl

    ok 1

    1..1

    7.11. More than one test

    We can then setup lots of tests and run them through one invocation of bc.

    Copyright 2007, Gabor Szabo @ Perl Training Israel 36

  • 8/8/2019 Perl in Testing

    45/89

    Chapter 7. Command line application

    Example 7-5. examples/bc/bc5.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Expect;

    use Test::More qw(no_plan);

    $Expect::Log_Stdout = 0;

    my @sets = (

    [23+7, 30],

    [23+7, 30],

    [11+1, 10],

    [2*21, 42],

    );

    my $e = Expect->new;

    $e->spawn("bc") or die "Could not start bc\n";

    $e->expect(undef, "warranty") or die "no warranty\n";

    foreach my $set (@sets) {

    $e->send("$$set[0]\n");

    ok($e->expect(1, $$set[1]), $$set[0]);

    }

    $e->send("quit\n");

    7.12. Output

    >perl examples/bc/bc5.pl

    # You named your test 30. You shouldnt use numbers for your test names.# Very confusing.

    o k 1 - 3 0

    ok 2 - 23+7

    not ok 3 - 11+1

    # Failed test (examples/bc/bc5.pl at line 20)

    ok 4 - 2*21

    1..4

    # Looks like you failed 1 tests of 4.

    The first entry in the @sets array is not good as the operation will be executed by Perl and not by bc.

    7.13. External test file

    Separating the test cases from the code.

    Copyright 2007, Gabor Szabo @ Perl Training Israel 37

  • 8/8/2019 Perl in Testing

    46/89

    Chapter 7. Command line application

    Example 7-6. examples/bc/bc5a.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Expect;

    use Test::More qw(no_plan);

    $Expect::Log_Stdout = 0;

    open my $fh, "

  • 8/8/2019 Perl in Testing

    47/89

    Chapter 7. Command line application

    Either save the new results as the new expectation or discard it and discard the current version of the

    application

    7.15. Random and regression testing

    Example 7-7. examples/bc/bc6.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use File::Compare;

    use Expect;

    $Expect::Log_Stdout = 0;

    if (not @ARGV or $ARGV[0] ne "random" and $ARGV[0] ne "regress") {

    die "Usage: $0 [random|regress]\n";

    }

    if ($ARGV[0] eq "regress" and not -e "tests.txt") {

    die "Cannot run regression before running random tests!\n";

    }

    if ($ARGV[0] eq random) {

    my $e = Expect->new;

    $e->raw_pty(1);

    $e->log_file("random.log", "w");

    $e->spawn("bc") or die "Could not start bc\n";

    $e->expect(1, "warranty") or die "no warranty\n";

    open my ($test_file), ">tests.txt" or die "Cannot open tests file for writing\n"

    foreach (1..3) {

    my ($a, $b) = (rand, rand);

    my $op = qw(+ * - /)[int rand 4];

    my $line = "$a $op $b\n";

    print $test_file $line;

    $e->send($line);

    $e->expect(1,[qr/\d+/]);

    }

    $e->send("quit\n");

    }

    if ($ARGV[0] eq regress) {

    my $e = Expect->new;

    Copyright 2007, Gabor Szabo @ Perl Training Israel 39

  • 8/8/2019 Perl in Testing

    48/89

    Chapter 7. Command line application

    $e->raw_pty(1);

    $e->log_file("regress.log", "w");

    $e->spawn("bc") or die "Could not start bc\n";

    $e->expect(1, "warranty") or die "no warranty\n";

    open my ($test_file), "tests.txt" or die "Cannot open tests file for reading\n";

    while (my $line = ) {

    $e->send($line);

    $e->expect(1, [qr/\d+/]);

    }

    $e->send("quit\n");

    $e->log_file(undef); # close the log file

    if (compare("random.log", "regress.log")) {

    print "Regression failed\n";

    } else {

    print "Regression successful\n";

    }

    }

    # Two parts

    # - random tests

    # - regression tests

    #

    #

    # run random tests

    # save test cases in a test cases file

    # save all the results in a log file

    #

    #

    # run all the tests from the test cases file and log results to a new log file

    # compare original log with new log and complain if they are not the same.

    #

    7.16. Random and regression testing - slight

    improvement

    Example 7-8. examples/bc/bc7_diff.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Text::Diff; # instead of File::Compare

    # and diff instead of compare

    Copyright 2007, Gabor Szabo @ Perl Training Israel 40

  • 8/8/2019 Perl in Testing

    49/89

    Chapter 7. Command line application

    if (my $diff = diff ("random.log", "regress.log")) {

    print "Regression failed\n\n";

    print $diff;

    } else {

    print "Regression successful\n";

    }

    7.17. Results

    ~/work/training/testing/examples/bc>perl bc7.pl regress

    Regression failed

    --- random.log Mon Apr 19 23:09:51 2004

    +++ regress.log Mon Apr 19 23:38:54 2004

    @@ -3,5 +3,5 @@

    This is free software with ABSOLUTELY NO WARRANTY.

    For details type warranty.

    1

    -.9114942335378392

    +.8114942335378392

    .8404963860314223

    7.18. Recording session

    A simple example how to "record" a session for later analysing and changing into a real script.

    Example 7-9. examples/expect/record.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Expect;

    my $exp = Expect->spawn("bash") or die "Cannot spawn bash child\n";

    $exp->log_file($ARGV[0] || "record.log");

    #$exp->raw_pty(1);

    $exp->interact();

    Copyright 2007, Gabor Szabo @ Perl Training Israel 41

  • 8/8/2019 Perl in Testing

    50/89

    Chapter 7. Command line application

    7.19. Capturing both STDOUT and STDERR

    Write out the expected STDIN to a file called "in"

    Run the app system "$app < in >out 2>err";

    read in the out and err files an examine them

    7.20. Capturing both STDOUT and STDERR manually

    Example 7-10. examples/io/capture.pl

    #!/usr/bin/perl -w

    use strict;

    use Test::More tests => 2;

    my $app = "./examples/io/application.pl";

    my @in = (10, 21, hello, 3x);

    my $in = join "\n", @in;

    my @expected_out = (20, 42);

    my @expected_err = (

    "The input hello contains no numeric values",

    "The input 3x contains no numeric values",

    );

    {

    open my $fh, ">", "/tmp/in" or die $!;

    print $fh $in;

    }

    system "$app < /tmp/in > /tmp/out 2> /tmp/err";

    {

    open my $fh, "

  • 8/8/2019 Perl in Testing

    51/89

    Chapter 7. Command line application

    7.21. Capturing both STDOUT and STDERR using

    IPC::Run3

    Example 7-11. examples/io/capture_ipc.pl

    #!/usr/bin/perl -wuse strict;

    use Test::More tests => 2;

    use IPC::Run3;

    my $app = "./examples/io/application.pl";

    my @in = (10, 21, hello, 3x);

    my $in = join "\n", @in;

    my @expected_out = (20, 42);

    my @expected_err = (

    "The input hello contains no numeric values","The input 3x contains no numeric values",

    );

    {

    my $out;

    my $err;

    run3 [$app], \$in, \$out, \$err;

    my $expected_out = join("\n", @expected_out) . "\n";

    is($out, $expected_out, "IPC Output");

    my $expected_err = join("\n", @expected_err) . "\n";

    is($err, $expected_err, "IPC Error");

    }

    Copyright 2007, Gabor Szabo @ Perl Training Israel 43

  • 8/8/2019 Perl in Testing

    52/89

    Chapter 8. Networking devices

    8.1. Introduction - pick the right abstraction level

    When trying to connect some network device using Perl you have a number of choices.

    See the full stack of HTTP connections:

    built in socket function and the Socket module

    IO::Socket::INET using IO::Socket

    Net::Telnet, Net::FTP, Net::SSH (wrapping ssh), Net::SSH::Perl, Net::*

    LWP::Simple, LWP

    WWW::Mechanize

    WWW::GMail

    At the lowest level you can use the built in socket function.

    Using the Socket library provides several extra functions

    and constants that will make your code cleaner and more portable.

    See also perlipc

    8.2. Socket level programming using Socket.pmExample 8-1. examples/network/socket.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Socket qw(:DEFAULT :crlf);

    # Using the built in "socket" function with various helper variables

    # and functions from the standard Socket.pm module

    # get the protocol id (on Linux from /etc/protocols)my $protocol_id = getprotobyname(tcp);

    # build C structure in_addr from hostip

    # if hostname is given it tries to resolve hostname to ip first (and returns

    # undef if not successful)

    #my $host = 127.0.0.1;

    my $host = localhost;

    44

  • 8/8/2019 Perl in Testing

    53/89

    Chapter 8. Networking devices

    #my $host = 66.249.85.99;

    #my $host = www.google.com;

    my $host_struct = inet_aton($host);

    # inet_ntoa($host_struct) returns the resolved ip address

    my $port = 80;

    socket(my $socket, PF_INET, SOCK_STREAM, $protocol_id) or die $!;

    my $sockaddr_in = pack_sockaddr_in($port, $host_struct);

    connect($socket, $sockaddr_in) or die $!;

    # turn off buffering on the socket

    {

    my $old = select($socket);

    $ | = 1 ;

    select($old);

    }

    print $socket "GET /$CRLF";

    while (my $line = ) {

    print $line;

    }

    8.3. Socket level programming using IO::Socket

    Example 8-2. examples/network/io_socket.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use IO::Socket;

    # IO::Socket is a higher level abstraction

    # Hides many of the ugly part we had to know in case of the socket() function.

    # Provides an OOP interface.

    #my $host = 127.0.0.1;

    my $host = localhost;

    #my $host = www.google.com;

    #my $host = 209.85.135.103;

    #my $host = www.perl.org.il;

    my $port = 80;

    my $CRLF = "\015\012";

    my $socket = IO::Socket::INET->new(

    PeerAddr => $host,

    PeerPort => $port,

    Copyright 2007, Gabor Szabo @ Perl Training Israel 45

  • 8/8/2019 Perl in Testing

    54/89

    Chapter 8. Networking devices

    Proto => tcp,

    ) or die $!;

    $socket->send("GET /$CRLF") or die $!;

    my $SIZE = 100;

    my $data = ;

    while ($socket->read($data, $SIZE, length $data) == $SIZE) {};

    print $data;

    8.4. Newline

    \n is a newline on our current system (is NOT always ASCII LF)

    \r is (is NOT always ASCII CR)

    use \015\012 to say CR+LF on networking applications

    8.5. Net::Telnet

    Example 8-3. examples/telnet/telnet.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Net::Telnet ();

    my $t = Net::Telnet->new();

    $t->open(localhost);

    $t->login(smoke, 123456);

    my @lines = $t->cmd("who");

    print @lines;

    print "\n";

    print "Who am i: ", $t->cmd("whoami"), "\n\n";

    8.6. Net::Telnet for HTTP

    Example 8-4. examples/telnet/telnet_http.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Net::Telnet ();

    Copyright 2007, Gabor Szabo @ Perl Training Israel 46

  • 8/8/2019 Perl in Testing

    55/89

    Chapter 8. Networking devices

    my $t = Net::Telnet->new(

    Timeout => 10,

    Host => localhost,

    Port => 80,

    );

    $t->print("GET /\n") or die $!;

    while (my $line = $t->getline) {

    print $line;

    }

    8.7. Net::Telnet configure VLAN

    Example 8-5. examples/telnet/configure_vlan.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Net::Telnet;

    open my $out, ">>", "out.log" or die $!;

    my $t = Net::Telnet->new(

    Timeout => 2,

    #Prompt => />/,

    input_log => "input.log",

    );

    $t->open("172.30.40.146");

    $t->waitfor(/User:.*$/);

    $t->print("admin");

    $t->waitfor(/Password:/);

    $t->print("");

    $t->waitfor(/>/);

    $t->prompt(/\(Switching\) >/);

    my @lines = $t->cmd("show vlan 5");

    if (grep /VLAN ID: 5/, @lines) {

    print "VLAN is already configured\n";

    print "Please remove it manually and rerun the program\n";

    exit;

    #$t->cmd("logout");

    }

    $t->print("enable");

    $t->waitfor(/Password:/);

    Copyright 2007, Gabor Szabo @ Perl Training Israel 47

  • 8/8/2019 Perl in Testing

    56/89

    Chapter 8. Networking devices

    $t->prompt(/\(Switching\) #/);

    $t->print("");

    $t->prompt(/\(Switching\) \(Vlan\) #/);

    @lines = $t->cmd("vlan database");

    @lines = $t->cmd("vlan 5");

    #print @lines;

    #if (grep /VLAN already/, @lines) {

    # print "QQ 1\n";

    #}

    @lines = $t->cmd("vlan 5000");

    #print @lines;

    @lines = $t->cmd("vlan 5");

    #print @lines;

    #if (grep /VLAN already/, @lines) {

    # print "QQ 2\n";

    #}

    #@lines = $t->cmd("no vlan 5");

    $t->prompt(/\(Switching\) #/);

    $t->cmd("exit");

    $t->prompt(/--More-- or \(q\)uit/);

    @lines = $t->cmd("show ?");

    $t->output_record_separator("");

    push @lines, $t->cmd(" ");

    $t->prompt(/\(Switching\) #show/);

    push @lines, $t->cmd(" ");

    #print @lines;

    $t->output_record_separator("\n");

    $t->prompt(/\(Switching\) #/);

    @lines = $t->cmd(" vlan 5"); # show was left on the promt line !

    #print @lines;

    @lines = $t->cmd("show vlan 7");

    #print @lines;

    @lines = $t->cmd("show slot");

    #print @lines;

    print $out @lines;

    $t->prompt(/\(Switching\) \(Vlan\) #/);

    @lines = $t->cmd("vlan database");

    @lines = $t->cmd("no vlan 5");

    Copyright 2007, Gabor Szabo @ Perl Training Israel 48

  • 8/8/2019 Perl in Testing

    57/89

    Chapter 8. Networking devices

    $t->prompt(/\(Switching\) #/);

    $t->cmd("exit");

    print "done: $_\n";

    print $out "done: $_\n";

    8.8. ftp using Net::FTP

    Example 8-6. examples/network/upload.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Net::FTP;

    use File::Basename qw(dirname);

    use File::Spec;

    my $DEBUG = 1;

    if (not @ARGV) {

    print "Usage:\n";

    print " $0 FILE [FILES]\n";

    exit;

    }

    my $ftp = Net::FTP->new(192.168.1.100) or die $!;

    $ftp->login(gabor, the password of gabor) or die $!;

    my $pwd = $ftp->pwd;

    foreach my $file (@ARGV) {

    my $dir = dirname $file;

    $ftp->cwd($pwd);

    $ftp->cwd($dir);

    $ftp->put($file);

    }

    8.9. ssh using Net::SSH

    Wrapping the external ssh command. Therefore working only in UNIX/Linux.See also Net::SSH::Perl.

    Copyright 2007, Gabor Szabo @ Perl Training Israel 49

  • 8/8/2019 Perl in Testing

    58/89

    Chapter 8. Networking devices

    Example 8-7. examples/network/ssh.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use Net::SSH qw(sshopen2);

    use IO::File;

    my $output = IO::File->new;

    my $input = IO::File->new;

    sshopen2("localhost", $output, $input) or die $!;

    print $input "set\n";

    print $input "echo DONE\n";

    print $input "who\n";

    print $input "echo DONE\n";

    print $input "date\n";

    print $input "echo DONE\n";

    print $input "cat README\n";

    print $input "exit\n";

    my @out = ;

    my $c=0;

    my @section;

    while (my $line = shift @out) {

    if ($line =~ /^DONE$/) {

    $c++;

    next;

    }

    push @{$section[$c]}, $line;

    }

    foreach my $sect (@section) {

    print @$sect;

    print "--------------------\n";

    }

    8.10. LWP::Simple

    Example 8-8. examples/network/lwp_simple.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    my $url = http://localhost/;

    if (defined $ARGV[0]) {

    $url = $ARGV[0];

    }

    Copyright 2007, Gabor Szabo @ Perl Training Israel 50

  • 8/8/2019 Perl in Testing

    59/89

    Chapter 8. Networking devices

    use LWP::Simple qw(get);

    my $page = get($url);

    if (defined $page) {

    print $page;

    } else {

    print "Could not fetch $url\n";

    }

    8.11. LWP

    Example 8-9. examples/network/lwp.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    my $url = http://localhost/;

    if (defined $ARGV[0]) {

    $url = $ARGV[0];

    }

    use LWP::UserAgent;

    my $ua = LWP::UserAgent->new;

    $ua->agent("Internet Explorer/17.1");

    my $req = HTTP::Request->new(GET => $url);

    my $res = $ua->request($req);

    if ($res->is_success) {

    print $res->content;

    } else {

    print $res->status_line, "\n";

    }

    8.12. WWW::Mechanize

    Example 8-10. examples/network/mechanize.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use WWW::Mechanize;

    my $w = WWW::Mechanize->new();

    $w->get(http://www.google.com/);

    $w->submit_form(

    Copyright 2007, Gabor Szabo @ Perl Training Israel 51

  • 8/8/2019 Perl in Testing

    60/89

    Chapter 8. Networking devices

    fields => {

    q => perl israel

    },

    );

    $w->follow_link( n => 5 );

    print $w->title;

    8.13. WWW::GMail

    Example 8-11. examples/network/gmail.pl

    #!/usr/bin/perl

    use strict;

    use warnings;

    use WWW::GMail;

    my $w = WWW::GMail->new(

    username => "USERNAME",

    password => "PASSWORD",

    );

    my $ret = $w->login();

    if ($ret == -1) {

    die "password incorrect\n";

    } elsif ($ret == 0) {

    die "unable to login $w->{error}\n";

    }

    my @messages = $w->get_message_list(inbox);

    foreach my $msg (@messages) {

    print "Subject: $msg->[6]\n";

    }

    8.14. Exercise: CNN Developing story

    Write a script that will print the title of the current "DEVELOPING STORY" on

    CNN (if there is one). That is click on the link titled "DEVELOPING STORY"

    and then fetch the TITLE tag from the new page.

    Copyright 2007, Gabor Szabo @ Perl Training Israel 52

  • 8/8/2019 Perl in Testing

    61/89

    Chapter 8. Networking devices

    8.15. Exercise: Search on Financial Times

    Go to the Financial Times (www.ft.com) and search for the workd "perl".

    Print out how many items match.

    8.16. Exercise: Compare exchange rates

    Go to various sites, check the AAA/BBB exchnage rate on each one of them.

    Print out the current exchange rate and also send an e-mail to yourself

    with the information.

    Also put a ARBITRAGE warning if there is a difference between the rates.

    AAA and BBB can be any two currencies you are interested in.

    Try the following sites:

    http://www.xe.com/

    http://www.oanda.com/

    http://www.bankofcanada.ca/en/rates/exchform.html

    8.17. Telnet to Unix machines

    Pick a Unix/Linux machine your are using and write a script that will telnet

    to it, execute report what network cards does it have (ifconfig on Linux)

    and what is its routing table (route -n on Linux).

    Copyright 2007, Gabor Szabo @ Perl Training Israel 53

  • 8/8/2019 Perl in Testing

    62/89

    Chapter 9. Servers

    9.1. Net::Server

    We are going to use the Net::Server module to create various server processes.

    9.2. Skeleton Server

    First we create