the perl api for the mortally terrified (beta)

The Perl APIfor the

Mortally Terrified

Mike Friedman(friedo)

London Perl WorkshopNovember 30, 2013

MeMike Friedman

Mike Friedman(friedo)

This talk is the product of a lengthy journey.

This talk is the product of a lengthy journey.

It's also a work in progress.

I have some limited experience in integrating Perl and C with XS, and would enjoy the opportunity to expand my skills in this area.

i R dum.

- Learned from this book.

- Learned from this book.- It’s very good

- Learned from this book.- It’s very good- The printing I got was messed up.

- Learned from this book.- It’s very good- The printing I got was messed up.- Like every " was replaced by ©

The Perl APIWhat’s it good for?

Make things fast:


Make things fast: - Write your slow Perl code in C


Make things fast: - Write your slow Perl code in C - Keep your fast Perl code in Perl


Make Perl more useful:


Make Perl more useful: - Interface with libraries written in C


Make Perl more useful: - Interface with libraries written in C - Create modules exposing external APIs to Perl


Make Perl do...stuff


Make Perl do...stuff - Change opcodes on the fly?


Make Perl do...stuff - Change opcodes on the fly? - Extend the parser?


Make Perl do...stuff - Change opcodes on the fly? - Extend the parser? - Make exceptions work?


Make Perl do...stuff - Change opcodes on the fly? - Extend the parser? - Make exceptions work? - Why not?

A Crash Course in

C

C Crash Course

Basic Types (there are more)

C Crash Course


char foo = ‘a’;

C Crash Course


char foo = ‘a’;int bar = 42;

C Crash Course


char foo = ‘a’;int bar = 42;long baz = 198547;

C Crash Course


char foo = ‘a’;int bar = 42;long baz = 198547;float quux = 3.14159;

C Crash Course


char foo = ‘a’;int bar = 42;long baz = 198547;float quux = 3.14159;double splat = 3.14159265358979;

C Crash Course

Pointers

C Crash Course

Pointers

char *foo; /* ptr to char */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */long *baz; /* ptr to long */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */long *baz; /* ptr to long */float *quux; /* ptr to float */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */long *baz; /* ptr to long */float *quux; /* ptr to float */double *splat; /* ptr to double */

C Crash Course

Pointers

char *foo; /* ptr to char */int *bar; /* ptr to int */long *baz; /* ptr to long */float *quux; /* ptr to float */double *splat; /* ptr to double */void *fnarf; /* ptr to something */

C Crash Course

Pointers are integer variables which hold a memory address.

C Crash Course

C Perl

C Crash Course

int i = 42; my $i = 42;

C Perl

C Crash Course

int i = 42; my $i = 42;int *i_ptr = &i; my $i_ref = \$i;

C Perl

C Crash Course


Read unary & as “address-of.”

C Perl

C Crash Course



int j; my $j;

C Perl

C Crash Course



int j; my $j;j = *i_ptr; $j = $$i_ref;

C Perl

C Crash Course



int j; my $j;j = *i_ptr; $j = $$i_ref;

Read unary * as “dereference.”

C Perl

C Crash Course

Aggregates

C Crash Course

Aggregates

int foo[10]; /* array of ints */

C Crash Course

Aggregates

int foo[10]; /* array of ints */float bar[12]; /* array of floats */

C Crash Course

Aggregates

int foo[10]; /* array of ints */float bar[12]; /* array of floats */char baz[42]; /* a string! */

C Crash Course

Aggregates

C Crash Course

Aggregatesstruct person { int age; char *name; double acct_balance;};

C Crash Course

Aggregatesstruct person { int age; char *name; double acct_balance;};

age *name acct_balance

...

C Crash Course

Aggregates

C Crash Course

Aggregatesunion thingy { long num; char chars[4]; foo_t *myfoo;};

C Crash Course

Aggregatesunion thingy { long num; char chars[4]; foo_t *myfoo;};

f3 de 42 9a

thingy.num

thingy.chars

thingy.myfoo

C Crash Course

Using structs

C Crash Course

struct person the_dude;the_dude.age = 42;the_dude.name = “jeffrey lebowski”;the_dude.acct_balance = 1.79;

Using structs

C Crash Course

struct person the_dude;the_dude.age = 42;the_dude.name = “jeffrey lebowski”;the_dude.acct_balance = 1.79;

Using structs

00 2a 3f 39 d2 90 a4 70 3d 0a d7 a3 fc 3f

age *name acct_balance

j e f f r e y ...

C Crash Course

Typedefstypedef struct { int age; char *name; double acct_balance;} person;

person the_dude;

What is XS?

perldoc perlxs:

XS is an interface description file format used to create an extension interface between Perl and C code (or a C library) which one wishes to use with Perl. The XS interface is combined with the library to create a new library which can then be either dynamically loaded or statically linked into perl. The XS interface description is written in the XS language and is the core component of the Perl extension interface.

XS is a specialized templating language for C.

Some XS Code

Some XS Code

This stuff is XS.

Some C Code made by xsubpp

XS != Perl API

For most things

For most things

You don't actuallyhave to use XS.

(OK. You don’t have to know that you’re using it.)

Inline::C

use Inline C;hello('Mike');hello('Cookie Monster');

__END__

__C__void hello(char* name) { printf("Hello %s!\n", name);}

https://metacpan.org/module/Inline


use Inline C;hello('Mike');hello('Grover');

__END__

__C__void hello(char* name) { printf("Hello %s!\n", name);}



Inline::C Code in Perl


Digest::MD5


Digest::MD5

C Code Parse::RecDescent


Digest::MD5

Create build in a temp dir



Digest::MD5



Write XS


Digest::MD5



Write XS

Compile and install


Digest::MD5



Write XS

Compile and install

�

How do we make Perl do stuff in C?

How do we make Perl do stuff in C?

We need to understand Perl’s data structures.

Handwaving Ahead

A $scalar in Perl is represented by a struct called SV.

SvNULL

SvRV SvNVSvPV SvIV

(These are not all of them.)

What does SV look like?

sv_any sv_refcnt sv_flags

{

flags type

What does SV look like?


{

flags type

If this scalar is undef, then sv_any is NULL

What does SvPV (a string) look like?


SV

xpvpvx current len alloc len

a s t r i n gchar* \0

What do SvIV and SvNV look like?


SV

xpviv

pvx current len alloc len

3 1 3 3 7 \0char*

69 7a 00 00

ivx

What do SvIV and SvNV look like?


SV

xpvnv

pvx current len alloc len

3 . 1 4 1 5char*

6e 86 1b f0

ivxf9 21 09 40

nvx

9 \0

WRITE CODE NOW?!??!!11!?

Let's make something fast.

#!/usr/bin/env perluse strict;use warnings;use Inline 'C';

print 'Give me a number: ';my $num_a = <STDIN>;

print 'Give me another number: ';my $num_b = <STDIN>;

printf "The sum is: %s\n", add( $num_a, $num_b );__END____C__

int add( int a, int b ) { return a + b;}

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 11The sum is: 53

Run it!

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 31.337The sum is: 73

Run it again!

bash-3.2$ perl cexp1.plGive me a number: 42Give me another number: 31.337The sum is: 73

Run it again!

What happened?

Inline::C is

CRAZY SMART

intintintint

Inline::C is

CRAZY SMART

use Inline 'C';

print 'Give me a number: ';my $num_a = <STDIN>;print 'Give me another number: ';my $num_b = <STDIN>;$num_a += 0; $num_b += 0;printf "The sum is: %s\n", add( $num_a, $num_b );__END____C__SV *add( SV *a, SV *b ) {

if ( SvIOK( a ) && SvIOK( b ) ) { return newSViv( SvIV( a ) + SvIV( b ) ); } else if ( SvNOK( a ) && SvNOK( b ) ) { return newSVnv( SvNV( a ) + SvNV( b ) ); } else { croak( "I don't know what to do!" ); }}

bash-3.2$ perl cexp2.pl Give me a number: 42Give me another number: 11The sum is: 53bash-3.2$ perl cexp2.pl

Run it!


Run it!

bash-3.2$ perl cexp2.pl Give me a number: 53.4Give me another number: 6.54The sum is: 59.94


Run it!

bash-3.2$ perl cexp2.pl Give me a number: 53.4Give me another number: 6.54The sum is: 59.94

Give me a number: 42Give me another number: 6.54I don't know what to do! at cexp2.pl line 16, <STDIN> line 2.

Once you get this far, everything you need

is in perlapi.

Let's do something with arrays.

What does an AV look like?


SV

xpvav

array fill max

SV*[]

alloc array len*some fields omitted

SV ...

SV ...


print "Give me some numbers: ";my @numbers = map { $_ += 0 } split /,/, <STDIN>;

my $result = squares( \@numbers );printf "The squares are: %s\n", join ", ", @$result;

__END__

__C__

SV *squares( SV *numbers ) { AV *array = (AV *)SvRV( numbers ); AV *return_array = newAV(); SV **tmp;

int len = AvFILL( array ); int i, val;

for( i = 0; i <= len; i++ ) { tmp = av_fetch( array, i, 1 ); if( !SvIOK( *tmp ) ) { croak( "Can't handle this value!" ); } val = SvIV( *tmp ); val = val * val;

av_push( return_array, newSViv( val ) ); }

return newRV_inc( (SV *)return_array );}

__C__

SV *squares( SV *numbers ) { AV *array = (AV *)SvRV( numbers ); AV *return_array = newAV(); SV **tmp;

int len = AvFILL( array ); int i, val;

for( i = 0; i <= len; i++ ) { tmp = av_fetch( array, i, 1 ); if( !SvIOK( *tmp ) ) { croak( "Can't handle this value!" ); } val = SvIV( *tmp ); val = val * val;

av_push( return_array, newSViv( val ) ); }

return newRV_noinc( (SV *)return_array );}

bash-3.2$ perl cexp3.pl Give me some numbers: 4,5,6,23The squares are: 16, 25, 36, 529

Run it!

Let's do something with hashes.

What does an HV look like?


(Diagram missing


(Diagram missing

(So I stole this one))


print "Give me some words: ";my @words = split /,/, <STDIN>;chomp @words;my $result = uniques( \@words );printf "The uniques are: %s\n", join ", ", %$result;

__END__

__C__

SV *uniques( SV *words ) { AV *array = (AV *)SvRV( words ); HV *result = newHV(); SV **tmp;

int len = AvFILL( array ); int i; char *val;

for( i = 0; i <= len; i++ ) { tmp = av_fetch( array, i, 1 ); if( !SvPOK( *tmp ) ) { croak( "Can't handle this value!" ); }

val = SvPV_nolen( *tmp );

hv_store( result, val, strlen( val ), newSV(0), 0 ); }

return newRV_noinc( (SV *)result );}

bash-3.2$ perl cexp4.pl Give me some words: foo,bar,baz,baz,foo,quux,narf,poitThe uniques are: bar, narf, baz, poit, quux, foo

Run it!

What isMagic?

Not that terrifying.

Not that terrifying.(code sample missing)

Magic is a linked list of function pointers attached to a *SV.


The functions implement custom read/write behavior.


The functions implement custom read/write behavior.

Special vars like %ENV are implemented via Magic.

$ENV{foo} = "blah";

$ENV{foo} = "blah";

assignment op

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

Look up write function

$ENV{foo} = "blah";

Magic on %ENV ?

assignment op

Look up write function

call libc setenv()

tie() is also implemented via Magic.

Terror defeated!

Where do we go from here?

Resources:

perlgutsFor details on how to manipulate Perl's

data structures.

Resources:

perlapiFor a comprehensive reference to (almost)

everything you need to make perl do stuff.

Resources:

Perlguts Illustrated

For pretty diagrams and explication.

Resources:

Inline::C Cookbook

For common solutions to problemsinterfacing between Perl and C

Resources:

Resources:

Code samples

http://github.com/friedo/perl-api-terror

London Perl Workshop

Thank you.

Mike [email protected]

http://friedo.com/http://github.com/friedo/perl-api-terror

the perl api for the mortally terrified (beta)

Technology

c crash coursecperlint

c crash courseusing

c crash coursebasic

c crash coursec int

ptr ptr ptr ptr ptr

c crash coursetypedefs

perl api whats

int age char