copyright 2005 eric niebler xpressive: library design on the edge or, how i learned to stop worrying...

Copyright 2005 Eric Niebler

xpressive:Library Design on the Edge

or, How I Learned to Stop Worrying and Love Template Meta-Programming


“Why program by hand in five days what you can spend five years of your life automating?”

-- Terrence Parr, author ANTLR/PCCTS


Overview

Domain-Specific (Embedded) Languages xpressive and Dual-Mode DSEL Design Programming at the Compile-time / Runtime

boundary


Domain-Specific Languages

Mini-languages, everywhere!– GNU Make– Backus Naur Form– Regular expressions

Syntax, constructs and abstractions for working efficiently in some narrow domain

Not general purpose!


Why DSLs?

General purpose languages are low-level, procedural

DSLs are high-level and declarative Solution space shares concepts with problem

space DSL code is easier to write, read, reason

about and maintain.


Embedded DSLs

A DSL in a library All the wholesome goodness of a DSL in the

host language of your choice! C++ is a good host language

– operator overloading– low abstraction penalty


Expression Templates

(Ab)uses C++ operator overloading to approximate the syntax of the embedded language

Must map embedded language syntax and constructs into C++

Used by:– Blitz++ : high-performance scientific computing– Spirit : parser generator (EBNF)


DSEL example: EBNF Calculator

group ::= '(' expr ')'fact ::= integer | groupterm ::= fact (('*' fact) | ('/' fact))*expr ::= term (('+' term) | ('-' term))*

rule<> group, fact, term, expr;

group = '(' >> expr >> ')';fact = integer | group;term = fact >> *(('*' >> fact) | ('/' >>

fact));expr = term >> *(('+' >> term) | ('-' >>

term));


Types of DSELs: Dynamic

Example:SQLCommand c = "SELECT * from Employees";

Advantages:– Unconstrained syntax– Statements can be specified at runtime

Disadvantages:– Syntax errors discovered at runtime– Performance costs of interpretation


Types of DSELs: Static

Example:double d = (matrix * vector)(3, 4);

Advantages:– Syntax errors checked at compile-time– Aggressive inlining, domain-specific codegen

Disadvantages:– Constrained by rules for legal C++ expressions– Cannot accept new statements at runtime


Static DSELs...

Dynamic DSELs...

Can’t I have it both ways?

Hrrmmm ...


Dual-Mode DSEL Interface

Provide both a static and dynamic interface! Sounds good but ...

– Can we really get the advantages of both?– Can we share the implementation to avoid code

duplication?


"\\w""\\w+""a\\w""a|b""(\\w)\\1""[^a-z]""(?=foo)"

_w+_w'a' >> _was_xpr('a') | 'b'(s1= _w) >> s1~range('a', 'z')before("foo")

Expression TemplateRegular Expression


Get a Date

// Match a date of the form 10-03-2005

"\\d\\d?-\\d\\d?-\\d\\d(?:\\d\\d)?";

regex date = _d >> !_d >> '-' // match month >> _d >> !_d >> '-' // match day >> _d >> _d >> !(_d >> _d); // match year

# Match a date of the form 10-03-2005

/\d\d?-\d\d?-\d\d(?:\d\d)?/

// Match a date of the form 10-03-2005regex date = regex::compile( "\\d\\d?-\\d\\d?-\\d\\d(?:\\d\\d)?");


Regex aliases, anyone?

regex date = /* ... */;

// A line in a log file is a date followed by a// space, and everything up to the newline.regex log = date >> ' ' >> +~set['\n'];

regex date = /* ... */;

// A line in a log file is a date followed by a// space, and everything up to the newline.regex log = date >> ' ' >> +~set['\n'];


Semantic Constraints

// Only match valid datesregex date = (_d >> !_d)[if_is_month()] >> '-' >> (_d >> !_d)[if_is_day()] >> '-' >> (_d >> _d >> !(_d >> _d))[if_is_year()];

// Only match valid datesregex date = (_d >> !_d)[if_is_month()] >> '-' >> (_d >> !_d)[if_is_day()] >> '-' >> (_d >> _d >> !(_d >> _d))[if_is_year()];


Two great tastes ...... that taste great together

// A line in a log file is a date followed by a// space, and everything up to the newline.regex date = regex::compile(get_date_pattern());

regex log = date >> ' ' >> +~set['\n'];


“Some people, when confronted with a problem, think, ‘I know, I’ll use

regular expressions.’ Now they have two problems.”

--Jamie Zawinski, in comp.lang.emacs


Recursive regexen!

regex parens;parens // A balanced set of parens ... = '(' // is an opening paren ... >> // followed by ... *( // zero or more ... keep( +~(set='(',')') ) // of a bunch of things that are // not parens ... | // or ... by_ref(parens) // a balanced set of parens ) // (ooh, recursion!) ... >> // followed by ... ')' // a closing paren ;


A Regex Calculator?!

regex group, fact, term, expr;

group = '(' >> by_ref(expr) >> ')';fact = +_d | group;term = fact >> *(('*' >> fact) | ('/' >>

fact));expr = term >> *(('+' >> term) | ('-' >>

term));


Wife: It's a floor wax!

Husband: No, it's a dessert topping!

Announcer: Stop! You're both right. It's a floor wax and a dessert

topping!

-- Saturday Night Live

C++ library design at the edge


STL, MPL and Fusion, Oh My!

// Just the data, ma’amstd::list<int> integers;

// Just the types, ma’amtypedef mpl::list<int, double, std::string>types;

// Types and data, please!fusion::tuple<int, double, std::string> data = fusion::make_tuple(1, 3.14, "hello");


The Fusion Library

Heterogeneous data structures STL-influenced

– containers, iterators, algorithms

MPL-compatible by Joel de Guzman, part of Boost.Spirit

– http://spirit.sourceforge.net


A Simple Fusion-esque List

struct nil = {};

template<class Car, class Cdr = nil>struct cons { Car car; Cdr cdr; cons(Car const & a, Cdr const & d = Cdr()) : car(a), cdr(d) {}};

inline cons<Car,Cdr>make_cons(Car const & a, Cdr const & d){ return cons(a, d); }


Simple Fusion-esque List, cont.

cons<int,cons<double,cons<std::string> > > data =

make_cons(1, make_cons(3.14, make_cons(std::string("hello"))));


Fusion-esque algorithms

template<typename F>void for_each(nil, F) {}

template<class Car, class Cdr, class F>void for_each(cons<Car, Cdr> const & l, F f){ f(l.car); for_each(l.cdr, f);}


Programming challenge!!!

Write the type of a std::pair<First, Second> where Second is a pointer to the whole std::pair ...

std::pair<int, ??? *>std::pair<int, std::pair<int, ???*> *>std::pair<int, std::pair<int, std::pair<int, ???*> *> *>std::pair<int, std::pair<int, std::pair<int, std::pair<int, std::pair<int, std::pair<int,

std::pair<int, std::pair<int,


Dual-Mode DSEL Design Strategy

recursive algorithms one modular core two binding policies: static and dynamic acyclic data structures


xpressive Matchers

// Match any single characterstruct any_matcher{ template< class Iter, class Next > bool match(Iter i1, Iter i2, Next const &n)

const { if ( i1 == i2 ) { return false; } return n.match( ++i1, i2 ); }};


xpressive static Scaffold

template< class Matcher, class Next >struct static_xpression { Matcher matcher; Next next;

template< class Iter > bool match( Iter i1, Iter i2 ) const { return matcher.match( i1, i2, next ); }};


xpressive dynamic Scaffold

template< class Iter >struct matchable { virtual bool match( Iter, Iter ) const = 0;};


xpr: dynamic Scaffold, cont.

template< class Matcher, class Iter >struct dynamic_xpression : matchable< Iter > { Matcher matcher; matchable< Iter > * pnext;

bool match( Iter i1, Iter i2 ) const { return matcher.match( i1, i2, *pnext ); }};


Separation of Concerns

Matchers implement core functionality Scaffolds implement binding policy (static or

dynamic) Matchers are binding-neutral, reusable Best of both worlds

– perf of static binding– flexibility of dynamic dispatch


References

xpressive:– http://boost-sandbox.sf.net/libs/xpressive

Spirit and Fusion– by Joel de Guzman– http://spirit.sf.net


Questions?

copyright 2005 eric niebler xpressive: library design on the edge or, how i learned to stop worrying...

Documents

eric nieblerhrrmmm

eric nieblerget

compile dd

eric nieblertypes of

setnregex date

eric nieblerwhy dsls

eric nieblerwhy program

term term term