learning erlang (from a prolog dropout's perspective)

26-APR-2008 1000speakers:4 by JJ1BDX: Creative Commons CC-BY 3.0 licensed

1

Learning Erlang(from a Prolog dropout's perspective)

Kenji Rikitake, JJ1BDX26-APR-2008

For 1000speakers:4 conference


2

Disclaimers

*Strictly NO WARRANTY(Ab)use this presentation at your own risk

*JJ1BDX is the author and the only responsible person for this presentation

My (former) employers or anyone else have nothing to do with this work and the contents

*This is a work-in-progressYou may find a bunch of errors and glitches


3

Who am I?

*... I've been the JJ1BDX since 1976JJ1BDX is my amateur radio callsign in Japan

*I’ve been an Internet activist since 1986*In 1986, I was partying around with fellow radio and computer hackers, like what you guys are doing now in the 1000speakers conference and other events. So I come here to enjoy. :-)


4

Why I didn't like prolog (in 1980s)

*programming only by matching? *programming without assignment? *you can't really compute numbers? *parallelism on the desktop?*runtime (slow) virtual machines?

...NO WAY!(and I spent too much time on NetNews)


5

...then why Erlang NOW??

*...works fast enough on modern PCsit works OK even on Windoze! (UNIX rules)

*...has a bunch of practical applicationsyaws, ejabberd, ATM packet exchanges

*...can get the most out of parallelismthreads and shared memory are headaches

*...is new to me and I’m sure I can learn something new. That’s for sure.


6

Why Erlang is hard to learn?

*extraordinary syntaxembedded Prolog-ism

Horn clauses and pattern-matching branchesmessage-driven parallelism

*I’m preoccupied by C and FORTRANleaning tail recursion is not a trivial tasklocal variable declaration is not explicit

*lots of new thingsmodules, data structures, Mnesia, OTP...


7


8

What I'm going to show you

*IPv6-related string manipulationgeneration of 10,000 random IPv6 addressesparsing IPv6 colon-notation addresses into Erlang-native tuple formsgenerating reverse-lookup names from the Erlang tuples

colon-form addresses -> ip6.arpa name

*some crude profiling resultswith parallelism: using SMP Erlang VM


9


10

Erlang IP address tuples

*IPv4{127,0,0,1} -> localhost(4 x 8-bit elements)

*IPv6{0,0,0,0,0,0,0,1} -> ::1 (... well, localhost)(8 16-bit big-endian elements)

*Address conversion function readyinet_parse:address() for both IPv4 and IPv6


11


12

Reverse lookup is getting harder in IPv6 than IPv4

*IPv4:just reversing the tuple elements is enough

127.0.0.1 -> 1.0.0.127.in-addr.arpalists:reverse() for the tuple elements

*IPv6:parsing and string manipulation needed

2001:1:3fff:dbde::1 -> 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.e.d.b.d.f.f.f.3.1.0.0.0.1.0.0.2.ip6.arpa... binary bit pack/unpack operation is effective


13


14


15

Concurrency and map()ping

*simple list iteration: map()pingApplying a function to all the list memberslists:map(fun(X) -> function(X) end, Arglist)

*parallelism: parallel map()Joe Armstrong’s pmap()

(in Programming Erlang: Software for a Concurrent World)spawn()ing light-weight process per element preserving result list sequence

caution: execution sequence implementation dependent – no side effect allowed in the function


16

map() .vs. pmap() results* 10,000 addresses / Core2Duo 2.33GHz

(mean values of 5-time measurements)

1.202 seconds(-10.5%)

1.343 secondsSMP (2 cores)(2 schedulers)

2.029 seconds(+55.0%)

1.309 secondsnon-SMP(single scheduler)

Armstrong’spmap()

lists:map()completion timefor test functions


17

Spawning overhead of lightweight processes

*cost of creating lightweight processes3 to 12 microseconds/process

*messaging overhead between processes*more efficient utilization of CPUs needed

granularity: per-function computationnumber of simultaneous processesefficient process spawning

e.g., result list sequence need not be preserved


18

Conclusions and lessons

*Erlang is weird, but worth learninggetting out of the modules is essential

*parallelism works (even for 2 CPUs!)write parallelism-aware functions:

Erlang’s idioms help parallel programming

*programmers need to learn parallelismErlang’s idea applicable to other languageseffective serialization is also critical; some algorithms have to run fast (e.g., crypto)


19

Appendix: lighter function means less parallelism gain

* 10,000 addresses / Core2Duo 2.33GHz(mean values of 5-time measurements)

*Faster implementationio_lib:format() -> primitive hex conversion

0.198 seconds(+53%)

0.129 secondsSMP (2 cores)(2 schedulers)

Armstrong’spmap()

lists:map()completion timefor test functions


20


21

learning erlang (from a prolog dropout's perspective)

Technology