parsing for fun and profit

Parsingfor Fun and Profit(but mainly fun)

Ash [email protected]

PatchSpace LtdSaturday, 23 February 13

mailto:[email protected]


What?

Saturday, 23 February 13

Parsing

Adding structure and meaning to text


Parsing Human Languages

Jake stretched his legs“Jake”, “stretched”, “his”, “legs”“Jake”<noun>, “stretched”<verb, past>, “his”<possessive pronoun>, “legs”<noun>“Jake” <noun, subject>, “stretched”, (“his”, “legs”)<noun phrase, object>


Parsing Computer Languages

“foo = bar + 123”“foo”, “=”, “bar”, “+”, “123”“foo”<var>, “=”<assignment_op>, “bar”<var>, “+”<op_plus>, “123”<int_literal>


Why?


Not just compiling!Compilers breathe fire.


Pretty PrintingSaturday, 23 February 13

Pretty Printing

gofmt

http://gofmt.com/


http://gofmt.com/

http://gofmt.com/

Code Smell Detectorshttps://rubygems.org/gems/reek


https://rubygems.org/gems/reek

https://rubygems.org/gems/reek

Code Smell DetectorsSaturday, 23 February 13

Other ideasCode metricsBug detectorsDomain-specific languagesLanguage translators (e.g. Ruby -> PHP)Code obfuscatorsAlternative syntaxes (e.g. CoffeeScript)Refactoring tools


How?


Step 13 year computer science

degree


Lexing/Tokenising

if x > 100 then return “big” else return “small”if x > 100 then return “big” else return “small”


Tree Buildingif x > 100 then return “big” else return a + b

if

x

>

100

then

return

“big”

else

return

a+

b


Parsing Expression Grammars

Like regular expressions, but can handle recursion, e.g. HTMLNot actually that much harder to use


Regexes and HTML


Treetop PEG grammarSaturday, 23 February 13

Doing Sums


Switch to Sublime Text, idiot

Code is now available:https://github.com/patchspace/parsing_for_fun_and_profit/


https://github.com/patchspace/parsing_for_fun_and_profit/


A Ruby Syntax Highlighter


What

A tool to read in simple Ruby source and output syntax highlighted HTML


Why

Because I thought it would be funIt wasBecause I thought it would be easy…


Why


HowBuild a parse tree of the Ruby sourceWalk the tree and spit out a <span> element for each bit of textOh yes, make sure each line goes in <div> and <pre> tagsWrap it in <html>And for bonus points, do some fancy method highlighting


Switch to Chrome, idiot


Switch to Sublime Text again, idiot

Code is now available:https://github.com/patchspace/parsing_for_fun_and_profit/




We’re doing this the hard way

Ruby’s grammar is too complex and undefined to easily implement as a PEGTools for parsing Ruby already exist


Ripper (Ruby 1.9.3)Saturday, 23 February 13

Learn more!

Skip theoretical physics, start by playing with Lego


Do moreIdeas you might like to try:

CSV parserJSON parser (return arrays & hashes)XML parserJSON highlighterA simple JavaScript minifier (just kill whitespace)


Thank you

Ash [email protected]

PatchSpace LtdSaturday, 23 February 13



parsing for fun and profit

Technology