don't be afraid of abstract syntax trees
DESCRIPTION
ASTs are an incredibly powerful tool for understanding and manipulating JavaScript. We'll explore this topic by looking at examples from ESLint, a pluggable static analysis tool, and Browserify, a client-side module bundler. Through these examples we'll see how ASTs can be great for analyzing and even for modifying your JavaScript. This talk should be interesting to anyone that regularly builds apps in JavaScript either on the client-side or on the server-side.TRANSCRIPT
![Page 1: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/1.jpg)
Don’t Be Afraid of
ASTsJamund Ferguson
![Page 2: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/2.jpg)
Our Basic Plan
1. High-level overview
2. Static Analysis with ASTs
3. Transforming and refactoring
4. A quick look at the Mozilla Parser API
(de-facto standard AST format)
![Page 3: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/3.jpg)
An abstract syntax tree
is basically a DOM for
your code.
![Page 4: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/4.jpg)
An AST makes it easier to
inspect and manipulate
your code with confidence.
![Page 5: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/5.jpg)
{"type": "Program","body": [
{"type": “VariableDeclaration","kind": "var""declarations": [
{"type": "VariableDeclarator","id": {
"type": "Identifier","name": "fullstack"
},"init": {
"type": "BinaryExpression","left": {
"type": "Identifier","name": "node"
},"operator": "+","right": {
"type": "Identifier","name": "ui"
}}
}],
}]
}
![Page 6: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/6.jpg)
var fullstack = node + ui;
![Page 7: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/7.jpg)
{"type": "Program","body": [
{"type": “VariableDeclaration","kind": "var""declarations": [
{"type": "VariableDeclarator","id": {
"type": "Identifier","name": "fullstack"
},"init": {
"type": "BinaryExpression","left": {
"type": "Identifier","name": "node"
},"operator": "+","right": {
"type": "Identifier","name": "ui"
}}
}],
}]
}
![Page 8: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/8.jpg)
Things Built On ASTs
• Syntax Highlighting
• Code Completion
• Static Analysis (aka
JSLint, etc.)
• Code Coverage
• Minification
• JIT Compilation
• Source Maps
• Compile to JS
Languages
So much more…
![Page 9: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/9.jpg)
Static Analysis
![Page 10: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/10.jpg)
It’s not just about
formatting.
![Page 11: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/11.jpg)
Fix a bug. Add a unit test.
Fix a similar bug…
![Page 12: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/12.jpg)
Write some really
solid static analysis.
Never write that same
type of bug again.
![Page 13: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/13.jpg)
function loadUser(req, res, next) {User.loadUser(function(err, user) {
req.session.user = user;next();
});}
Bad Example
We forgot to handle the error!
![Page 14: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/14.jpg)
![Page 15: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/15.jpg)
1. Each time a function is declared check
if there is an error* parameter
If so set a count to 0;
Increment count when error is used
At the end of the function warn when
count is empty
* the parameter name can be defined by the user
handle-callback-err
![Page 16: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/16.jpg)
Static Analysis
• Complexity Analysis
• Catching Mistakes
• Consistent Style
![Page 17: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/17.jpg)
History Lesson
• 1995: JavaScript
• 2002: JSLint started by Douglas Crockford
• 2011: JSHint comes out as a fork of JSLint.
Esprima AST parser released.
• 2012: plato, escomplex, complexity-report
• 2013: Nicholoas Zakas releases ESLint. Marat
Dulin releases JSCS.
![Page 18: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/18.jpg)
My static analysis tool
of choice is ESLint.
![Page 19: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/19.jpg)
![Page 20: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/20.jpg)
JSHintMixes the rule engine with the parser
![Page 21: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/21.jpg)
Examples
![Page 22: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/22.jpg)
no-console
return {"MemberExpression": function(node) {
if (node.object.name === "console") {context.report(node, "Unexpected console statement.”);
}}
};
https://github.com/eslint/eslint/blob/master/lib/rules/no-console.js
![Page 23: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/23.jpg)
no-loop-func
function checkForLoops(node) {var ancestors = context.getAncestors();if (ancestors.some(function(ancestor) {
return ancestor.type === "ForStatement" ||ancestor.type === "WhileStatement" ||ancestor.type === "DoWhileStatement";
})) {context.report(node, "Don't make functions within a loop");
}}
return {"FunctionExpression": checkForLoops,"FunctionDeclaration": checkForLoops
};
![Page 24: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/24.jpg)
max-params
var numParams = context.options[0] || 3;
function checkParams(node) {if (node.params.length > numParams) {
context.report(node, "This function has too many parameters ({{count}}). Maximum allowed is {{max}}.", {
count: node.params.length,max: numParams
});}
}
return {“FunctionDeclaration”: checkParams,“FunctionExpression”: checkParams
}
![Page 25: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/25.jpg)
no-jquery
function isjQuery(name) {return name === '$' || name === 'jquery' || name === 'jQuery';
}
return {“CallExpression”: function(node) {
var name = node.callee && node.callee.name;if (isjQuery(name)) {
context.report(node, 'Please avoid using jQuery here.’);}
}}
![Page 26: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/26.jpg)
More Complex Rules
• indent
• no-extend-native
• no-next-next
• security
• internationalization
![Page 27: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/27.jpg)
Other Areas for Static
Analysis
Code complexity and visualization is another
area where static analysis is really useful. Plato is
an exciting start, but I believe there are tons of
more interesting things that can be done in this
area.
![Page 28: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/28.jpg)
Recap
• Static Analysis can help you catch real bugs and keep
your code maintainable
• ESLint and JSCS both use ASTs for inspecting your
code to make it easy to cleanly to add new rules
• Static analysis can also help you manage your code
complexity as well
• What exactly does a for loop sound like?
![Page 29: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/29.jpg)
Transforms
![Page 30: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/30.jpg)
Sometimes you want to step
into the future, but something
is keeping you in the past.
![Page 31: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/31.jpg)
Maybe it’s Internet
Explorer
![Page 32: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/32.jpg)
Maybe it’s the size of
your code base
![Page 33: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/33.jpg)
ASTs to the rescue!
![Page 34: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/34.jpg)
Tools like falafel and recast give
you an API to manipulate an AST
and then convert that back into
source code.
![Page 35: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/35.jpg)
Two Types of AST
TransformationsRegenerative
Regenerate the full file from the AST. Often losing
comments and non-essential formatting. Fine for code
not read by humans (i.e. browserify transforms).
Partial-source transformation
Regenerate only the parts of the source that have
changed based on the AST modifications. Nicer for
one-time changes in source.
![Page 36: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/36.jpg)
Build a Simple
Browserify Transform
![Page 37: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/37.jpg)
var fullstack = node + ui;var fullstack = node + browserify;
![Page 38: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/38.jpg)
4 Steps
1. Buffer up the stream of source code
2. Convert the source into an AST
3. Transform the AST
4. Re-generate and output the source
![Page 39: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/39.jpg)
Step 1
var through = require(‘through');var buffer = [];
return through(function write(data) {buffer.push(data);
}, function end () {var source = buffer.join(‘’);
});
Use through to grab the source code
![Page 40: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/40.jpg)
Step 2
var falafel = require(‘falafel’);function end () {
var source = buffer.join(‘’);var out = falafel(source, parse).toString();
}
Use falafel to transform create an AST
![Page 41: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/41.jpg)
Step 3
function parse(node) {if (node.type === 'Identifier' &&
node.value === ‘ui’) {node.update('browserify');
}}
Use falafel to transform the AST
![Page 42: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/42.jpg)
Step 4
function end () {var source = buffer.join(‘’);var out = falafel(source, parse).toString();this.queue(out);this.queue(null); // end the stream
}
Stream the source with through and close the stream
![Page 43: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/43.jpg)
var through = require('through');var falafel = require('falafel');
module.exports = function() {var buffer = [];return through(function write(data) {
buffer.push(data);}, function end() {
var source = buffer.join('\n');var out = falafel(source, parse).toString();this.queue(out);this.queue(null); // close the stream
});};
function parse(node) {if (node.type === 'Identifier' &&
node.name === 'ui') {node.update('browserify');
}}
![Page 44: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/44.jpg)
It Works!
browserify -t ./ui-to-browserify.js code.js
(function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=typeof require=="function"&&require;if(!u&&a)return a(o,!0);if(ivar fullstack = node + browserify;},{}]},{},[1]);
![Page 45: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/45.jpg)
Lots of code to do
something simple?
![Page 46: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/46.jpg)
Probably, but…
It will do exactly what is
expected 100% of the time.
![Page 47: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/47.jpg)
And it’s a building block
for building a bunch of
cooler things.
![Page 48: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/48.jpg)
What sort of cooler
things?
![Page 49: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/49.jpg)
How about
performance?
![Page 50: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/50.jpg)
![Page 51: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/51.jpg)
V8 doesn’t do it, but
there’s nothing stopping
you*.
![Page 52: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/52.jpg)
*Except it’s hard.
![Page 53: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/53.jpg)
A Basic Map/Filter
var a = [1, 2, 3];var b = a.filter(function(n) {
return n > 1;}).map(function(k) {
return k * 2;});
![Page 54: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/54.jpg)
Faster Like This
var a = [1, 2, 3];var b = [];
for (var i = 0; i < a.length; i++) {if (a[i] > 1) {
b.push(a[i] * 2);}
}
![Page 55: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/55.jpg)
![Page 56: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/56.jpg)
A Basic Recast Script
var recast = require(‘recast’);var code = fs.readFileSync(‘code.js', 'utf-8');var ast = recast.parse(code);var faster = transform(ast);var output = recast.print(faster).code;
![Page 57: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/57.jpg)
function transform(ast) {var transformedAST = new MapFilterEater({
body: ast.program.body}).visit(ast);
return transformedAST;}
var Visitor = recast.Visitor;var MapFilterEater = Visitor.extend({
init: function(options) {},visitForStatement: function(ast) {},visitIfStatement: function(ast) {},visitCallExpression: function(ast) {},visitVariableDeclarator: function(ast) {}
});
![Page 58: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/58.jpg)
How Does it Work?
1. Move the right side of the b declaration into a for loop
2. Set b = []
3. Place the .filter() contents inside of an if statement
4. Unwrap the .map contents and .push() them into b5. Replace all of the local counters with a[_i]
![Page 59: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/59.jpg)
![Page 60: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/60.jpg)
![Page 61: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/61.jpg)
![Page 62: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/62.jpg)
![Page 63: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/63.jpg)
And Voila….
var a = [1, 2, 3];var b = [];
for (var i = 0; i < a.length; i++) {if (a[i] > 1) {
b.push(a[i] * 2);}
}
![Page 64: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/64.jpg)
Worth the effort?
YES!
![Page 65: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/65.jpg)
The most well-read documentation
for how to engineer your app is the
current codebase.
![Page 66: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/66.jpg)
If you change your code,
you can change the
future.
![Page 67: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/67.jpg)
KnowledgeWhat is an AST and what does it look like?
![Page 68: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/68.jpg)
Parser
1. Read your raw JavaScript source.
2. Parse out every single thing that’s happening.
3. Return an AST that represents your code
![Page 69: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/69.jpg)
Esprima is a very popular*
parser that converts your code
into an abstract syntax tree.
*FB recently forked it to add support for ES6 and JSX
![Page 70: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/70.jpg)
Parsers
narcissus
ZeParser
Treehugger
Uglify-JS
Esprima
Acorn
![Page 71: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/71.jpg)
![Page 72: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/72.jpg)
Esprima follows the Mozilla Parser
API which is a well documented AST
format used internally by Mozilla (and
now by basically everyone else*)
![Page 73: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/73.jpg)
var fullstack = node + ui;
![Page 74: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/74.jpg)
{"type": "Program","body": [
{"type": “VariableDeclaration","kind": "var""declarations": [
{"type": "VariableDeclarator","id": {
"type": "Identifier","name": "fullstack"
},"init": {
"type": "BinaryExpression","left": {
"type": "Identifier","name": "node"
},"operator": "+","right": {
"type": "Identifier","name": "ui"
}}
}],
}]
}
![Page 75: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/75.jpg)
{"type": "Program","body": [
{"type": “VariableDeclaration","kind": "var""declarations": [
{"type": "VariableDeclarator","id": {
"type": "Identifier","name": "fullstack"
},"init": {
"type": "BinaryExpression","left": {
"type": "Identifier","name": "node"
},"operator": "+","right": {
"type": "Identifier","name": "ui"
}}
}],
}]
}
![Page 76: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/76.jpg)
Node Types
SwitchCase (1)
Property (1)
Literal (1)
Identifier (1)
Declaration (3)
Expression (14)
Statement (18)
![Page 77: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/77.jpg)
Expression Types
• FunctionExpression
• MemberExpression
• CallExpression
• NewExpression
• ConditionalExpression
• LogicalExpression
• UpdateExpression
• AssignmentExpression
• BinaryExpression
• UnaryExpression
• SequenceExpression
• ObjectExpression
• ArrayExpression
• ThisExpression
![Page 78: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/78.jpg)
Statement Types• DebuggerStatement
• ForInStatement
• ForStatement
• DoWhileStatement
• WhileStatement
• CatchClause
• TryStatement
• ThrowStatement
• ReturnStatement
• SwitchStatement
• WithStatement
• ContinueStatement
• BreakStatement
• LabeledStatement
• IfStatement
• ExpressionStatement
• BlockStatement
• EmptyStatement
![Page 79: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/79.jpg)
Debugging ASTs
![Page 80: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/80.jpg)
• When debugging console.log(ast) will not print a
large nested AST properly. Instead you can use
util.inspect:
var util = require('util');var tree = util.inspect(ast, { depth: null });console.log(tree);
• When transforming code start with the AST you
want and then work backward.
• Often this means pasting code using the Esprima
online visualization tool or just outputting the trees
into JS files and manually diffing them.
![Page 81: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/81.jpg)
Oftentimes it helps to print out the code
representation of a single node.
In recast you can do:var source = recast.prettyPrint(ast, { tabWidth: 2 }).code;
In ESLint you can get the current node with:var source = context.getSource(node)
![Page 82: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/82.jpg)
ASTs can turn your
code into play-dough
![Page 83: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/83.jpg)
Totally worth the effort!
![Page 84: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/84.jpg)
the end
@xjamundx
![Page 85: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/85.jpg)
Related
- http://pegjs.majda.cz/
- https://www.kickstarter.com/projects/michaelficarra/make-a-better-coffeescript-compiler
- http://coffeescript.org/documentation/docs/grammar.html
- https://github.com/padolsey/parsers-built-with-js
- https://github.com/zaach/jison
- https://github.com/substack/node-falafel - eprima-based code modifier
- https://github.com/substack/node-burrito - uglify-based AST walking code-modifier
- https://github.com/jscs-dev/node-jscs/blob/e745ceb23c5f1587c3e43c0a9cfb05f5ad86b5ac/lib/js-file.js - JSCS’s way of walking the AST
- https://www.npmjs.org/package/escodegen - converts an AST into real code again
- https://www.npmjs.org/package/ast-types - esprima-ish parser
- http://esprima.org/demo/parse.html - the most helpful tool
https://github.com/RReverser/estemplate - AST-based search and replace
https://www.npmjs.org/package/aster - build system thing
Technical Papers
http://aosd.net/2013/escodegen.html
Videos / Slides
http://slidedeck.io/benjamn/fluent2014-talk
http://vimeo.com/93749422
https://speakerdeck.com/michaelficarra/spidermonkey-parser-api-a-standard-for-structured-js-representations
https://www.youtube.com/watch?v=fF_jZ7ErwUY
https://speakerdeck.com/ariya/bringing-javascript-code-analysis-to-the-next-level
Just in Time Compilers
http://blogs.msdn.com/b/ie/archive/2012/06/13/advances-in-javascript-performance-in-ie10-and-windows-8.aspx
https://blog.mozilla.org/luke/2014/01/14/asm-js-aot-compilation-and-startup-performance/
https://blog.indutny.com/4.how-to-start-jitting
Podcasts
http://javascriptjabber.com/082-jsj-jshint-with-anton-kovalyov/
http://javascriptjabber.com/054-jsj-javascript-parsing-asts-and-language-grammar-w-david-herman-and-ariya-hidayat/
![Page 86: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/86.jpg)
RANDOM EXTRA
SLIDES
![Page 87: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/87.jpg)
![Page 88: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/88.jpg)
Static analysis tools like ESLint
and JSCS provide an API to let you
inspect an AST to make sure it’s
following certain patterns.
![Page 89: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/89.jpg)
![Page 90: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/90.jpg)
![Page 91: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/91.jpg)
![Page 92: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/92.jpg)
![Page 93: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/93.jpg)
function isEmptyObject( obj ) {for ( var name in obj ) {return false;
}return true;
}
![Page 94: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/94.jpg)
static analysis > unit testing > functional testing
![Page 95: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/95.jpg)
![Page 96: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/96.jpg)
function loadUser(req, res, next) {User.loadUser(function(err, user) {
if (err) { next(err);
}req.session.user = user;next();
});}
Another Example
![Page 97: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/97.jpg)
Abstract Christmas Trees
Program
VariableDeclarator
FunctionExpression
ExpressionStatement
Identifier
Identifier
VariableDeclaration
![Page 98: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/98.jpg)
![Page 99: Don't Be Afraid of Abstract Syntax Trees](https://reader031.vdocuments.mx/reader031/viewer/2022020218/55966c871a28abf0748b45c8/html5/thumbnails/99.jpg)
Tools like falafel and recast give
you an API to manipulate an AST
and then convert that back into
source code.