plumbin pipelines - a gulp.js workshop

Plumbin' Pipelines with Gulp.js

@Dynatrace

@Ruxit

@ddprrt

fettblog.eu

HTML

CSS

JavaScript

Sass CoffeeScript LESS

P o s t C S S H A M L J a d e

U g l i f y E S 6 R e a c t J S

Browserify AngularJS

Ember CSSMin JSLint

ESHint ImageOptim Mocha

Jasmine TypeScript

1530 lines of code

original Ant tasks used:

concat — copy — delete — mkdir

Let's talk a short bit about the JS build tool

revolution

Grunt started a boom

Gruntfiles get long

Grunt tasks get slow

lots of reads

and writes

And then came Gulp

Disclaimer

I (occasionally) contribute

to Gulp

I'm writing a book on Gulp

http://bit.ly/gulp-tool-book

39% off with 39baumgarcoupon code!

I don't know Java anymore

Java is to JavaScript whatAlf is to Gandalf

Let's go!

The ecosystem

Runtime environment

JavaScript

Package System for Node

Host of all things Gulp plugins

CLI Gulpfile

PluginA

PluginA

CLI Gulpfile

PluginA

PluginA

starts

CLI Gulpfile

PluginA

PluginA

loadsstarts

CLI Gulpfile

PluginA

PluginA

loadsstarts

uses

npm install -g gulp-cli

npm init

npm install --save-dev gulp

touch gulpfile.js

npm install -g gulpjs/gulp-cli#4.0

npm init

npm install --save-dev gulpjs/gulp#4.0

touch gulpfile.js

The basics:

Streams

gulp.src(…) gulp.dest(…)

Reads files Writes files

gulp.src(…) gulp.dest(…)

gulp.src(…) .pipe(uglify()) gulp.dest(…)

gulp.src(…) .pipe(uglify()) gulp.dest(…).pipe(concat())

to the command line!

Gulp API

• gulp.task creates a new task

• It requires to return either a Stream, a Promise or an Observable

• gulp.src “globs” files and returns a stream of virtual file objects

• each file can be piped through a process (jshint, uglify, less, etc.)

• gulp.dest saves the file back to the file system

gulp.task('styles', function() { return gulp.src('app/styles/main.less') .pipe(less()) .pipe(minifyCSS()) .pipe(prefix()) .pipe(gulp.dest('dist/styles')); });

gulp.task('styles', function() { return gulp.src('app/styles/main.less') .pipe(less()) .pipe(minifyCSS()) .pipe(prefix()) .pipe(gulp.dest('dist/styles')); });

defines a new task

gulp.task('styles', function() { return gulp.src('app/styles/main.less') .pipe(less()) .pipe(minifyCSS()) .pipe(prefix()) .pipe(gulp.dest('dist/styles')); });

with a defined name

gulp.task('styles', function() { return gulp.src('app/styles/main.less') .pipe(less()) .pipe(minifyCSS()) .pipe(prefix()) .pipe(gulp.dest('dist/styles')); });

we load a certain file

(or files)

Starting here, we have virtual

files in-memory instead of

real files we

gulp.task('styles', function() { return gulp.src('app/styles/main.less') .pipe(less()) .pipe(minifyCSS()) .pipe(prefix()) .pipe(gulp.dest('dist/styles')); });

and pipe it through a

series of operations

gulp.task('styles', function() { return gulp.src('app/styles/main.less') .pipe(less()) .pipe(minifyCSS()) .pipe(prefix()) .pipe(gulp.dest('dist/styles')); });

before we save it

again on the "real" file system

Tasks

scripts styles lint

scripts

styles

lintgulp.parallel

scripts

styles

lint

gulp.parallel

gulp.series

to the command line!

Gulp API

• The second parameter of gulp.task is always a function.

• gulp.series is a task function that runs tasks in sequential order.

• gulp.parallel is a task function that starts every task concurrently

• Both task functions accept task names and other functions as parameters.

• They can be combined infinitly

gulp.task('default', gulp.series('clean', gulp.parallel('styles', 'scripts'), ‘server' ) );

gulp.task('default', gulp.series('clean', gulp.parallel('styles', 'scripts'), ‘server' ) );

runs in series

gulp.task('default', gulp.series('clean', gulp.parallel('styles', 'scripts'), ‘server' ) );

runs in parallel

A development

environment

gulp.watch

scripts

*.js

Bowser-Sync!

r

to the command line!

Gulp API

• gulp.watch creates a file watcher and listens to changes

• changes include ‘change’, ‘add’, ‘unlink’ and others

• BrowserSync is a development tool that can be fully integrated in Gulp.

• Watchers trigger a browserSync.reload call

function watcher(done) { gulp.watch('styles/**/*.less', gulp.parallel(‘styles’)); done(); }

function watcher(done) { gulp.watch('styles/**/*.less', gulp.parallel(‘styles’)); done(); }

watches this Glob pattern

function watcher(done) { gulp.watch('styles/**/*.less', gulp.parallel(‘styles’)); done(); }

starts this task on

change, unlink, add

gulp.task('server', function(done) { bSync({ server: { baseDir: ['dist', 'app'] } }) done(); });

gulp.task('server', function(done) { bSync({ server: { baseDir: ['dist', 'app'] } }) done(); }); BrowserSync set up to start

a dev server, serving dist and

app statically

gulp.watch('dist/**/*', bSync.reload);

And a watcher that triggers a

reload

Incremental

builds

Some tasks take long

gulp.src(‘scripts/*.js’)

.pipe(uglify())

.pipe(gulp.dest())

.pipe(concat())

Some tasks take long

gulp.src(‘scripts/*.js’)

.pipe(uglify())

.pipe(gulp.dest())

.pipe(concat())

Some tasks take long

gulp.src(‘scripts/*.js’)

.pipe(uglify())

.pipe(gulp.dest())

.pipe(concat())

Too much is going on!

Each change: Uglify all

the files?

Some tasks take long

gulp.src(‘scripts/*.js’)

.pipe(uglify())

.pipe(gulp.dest())

.pipe(concat())

filter files

that have changed

filter files

that have changed

do performance

heavy operations

filter files

that have changed

do performance

heavy operations

remember the

old files

filter files

that have changed

do performance

heavy operations

remember the

old files

and continue

with the other ops

to the command line!

Gulp Plugins

• gulp-cached and gulp-remember can be used to create file caches

• The plugin filters non-changed files and ads them back to the stream once we are done with performance-heavy tasks

• Additionally to that, we can use gulp.lastRun in Gulp 4 to filter files during globbing

• gulp-newer allows us to do incremental copies/builds on a per-file basis

gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js')) .pipe(gulp.dest('dist/scripts')); });

gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js')) .pipe(gulp.dest('dist/scripts')); });

we use the cache to

check if files have

changed

gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js')) .pipe(gulp.dest('dist/scripts')); });

once we are done,

we remember all

the other files

we stored in

the cache

Part II

A short

source map interlude

Browserify

(for Babel/React)

so similar … yet so

different?

Gulp

Streams

Browserify Streams

Why not both??

var b = browserify({ entries: ['_js/main.js'] });

var bundle = function() { return b.bundle() .pipe(source(‘main.js’)) .pipe(buffer()) .pipe(uglify()) .pipe(gulp.dest('js')); }

var b = browserify({ entries: ['_js/main.js'] });

var bundle = function() { return b.bundle() .pipe(source(‘main.js’)) .pipe(buffer()) .pipe(uglify()) .pipe(gulp.dest('js')); }

b.bundle emits a

stream.

But no vinyl file

objects

var b = browserify({ entries: ['_js/main.js'] });

var bundle = function() { return b.bundle() .pipe(source(‘main.js’)) .pipe(buffer()) .pipe(uglify()) .pipe(gulp.dest('js')); }

vinyl-source-stream

wraps the original

stream into

a vinyl file object

var b = browserify({ entries: ['_js/main.js'] });

var bundle = function() { return b.bundle() .pipe(source(‘main.js’)) .pipe(buffer()) .pipe(uglify()) .pipe(gulp.dest('js')); }

vinyl-buffer converts

the stream contents

to a buffer for

plugins who need

such

Stream arrays

and merge Streams

A static

site generator

What does it do?

• Generates static HTML sites

• From a templating engine

• Can parse Markdown

• Can parse HTML

• Can create permalinks

• For different types (posts, pages)

The stack

• We use kramdown to convert Markdown to HTML

• We use nujucks for our templating engine

• We rename posts to feature blog permalink

• We rename pages to resemble pretty URLs

In pipes

gulp.src(‘posts/*.md')

.pipe(kramdown())

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

In pipes

gulp.src(‘posts/*.md')

.pipe(kramdown())

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

gulp.src(‘posts/*.html')

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

In pipes

gulp.src(‘posts/*.md')

.pipe(kramdown())

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

gulp.src(‘posts/*.html')

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

the same!

In pipes

gulp.src(‘pages/*.md')

.pipe(kramdown())

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

gulp.src(‘pages/*.html')

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

In pipes

gulp.src(‘pages/*.md')

.pipe(kramdown())

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

gulp.src(‘pages/*.html')

.pipe(wrap())

.pipe(nunjucks())

.pipe(rename())

.pipe(gulp.dest())

the same!

What if we could

reuse parts of the

stream?

to the command line!

to the command line!

gulp.task('default', function(cb) { var streams = elements.map(function(el) { return merge( gulp.src(el.dir + '/**.md').pipe(markdown()), gulp.src(el.dir + '/**.html') ).pipe(rename(el.renamefn)); }); return merge(streams).pipe(data(options)) .pipe(wrap(layoutStr)) .pipe(swig()) .pipe(gulp.dest('build')); });

gulp.task('default', function(cb) { var streams = elements.map(function(el) { return merge( gulp.src(el.dir + '/**.md').pipe(markdown()), gulp.src(el.dir + '/**.html') ).pipe(rename(el.renamefn)); }); return merge(streams).pipe(data(options)) .pipe(wrap(layoutStr)) .pipe(swig()) .pipe(gulp.dest('build')); });

we combine

multiple sources

to one stream

gulp.task('default', function(cb) { var streams = elements.map(function(el) { return merge( gulp.src(el.dir + '/**.md').pipe(markdown()), gulp.src(el.dir + '/**.html') ).pipe(rename(el.renamefn)); }); return merge(streams).pipe(data(options)) .pipe(wrap(layoutStr)) .pipe(swig()) .pipe(gulp.dest('build')); });

with Array.map

and merge we can

create stream

arrays

Material

Workshop files

https://github.com/frontend-tooling

https://github.com/frontend-tooling/sample-project-gulp

https://github.com/frontend-tooling/static-site-generator

http://fettblog.eu

http://speakerdeck.com/ddprrt

Reading Material

http://bit.ly/gulp-tool-book

39% off with 39baumgarcoupon code!

@ddprrt