my trip to the back side - en
TRANSCRIPT
![Page 1: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/1.jpg)
My trip to the
BACK SIDE
![Page 2: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/2.jpg)
@LemaNahuel
I press keys on javascript
![Page 3: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/3.jpg)
That is FWTV?
FWTV is the first Latin WebTV channel with designed exclusively for the Internet, where viewers can interact with programs offered both live and on-demand, anytime, anywhere content.
https://www.fwtv.tv/
![Page 4: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/4.jpg)
@Obaca2015 Me
![Page 5: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/5.jpg)
A long time ago ... (2013-2014)
![Page 6: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/6.jpg)
fwtv.tv
fwtv.tv/api/*
fwtv.tv/admin/*
Heroku with 4 dynos
![Page 7: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/7.jpg)
● Monolithic project● +150 endpoints and growing● callback hell● Frozen dependencies (Node .08, Mongoose,
Redis, Express, etc)
● No Documentation or Tests● Filesystem is no more scalable● Very long Files (average from 1000 to 2500 lines)● ...
![Page 8: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/8.jpg)
... ART* 500 ms - 1000 ms or more … ¬¬
*Average Response Time
![Page 9: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/9.jpg)
![Page 10: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/10.jpg)
Preparing the ground for v2
![Page 11: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/11.jpg)
Se we start with the obvious steps
![Page 12: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/12.jpg)
api.
.tv
admin.
![Page 13: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/13.jpg)
Like miners we start to separate the projects
![Page 14: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/14.jpg)
● Tests implementation● Added documentación● Extreme implementation of EsLint and jsHint● Updated dependencies and Node to 0.12● Restructured the entire filesystem
○ routes/(public || private)/*.js
○ controllers/(public || private)/*.js
● Replaced all long files with small modules● ...
![Page 15: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/15.jpg)
… ART meeh … ¬¬
![Page 16: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/16.jpg)
… but now scale better!
![Page 17: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/17.jpg)
v2.0 > v2.1 - Go functional!
![Page 18: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/18.jpg)
Standardize the use of lodash● aprox 100 implementations of _.each()● more methods implemented as _.pluck(), _.
indexOf(), _.find(), _.without(), _.omit(), etc.
![Page 19: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/19.jpg)
_.each(req.body.genres, (genre) => {
if (_.isObject(genre)) {
newGenres.push(genre._id);
} else {
newGenres.push(genre);
}
});
var genres = req.body.genres,
genre = null,
i = 0,
l = genres && genres.length ? genres.length : 0;
for (i; i < l; i++) {
genre = genres[i];
if (typeof genre === 'object') {
newGenres.push(genre._id);
} else {
newGenres.push(genre);
}
}
For example:_.each()
![Page 20: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/20.jpg)
… ART with no improvements …
![Page 21: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/21.jpg)
… but the code is more readable!
![Page 22: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/22.jpg)
v2.1 > v2.2 - only what you need
![Page 23: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/23.jpg)
Implementation of “Extender”● Now you can specify which fields are required and
which are not.● Used especially for translations and data uses only
the API or the admin.
![Page 24: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/24.jpg)
For example:
function getTranslations() { if (LOCALE.indexOf('es') === -1) { return ',translations(' +LOCALE + '(description))'; }
return '';}
var fields = '?fields='; fields += 'landing'; fields += ',popularShows(_id,logo,slug,title,description,slogan' + getTranslations() + ')';
return $http.get(PATHS.HOME + fields);
![Page 25: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/25.jpg)
Query strings added to Page and Limit● Skip● Limit● LastId
![Page 26: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/26.jpg)
For example:
return $http.get(PATHS.SHOWS, {
params: {
limit: params.limit || 12,
lastId: params.lastId,
skip: params.skip
}
});
![Page 27: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/27.jpg)
… ART barely noticeable …
![Page 28: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/28.jpg)
… but the weight is lowered 100 kb - 250 kb to 1b-30 kb!
![Page 29: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/29.jpg)
v2.2 > v2.3 - the real meaning of parallel
![Page 30: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/30.jpg)
ASYNC● _.each() >
○ async.each()● callback hell >
○ async.waterfall() ○ async.parallel()
![Page 31: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/31.jpg)
For example:async.waterfall()
getHome(function(home) { setPopularAndLiveShows(home, function(popularShows, liveNow, newNow){ home.popularShows = popularShows; home.liveNow = liveNow; home.newNow = newNow;
setFeaturesVideos(home, function(featuredVideos){ home.featuredVideos = featuredVideos; helpers.handleResponse(res, null, home); }); });});
async.waterfall([function(cb) { getHome(function(home) { cb(null, home); });}, function(cb) { setPopularAndLiveShows(home, function(popularShows, liveNow, newNow) { home.popularShows = popularShows; home.liveNow = liveNow; home.newNow = newNow; cb(null, home); });}, function(home, cb) { setFeaturesVideos(home, function(featuredVideos){ home.featuredVideos = featuredVideos; cb(null, home); });}], function(err, home) { helpers.handleResponse(res, null, home);});
![Page 32: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/32.jpg)
For example:async.parallel()
async.parallel([function(cb) { setPopularVideos(home, likedProgramIds, function(popularVideos) { home.popularVideos = popularVideos; cb(null); }); }, function(cb) { setGenres(home, function(genres) { if(genres) { home.genres = genres; } cb(null, home); });}], function(err) { cb(null, home);});
![Page 33: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/33.jpg)
For example:async.each()
async.each(shows, function(show, cb) { ScheduledPromo.count({
program: show._id }).exec(function(err, scheduleds) { Program.findByIdAndUpdate(show._id, { $set: { 'hasElements.scheduled': !!scheduleds } }).exec(function(err) { cb(err); }); }); }, function(err) { helpers.handleResponse(res, null, { success: !err });});
![Page 34: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/34.jpg)
… ART incredible striking …
![Page 35: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/35.jpg)
… it is now 500 ms - 1000 ms to 150 ms - 300 ms!
![Page 36: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/36.jpg)
v2.2 > v3 - Sh*t just got real
![Page 37: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/37.jpg)
Upgrade to Node 4.* and npm dependencies
![Page 38: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/38.jpg)
… ART still the same …
![Page 39: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/39.jpg)
… but consumption of RAM of Heroku decreased significantly!
![Page 40: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/40.jpg)
v3 > v3.1 - sugar Marty ... syntaxis sugar everywhere
![Page 41: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/41.jpg)
Upgrade to Node 5.* and npm dependencies● Minimum implementation of ES6
○ EsLint + ES6
○ const y let
○ arrow functions
○ classes
![Page 42: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/42.jpg)
… ART unimproved and falling consumption of RAM ...
![Page 43: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/43.jpg)
… but the code is much more readable
![Page 44: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/44.jpg)
The journey so far● All up-to-date (Node && NPM)● Functional programming by lodash● Asynchronism, parallelism and callback hell resolved with
async
● ES6 minimally implemented● ART between 100 ms - 300 ms
![Page 45: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/45.jpg)
still something missing …
![Page 46: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/46.jpg)
… review all the access to MongoDB
![Page 47: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/47.jpg)
v3.1 > v3.2 - the final fight!
![Page 48: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/48.jpg)
The scenario:● Made with Mongoose● ~400 queries● most carried out by the front (.tv) and the
apps
![Page 49: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/49.jpg)
I’m Mr. Meeseeks!! Look at me!
![Page 50: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/50.jpg)
Step 1:● +400 queries 1 by 1:
○ .find() >
■ .findById() || .findOne()
○ .update() >
■ .findByIdAndUpdate() || .findOneAndUpdate()
○ .remove() >
■ .findByIdAndRemove() || .findOneAndRemove()
![Page 51: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/51.jpg)
For example:.find() > .findById()
Model.find({ _id: req.params.id }).populate('video grouping program genres').exec((err, doc) => { helpers.handleResponse(res, err, doc);});
Model.update({ _id: req.params.id }, {}).exec((err, doc) => { helpers.handleResponse(res, err, doc);});
Model.remove({ _id: req.params.id }).exec((err, doc) => { helpers.handleResponse(res, err, doc);});
Model.findById(req.params.id) .populate('video grouping program genres') .exec((err, doc) => { helpers.handleResponse(res, err, doc); });
Model.findByIdAndUpdate(req.params.id, {}, { new: true }) .exec((err, doc) => { helpers.handleResponse(res, err, doc); });
Model.findByIdAndRemove(req.params.id) .exec((err, doc) => { helpers.handleResponse(res, err, doc); });
![Page 52: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/52.jpg)
Step 2:● Implemented .lean() method on all .
find*() before .exec()
![Page 53: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/53.jpg)
For example:.lean()
Model.findById(req.params.id) .populate('video grouping program genres') .exec((err, doc) => { helpers.handleResponse(res, err, doc); });
Model.findByIdAndUpdate(req.params.id, {}, { new: true }) .exec((err, doc) => { helpers.handleResponse(res, err, doc); });
Model.findByIdAndRemove(req.params.id) .exec((err, doc) => { helpers.handleResponse(res, err, doc); });
Model.findById(req.params.id) .populate('video grouping program genres') .lean().exec((err, doc) => { helpers.handleResponse(res, err, doc); });
Model.findByIdAndUpdate(req.params.id, {}, { new: true }) .lean().exec((err, doc) => { helpers.handleResponse(res, err, doc); });
Model.findByIdAndRemove(req.params.id) .lean().exec((err, doc) => { helpers.handleResponse(res, err, doc); });
![Page 54: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/54.jpg)
ok, and now ...
![Page 55: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/55.jpg)
… the ART down to 5 ms - 35 ms!! MOTHERF*CKAAA!!!!
![Page 56: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/56.jpg)
Sum-up
![Page 57: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/57.jpg)
● Functional, clean and less code● Lightweight requests● Low consumption of RAM● Efficient queries● Same amount of dynos, greater capacity● Weekly NCU
![Page 58: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/58.jpg)
Questions?
![Page 59: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/59.jpg)
How many Mr. Meeseeks did you find?- 9- 13- 15
![Page 60: My trip to the BACK SIDE - EN](https://reader031.vdocuments.mx/reader031/viewer/2022021918/58a77c381a28abef478b4bb3/html5/thumbnails/60.jpg)
Thanks!
@LemaNahuel