the real world - plugging the enterprise into it (nodejs)
DESCRIPTION
NodeConfEu2014 presentationTRANSCRIPT
![Page 1: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/1.jpg)
THE REAL WORLDPlugging The Enterprise Into It.
![Page 2: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/2.jpg)
MOBILE ARCHITECT ‣Love Distributed Systems ‣Entropy Reducer ‣Payment systems ‣R&D Work ‣B2E and Commercial Banking Apps
Experience ‣ Front Office Trading Systems ‣ Messaging Middleware Integration ‣ Big Systems ‣ C/C++/C#/Java
MORE ‣ @akohli https://slideshare.net/akohli
series 2, episode 22, “Daddy Pig’s Office” http://www.channel5.com/shows/peppa-pig/episodes/daddy-pigs-office
![Page 3: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/3.jpg)
TODAYWhy Node
What we want to do
Node as the underpinning of real world or electronic asset interaction
Backing our interactions, eventing services
Not so much about monolith deconstruction
What we did
Initial proxy and protocol
Our performance and scalability testing
![Page 4: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/4.jpg)
![Page 5: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/5.jpg)
NODE IS MAINSTREAM
![Page 6: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/6.jpg)
![Page 7: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/7.jpg)
WHY NODE?✔ Node
• Asynchronous Eventing Model • We live in an async nonblocking
world • Ideal for mobile and sensor
applications • Everyone knows Javascript, right? • Community • Diverse protocol and lots of
modules • Rapid development and
Expediency
![Page 8: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/8.jpg)
HOMOLOGATEDor how we can use it in a big company
• Node is approved for internal usage
• Less Yak Shaving than other solutions
• different at least
• good internal communitybeware of dog, staff only
![Page 9: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/9.jpg)
- @adam_baldwin
“Walmart has had good success with HAPI and Node”
- @ eoinbrazil
“Node is good. I’ve heard good things about HAPI”
![Page 10: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/10.jpg)
ENTERPRISE MOBILE APPLICATIONS
![Page 11: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/11.jpg)
ENTERPRISE MOBILE APPLICATIONS
• Plurality of systems, services • web resources • web sites
• Connectivity challenges • direct • mediated
• Security • AuthN • AuthZ • Data Encryption at rest
![Page 12: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/12.jpg)
THE REFLEKTOR
Security Pass
Employee DevicesSensors
The Physical World
Security Services
AuthZAuthN
…
Eventing Engine Bridge
Payment Services
Access Services
Printing Services
App Services and Resources
the ReflektorBridge and New Services
![Page 13: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/13.jpg)
thePROXY First Release
![Page 14: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/14.jpg)
IT AIN’T EASYbut we gotta try
![Page 15: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/15.jpg)
PROXYING IS EASY
![Page 16: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/16.jpg)
DETAILS
![Page 17: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/17.jpg)
THE FLOW
• The Protocol
• Security - Gateway Access
• Federated Identity, my foot • NTLM, has it’s own approach
![Page 18: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/18.jpg)
PROTOCOLRequest
json body
target
headers
body/post-data
loginfo
request = { URL = "http://www.citigroup.net/", method = "GET", timeout = 19500, clientInfo = { identifier = “…E”, model = "iPad Simulator", systemName = "iPhone OS", systemVersion = "7.1", }, headers = { Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", Cookie = "CGPLNG=ENG; JSESSIONID_CGNR3=..”, "User-Agent" = "Mozilla/5.0 (iPad; CPU OS 7_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Mobile/11D167" }, logEntries = [ { URL = “https://cinternal.site/target/fooa”, downstreamDuration = 656, httpMethod = "GET", roundtripDuration = 3461, statusCode = 200 } ] }
![Page 19: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/19.jpg)
RESPONSE body = “<base64>", code = 200, duration = 31, headers = { "Accept-Ranges" = [ "bytes" ], "Content-Length" = [ 225 ], "Content-Type" = [ "text/html" ], Date = [ "Thu, 29 May 2014 15:28:29 GMT" ], Etag = [ "\"e1-4e50c74f\"" ], "Last-Modified" = [ "Sun, 21 Aug 2011 08:52:31 GMT" ] }, message = "OK"}
![Page 20: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/20.jpg)
NTLM AUTHENTICATIONEnterprise authentication protocol (Microsoft). !NTLM requires all phases to take place across a single HTTP connection. !NTLM messages are sent and received as request headers. !The server’s response from the NTLM type 3 message is the requested content. !This authentication process must be completed for every requested resource, unless an open connection is maintained.
![Page 21: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/21.jpg)
WORKINGImplementation Challenges • Storage of password on mobile device is prohibited,
but is required in the authentication process. • Persistent connection not available. • Latency issues – 3 requests for every web resource.
Solution • Ported from Apache Java implementation to Node.js.
• Hashed username / password pair stored on device, transmitted to server for authentication rather than raw password.
• hmac_md5(username, md4(password))
• NTLM message calculation split between client app and proxy server.
• Defaults used and optional parameters omitted – simplified messages.
• Observed desktop browsers wait for a 401 before beginning the authentication process. Pre-emptively sending the username / password hash eliminates the initial 401 response.
Process is reduced from 3 direct requests to a single client request, mapped to 2 proxy requests.
![Page 22: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/22.jpg)
SCALABILITY AND PERFORMANCE TESTING
![Page 23: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/23.jpg)
GITHUB.COM/SPUMKO/FLOD
$ flod -n 2000 -t 1500 -c 100..1000 -v http://target-place!!
![Page 24: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/24.jpg)
GITHUB.COM/SPUMKO/FLOD
$ flod -n 2000 -t 1500 -c 100..1000 -v http://target-place!!
num req per batch
timeout
range of concurrent request per batch - “rate”
![Page 25: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/25.jpg)
FLOD OUTPUT## 6k page results ec2-user@ip-10-199-51-233 node-hapi]$ flod -n 2000 -t 1500 -c 100..1000 -v http://localhost/loremipsum-6k-ish.htmlThis is Flod, version 0.2.2Copyright 2013 Walmart, http://github.com/spumko/flod!Benchmarking (hold on)...!Server Requests/sec Latency (ms) --------------------------------------- ------------ ---------------http://localhost/loremipsum-6k-ish.html 100 96.48 ± 18.54 http://localhost/loremipsum-6k-ish.html 200 164.24 ± 17.03 http://localhost/loremipsum-6k-ish.html 300 263.80 ± 62.44 http://localhost/loremipsum-6k-ish.html 400 359.61 ± 49.20 http://localhost/loremipsum-6k-ish.html 500 437.66 ± 58.69 http://localhost/loremipsum-6k-ish.html 600 481.29 ± 120.04http://localhost/loremipsum-6k-ish.html 700 606.74 ± 114.45http://localhost/loremipsum-6k-ish.html 800 555.08 ± 133.74http://localhost/loremipsum-6k-ish.html 900 674.08 ± 190.91http://localhost/loremipsum-6k-ish.html 1000 763.27 ± 69.25
## running with high timeout - doubling responses times vs nginx direct [ec2-user@ip-10-199-51-233 node-hapi]$ ../node_modules/flod/bin/flod -n 2000 -t 4500 -c 100..1000 -v http://localhost:8000This is Flod, version 0.2.2Copyright 2013 Walmart, http://github.com/spumko/flod!Benchmarking (hold on)...!Server Requests/sec Latency (ms) --------------------- ------------ ----------------http://localhost:8000 100 200.55 ± 39.40 http://localhost:8000 200 389.54 ± 67.39 http://localhost:8000 300 558.14 ± 112.57 http://localhost:8000 400 777.09 ± 160.01 http://localhost:8000 500 970.61 ± 305.76 http://localhost:8000 600 1032.37 ± 274.44http://localhost:8000 700 1216.49 ± 249.94http://localhost:8000 800 1483.31 ± 690.64http://localhost:8000 900 1559.54 ± 805.31http://localhost:8000 1000 1909.23 ± 845.81
![Page 26: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/26.jpg)
MODIFYING FLOD
• modified server to pull our decorated response timing information
• modified reporting/logging to include this information
• hope to contribute back to mainline
![Page 27: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/27.jpg)
ENVIRONMENT
• HTTP 1.1 no Keep-Alive, request payload is json • Client iOS ObjectiveC;Server is Node + Hapijs (with Some Good Monitoring)
Machine OS Type Processor Cores Memory
Int Server RHEL 6.4 VM Xeon 2.6GHz 2 4GB
Prod ServerWindows
Server 2k8r2
VM Xeon 1.8 Ghz 4 6GB
Dev Mac Mini Full i5 2.5 GHz 2 8 GB
![Page 28: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/28.jpg)
SCENARIOS• Closed network, direct connection,
Mac to Mac
• Client server on a redhat VM, loopback. Redhat VM
• Redhat client to Windows Server via network, Redhat to Windows
• via Mobile network/wifi could only support 100 transactions/s because of latency
Req/s Response (ms)
Mac to Mac 1000 2000
Redhat VM 1000 8500
RH to Windows 1000 30, 000
External 100 17, 000
![Page 29: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/29.jpg)
RESULTS• Consistent proxied service response
• ~20ms Mac ➔ Mac
• ~250ms RHEL ➔ Windows Server
• Gateway service < 50 ms
• We need better concurrency, request servicing
• Infrastructure adds significant overhead
![Page 30: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/30.jpg)
RESULTS
100# 200# 300# 400# 500# 600# 700# 800# 900# 1000#0#
10000#
20000#
30000#
40000#
50000#
60000#
70000#
Requests'per'second'
Millisecon
ds(m
s)'
99th'Percen7les'
Mac#To#Mac# RedhatVM# Redhat#to#Windows#
![Page 31: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/31.jpg)
OFF NETWORK
![Page 32: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/32.jpg)
ON NETWORK
![Page 33: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/33.jpg)
TABULAR Request Mac To Mac RHEL to Windows100/s 483 17,235200/s 783 21,209300/s 1127 17,479400/s 1493 22,937500/s 1859 28,872600/s 2171 34,253700/s 2487 39,878800/s 2878 46,970900/s 3285 51,0831000/s 3555 57,189
![Page 34: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/34.jpg)
EXCELLENT
![Page 35: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/35.jpg)
EXPERIENCE
• Enterprise and Legal approvals hard
• We are ahead of Ops, so waiting for VMs and infrastructure to catch up - software, machines, and network
• Some bits of node need tightening - especially around security and password storage
• Still learning and it is fun!
![Page 36: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/36.jpg)
SCALABILITY PACKETS
• Pile of VMs to auto-scale
• Need elastic environment with a smart load balancer and configuration management
• Great Details on Best practice
• https://gist.github.com/hueniverse/7686452
![Page 37: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/37.jpg)
THANKS!
![Page 38: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/38.jpg)
QUESTIONS? COMMENTS?
![Page 39: The Real World - Plugging the Enterprise Into It (nodejs)](https://reader036.vdocuments.mx/reader036/viewer/2022081401/55943d301a28abde5b8b45a2/html5/thumbnails/39.jpg)
NOUN PROJECTS THANKS
Smartphone designed by James Fenton from the Noun Project ! Creative Commons – Attribution (CC BY 3.0) Identification designed by Mark Shorter from the Noun Project Ibeacon designed by Stéphanie Rusch from the Nount Project ! Creative Commons – Attribution (CC BY 3.0) Arduino designed by uizin from the Noun Project !