clojure at ardoq
DESCRIPTION
Two years ago Ardoq set out to build the documentation platform that the IT-professional wants to use. Starting a “greenfield” project, we were free to choose what we considered to be the best technology for the job, so we ended up using Clojure for all backend components. In this talk I will share our experience using Clojure to build the http://ardoq.com/ documentation platform. A few highlights - Building comprehensive REST-APIs in Clojure - Rapid feedback using REPL driven development - Building a “realtime web” backend using Clojure, WebSockets and Redis. I will also share some of the mistakes we made along the way, as well as lessons learned and tips for anyone starting a new Clojure project.TRANSCRIPT
![Page 1: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/1.jpg)
Ardoq.com @ebaxt
Erik Bakstad
Clojure at
Ardoq
Oslo Socially Functional Programmers
![Page 2: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/2.jpg)
Ardoq.com @ebaxt
! 6 years as consultant (finance/telecom)
! Mostly Java & JavaScript professionally
! Discovered Clojure in 2009
! Groovy, Ruby, Python, SML, Racket, Scala …
! CTO & Co-founder of Ardoq
My background
![Page 3: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/3.jpg)
Ardoq.com @ebaxt
AGENDA
! What is Ardoq?
! Why we decided to use Clojure
! Overview Clojure at Ardoq + 2 Cases
! REPL/development environment
! Experience so far / Q&A / Discussions
![Page 4: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/4.jpg)
Ardoq.com @ebaxt
What is Ardoq?
![Page 5: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/5.jpg)
Ardoq.com @ebaxt
We‘re building larger, more complex systems
! Many different “sub-systems“
! Different technologies
! Wide range of integrations
! System-of-systems problem
![Page 6: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/6.jpg)
Ardoq.com @ebaxt
To achieve conceptual understanding we create artifacts
! Documents
! System maps
! Diagrams
! Flowcharts
These artifacts are hard to maintain!
![Page 7: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/7.jpg)
Ardoq.com @ebaxt
Existing tools are either ...
Enterprise architecture software
Very Complex
Word
Not powerfull enough
Excel PowerPoint
Wikis
V isio
![Page 8: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/8.jpg)
Ardoq.com @ebaxt
Ardoq is a documentation platform that offers
! Explorable, “Living” documentation
! Automatically visualizes dependencies and hierarchies
! REST-api for data import/export
! Supports custom visualizations via plugins
![Page 9: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/9.jpg)
Ardoq.com @ebaxt
DEMO
![Page 10: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/10.jpg)
Ardoq.com @ebaxt
Why Clojure?
![Page 11: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/11.jpg)
Ardoq.com @ebaxt
![Page 12: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/12.jpg)
Ardoq.com @ebaxt
![Page 13: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/13.jpg)
Ardoq.com @ebaxt
Ship software
Quickly Fun!
C onsistently (over time)
Quality
Try out different solutions
Understand How it works!
Leverage
![Page 14: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/14.jpg)
Ardoq.com @ebaxt
JavaScript
MongoDB
Clojure
BSON
JSON
{ "firstname": "Erik", "lastname": "Bakstad", "age": 30, "hobby": ["hockey", "cycling"] }
{ "firstname": "Erik", "lastname": "Bakstad", "age": 30, "hobby": ["hockey", "cycling"] }
{ :firstname "Erik" :lastname "Bakstad" :age 30 :hobby ["hockey", "cycling"] }
EDN
![Page 15: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/15.jpg)
Ardoq.com @ebaxt
! Extensive REST-api
! “Version control” on top of MongoDB
! Real-time web backend
! 3-party integrations (recurly, jira, segment.io, +++)
! Attachment service with pluggable storage provider
Stuff we‘ve built in Clojure
![Page 16: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/16.jpg)
Ardoq.com @ebaxt
Architecture
![Page 17: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/17.jpg)
Ardoq.com @ebaxt
Backbone.js !D3.js!jQuery!Require.js! JavaScript-app!
Clojure!backend!
MongoDB!! Redis!
H T TP Websockets SNS!S3!
Segment.io!Mailchimp!Mandrill!Recurly!JIRA!Tarsnap!!
H T TP
TCP
Plugins!JavaDoc / Excel! H T TP
![Page 18: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/18.jpg)
Ardoq.com @ebaxt
api!
com.ardoq.*
utils! service!
gateway!persistence!
MongoDB!!
SNS!S3!
Segment.io!Mailchimp!Mandrill!Recurly!JIRA!!!
hAp://www.infoq.com/presentaGons/DDDIClojure$
![Page 19: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/19.jpg)
Ardoq.com @ebaxt
Libraries
! Carmine
! amazonica
! core.async
! Pantomime
! clj-http/chime/clostach
! Ring/compojure/Liberator
! http-kit/clj-wamp
! Friend/oauthentic/clj-oauth
! clj-schema/antisamy
! Monger/ragtime
![Page 20: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/20.jpg)
Ardoq.com @ebaxt
Testing/debugging
! Started out with midje
! Now using clojure.test
! ring.mock
! Lein test selectors
! Mongodb + redis
! clojure.tools.trace
! IntelliJ with Cursive
! Embedded REPL
! criterium
![Page 21: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/21.jpg)
Ardoq.com @ebaxt
Editor/IDE
! Emacs + Cider
! clojure-mode + paredit
! clojure-test-mode
! clojure-refactor (just started)
! company-cider
![Page 22: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/22.jpg)
Ardoq.com @ebaxt
Infrastructure/monitoring
! S3 (wagon)
! SNS
! Zapier
! Tarsnap
! HipChat
! AWS / on-site
! Ansible /python/bash
! Circle-CI
! Logstash + Kibana
! Pingdom
![Page 23: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/23.jpg)
Ardoq.com @ebaxt
Case 1
REST-api
![Page 24: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/24.jpg)
Ardoq.com @ebaxt
Request Response
Servlet C ontainer Servlet AP I
Ring
Application
![Page 25: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/25.jpg)
Ardoq.com @ebaxt
{:scheme(:http,((:session({}((:request3method(:get,((:query3string("q=hello"((:cookies({}((:uri("/test"((:headers(({"accept3encoding"("gzip,(deflate"}((:body(...}$
(defn%handler([request](...()$
{:status(200((:headers({"Content3Type"((((((((((((("text/plain"}((:body("Hello(World!"}$
![Page 26: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/26.jpg)
Ardoq.com @ebaxt
(defn%middleware([handler]((((()$
![Page 27: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/27.jpg)
Ardoq.com @ebaxt
(defn%middleware([handler]((((fn%[req](((...((((handler(req)(((...())$
![Page 28: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/28.jpg)
Ardoq.com @ebaxt
(defroutes(api((((GET("/"([](...)((((POST("/"([](...))(((def%handler(((((logging((session((security(app))))((((run3jetty(handler({:port(8080}))$
![Page 29: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/29.jpg)
Ardoq.com @ebaxt
(defroutes(api((((GET("/"([](...))(
?
![Page 30: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/30.jpg)
Ardoq.com @ebaxt
![Page 31: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/31.jpg)
Ardoq.com @ebaxt
Liberator
hAp://clojureIliberator.github.io/liberator/$
(defroutes(api((((GET("/"([](...))(
Decision graph
![Page 32: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/32.jpg)
Ardoq.com @ebaxt
Decision graph
hAp://clojureIliberator.github.io/liberator/assets/img/decisionIgraph.svg$
(resource((((((((((:allowed3methods([:get]((((((((((:allowed?((fn%[_]((permission/is3admin?))((((((((((:available3media3types(["application/json"]((((((((((:exists?((fn%[_]({::result(...)((((((((((:handle3ok(::result)))$
![Page 33: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/33.jpg)
Ardoq.com @ebaxt
Authentication
hAps://github.com/cemerick/friend$
! Basic Auth
! Token Auth
! Oauth (Log in with Github, Google, Live Connect)
![Page 34: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/34.jpg)
Ardoq.com @ebaxt
Friend
hAps://github.com/cemerick/friend$
![Page 35: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/35.jpg)
Ardoq.com @ebaxt
Friend
hAps://github.com/cemerick/friend$
{:workflows([(ajax3login)(((((((((((((((token3workflow(:realm("Ardoq"(:credential3fn(...)(((((((((((((((workflows/http3basic(:realm("Ardoq")(((((((((((((((github3oauth(system)(((((((((((((((live3connect3oauth(system)(((((((((((((((google3oauth(system)]((:credential3fn((partial(verify3credentials(system)((:login3uri("/login"((:unauthorized3redirect3uri("/login"((:default3landing3uri("/index.html"}$
![Page 36: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/36.jpg)
Ardoq.com @ebaxt
Middleware
! wrap-impersonate
! wrap-organization
! wrap-session-expiry
! clojure-metrics
![Page 37: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/37.jpg)
Ardoq.com @ebaxt
Case 2
Real-time web
![Page 38: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/38.jpg)
Ardoq.com @ebaxt
API-1! API-2!
Nginx-1! Nginx-2!
Browser-1! Browser-2!POST$/api/component${“name”:$“foo”}$
NoGfy$user$of$BrowserI1’s$POST$
![Page 39: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/39.jpg)
Ardoq.com @ebaxt
Why not use something like Pusher.app or firebase?
![Page 40: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/40.jpg)
Ardoq.com @ebaxt
The Web Application Messaging Protocol
hAp://wamp.ws/$
![Page 41: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/41.jpg)
Ardoq.com @ebaxt
API-1! API-2!
Nginx-1! Nginx-2!
Browser-1! Browser-2!
Websocket + Wamp
Http-kit + cl j-wamp
Autobahn.js + f lash-fal l back
?
![Page 42: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/42.jpg)
Ardoq.com @ebaxt
API-1! API-2!
Nginx-1! Nginx-2!
Browser-1! Browser-2!
Redis PubSub + core.async Redis!
![Page 43: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/43.jpg)
Ardoq.com @ebaxt
Publish!Event! Redis!
Carmine!listener!
Core.async!mult-channel!
(loop! …! (wamp/send-event …)! (recur))!!
Browser-2!Browser-1!
Workspace-event Organization- event
dao!
![Page 44: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/44.jpg)
Ardoq.com @ebaxt
Publish!Event! Redis!
Carmine!listener!
Core.async!mult-channel!
(loop! …! (wamp/send-event …)! (recur))!!
Analytics!
Testing!
dao!
![Page 45: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/45.jpg)
Ardoq.com @ebaxt
REPL
Driven Development
![Page 46: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/46.jpg)
Ardoq.com @ebaxt
http://thinkrelevance.com/blog/2013/06/04/clojure-workflow-reloaded
https://github.com/juxt/jig
https://github.com/stuartsierra/component
https://www.youtube.com/watch?v=13cmHf_kt-Q
![Page 47: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/47.jpg)
Ardoq.com @ebaxt
The Clojure experience
So far
![Page 48: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/48.jpg)
Ardoq.com @ebaxt
![Page 49: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/49.jpg)
Ardoq.com @ebaxt
Other benefits
Tiny codebase Fast feedback Libraries (both Java and Clojure) Concurrency
![Page 50: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/50.jpg)
Ardoq.com @ebaxt
Challenges
Dynamic language - tradeoffs Lack of signatures LISP – with great power comes…
![Page 51: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/51.jpg)
Ardoq.com @ebaxt
Thank you!
Questions?
![Page 52: Clojure at ardoq](https://reader033.vdocuments.mx/reader033/viewer/2022052410/554baccab4c905ae618b54cb/html5/thumbnails/52.jpg)
Ardoq.com @ebaxt
THANK YOU!