service discovery in a microservice architecture using consul
TRANSCRIPT
Service Discovery in a Architecture
using Microservice
Consul
Who Am ICurrently doing Devops, Scala stu�
At Equeris, lean startup within EquensDocker, Consul, Scala, Cloud and other buzzwords.
Email me at: I write at: Twitter:
[email protected]://www.smartjava.org
@josdirksen
Follow alongSources, presentation etc:
Demo heavy sessionhttps://github.com/josdirksen/next-build-consul
Microservices?
What are microservices?Small, �ne-grained easy to replace components.Organized around capabilities.Di�erent languages and backends (whatever �ts best).Fault tolerant, resiliant, automated deployements.
"Small Autonomous services that work together",Sam Newman
From three tier to Microservices
Running Microservices is hardWhere is my other service or database?Am I healthy, is the other one healthy?Where do I store con�guration?How do I handle redundancy and failover?...
, says everyone"Distributed systems are hard"
ServiceDiscovery
Basic approachHardcoded IP Address or LookupDNS
Lookup is nice!Requires managing names (con�g �les), DNS ServerHow to handle failover?
DNS
Now with failoverPoint to a loadbalancerDNS
Works nicely with !How to check health and register services?Programmatic access to LB?
DNS
What would be nice
Does : Lightweight ( , REST) support failoverHas �exible and manages
Lookups DNSHealth checking Con�guration
Consul"Consul [..] provides an opinionated framework for
service discovery and eliminates the guess-workand development e�ort. Clients simply register
services and then perform discovery using a DNSor HTTP interface. Other systems require a home-
rolled solution." - consul.io
Main FeaturesService discovery through REST and DNSSimple registration using REST APIDistributed KV store for con�gurationProvides extensive health checking
All in one package
Good to knowMulti DC-readyAPI for distributed locksEasy HA SetupEvent system
Consul = Zookeeper + Nagios + DNSMasq +Scriptings + ...
Consul Architecture
Service Registration Flow1. Service calls Consul Agent with registration message:
.2. Agent communicates registration with Consul Server3. Agent checks health of Service.4. If check succeeds mark as Healthy, if not mark as Unhealthy,
communicate results with Server5. When a lookup for Service occurs, only return Healthy services
http://agent_host/v1/agent/service/register
Sample: registration message{ "Name": "service1", "address": "10.0.0.12", "port": 8080, "Check": { "http": "http://10.0.0.12:8080/health", "interval": "5s" } }
Send when a new service starts upCheck types: script, http, tcp, TTL, Docker
Sample: DNS Lookup$dig @nb-consul.local backend-service.service.consul
; <<>> DiG 9.8.3-P1 <<>> @nb-consul.local backend-service.service.consul ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27716 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION: ;backend-service.service.consul. IN A
;; ANSWER SECTION: backend-service.service.consul. 0 IN A 10.0.9.3 backend-service.service.consul. 0 IN A 10.0.9.2 backend-service.service.consul. 0 IN A 10.0.9.4
Consul provides a DNS ServerWorks great with Docker ( : will show in Demo)teaser
Sample: REST Lookup$ curl -s http://192.168.99.106:8500/v1/catalog/service/backend-service [{ "Node": "cf2f293e423c", "Address": "192.168.99.111", "ServiceID": "backend-service", "ServiceName": "backend-service", "ServiceAddress": "10.0.9.2", "ServicePort": 8080 },{ "Node": "072b4ea1abc1", "Address": "192.168.99.112", "ServiceID": "backend-service", "ServiceName": "backend-service", "ServiceAddress": "10.0.9.3", "ServicePort": 8080 }]
DEMO
Closer look: DNS LookupIn code:
resp, err := http.Get("http://backend-service:8081/") if err != nil { fmt.Println(err) } else { defer resp.Body.Close() body, _ := ioutil.ReadAll(resp.Body) w.Header().Set("Content-Type",resp.Header.Get("Content-Type")) w.Write(body) }
For docker:
dns: 192.168.99.106 dns_search: service.consul
ConsulEcosystem
Consul templateRender template based on Consul stateSetup reverse proxy: Nginx, Apache, haproxy
global daemon maxconn {{key "service/haproxy/maxconn"}}
defaults mode {{key "service/haproxy/mode"}}{{range ls "service/haproxy/timeouts"}} timeout {{.Key}} {{.Value}}{{end}}
listen http-in bind *:8000{{range service "release.web"}} server {{.Node}} {{.Address}}:{{.Port}}{{end}}
Demo
Envconsul"III. Con�g Store con�g in the environment",
http://12factor.net/
More settings > More complexity
$ envconsul \ -consul demo.consul.io \ -prefix redis/config \ redis-server [opts...]
Vault: help in managing secrets
DEMO ENVCONSUL
DEMO ENVCONSUL
Prometheus & Grafana
More informationlinks:
This presentation, sources and docker stu�:
https://www.consul.io/https://github.com/hashicorp/consul-templatehttps://github.com/hashicorp/envconsul
https://github.com/josdirksen/next-build-consul
Thank You!