Aviasales: миграция поискового движка в docker / Дмитрий Кузьменков (Aviasales)

Download Aviasales: миграция поискового движка в docker / Дмитрий Кузьменков (Aviasales)

Post on 16-Apr-2017

415 views

Category:

Engineering

7 download

TRANSCRIPT

  • Docker

    www.aviasales.ru

  • aviasales.ru

  • ++ 1500000

    aviasales.ru

  • ++ 1500000

    ++ 1 000

    aviasales.ru

  • ++ 1500000

    ++ 200 realtime

    ++ 1 000

    aviasales.ru

  • ++ 1500000

    ++ 15000

    ++ 200 realtime

    ++ 1 000

    aviasales.ru

  • ++ 1500000

    ++ 15000

    ++ 200 realtime

    ++ 1

    ++ 1 000

    aviasales.ru

  • Yasen ?

  • Yasen ?++ Yet Another Search ENgine

  • Yasen ?++ Yet Another Search ENgine

    ++ Python 3.4 & Tornado 4.2

  • Yasen ?++ Yet Another Search ENgine

    ++ 8

    ++ Python 3.4 & Tornado 4.2

  • Yasen ?++ Yet Another Search ENgine

    ++ 8

    ++ Python 3.4 & Tornado 4.2

    ++ 200 ms

  • HAProxy

  • HAProxy

    bee.0

    bee.1

    bee.2

    bee.3

  • HAProxy

    bee.0

    bee.1

    bee.2

    bee.3

    queue

  • HAProxy

    bee.0

    bee.1

    bee.2

    bee.3

    ant.0

    ant.1queue

  • HAProxy

    bee.0

    bee.1

    bee.2

    bee.3

    ant.0

    ant.1

    Redis

    queue

  • HAProxy

    bee.0

    bee.1

    bee.2

    bee.3

    MySQL

    Postgres

    Redis

    RabbitMQ

    ant.0

    ant.1

    Redis

    queue

  • HAProxy

    bee.0

    bee.1

    bee.2

    bee.3

    MySQL

    Postgres

    Redis

    RabbitMQ

    ant.0

    ant.1

    Redis

    watcher configs

    queue

  • (bee)

  • (bee)++ HTTP-

  • (bee)++ HTTP-

    ++ Tornado

  • (bee)++ HTTP-

    ++ Tornado

    ++ 30

  • (bee)++ HTTP-

    ++ Tornado

    ++ 30

    ++ 16

  • (watcher)

  • (watcher)++ 1

  • (watcher)++ 1

    ++

  • (watcher)++ 1

    ++

    ++ runtime

  • (ant)

  • (ant)++ : low, normal, high

  • (ant)++ : low, normal, high

    ++ Redis

  • (ant)++ : low, normal, high

    ++ Redis

    ++

  • (ant)++ : low, normal, high

    ++ Redis

    ++

    ++ Stateless

  • (burlesque)

  • (burlesque)++ 1

  • (burlesque)++ 1

    ++ Go + leveldb

  • (burlesque)++ 1

    ++ Go + leveldb

    ++ 3000+ rps concurrent read/write

  • (burlesque)++ 1

    ++ Go + leveldb

    ++ 3000+ rps concurrent read/write

    ++ HTTP-

  • (burlesque)++ 1

    ++ Go + leveldb

    ++ 3000+ rps concurrent read/write

    ++ HTTP-

    ++ Opensource: https://github.com/KosyanMedia/burlesque

    https://github.com/KosyanMedia/burlesque

  • ?

  • ?++

  • ?

    ++ 2

    ++

  • ?

    ++ 2

    ++ Monit

    ++

  • ?

    ++ 2

    ++ Chef

    ++ Monit

    ++

  • ?

  • ?

  • ?++

  • ?++

    ++

  • ?++

    ++ DevOps

    ++

  • ?++

    ++ DevOps

    ++ production

    ++

  • ?++

    ++ DevOps

    ++ production

    ++

    ++

  • ?

  • ++ Kubernetes

    ?

  • ++ Kubernetes

    ++ Rancher

    ?

  • ++ Kubernetes

    ++ Rancher

    ++ Swarm

    ?

  • Kubernetes

  • Kubernetes++ By Google

  • Kubernetes++ By Google

    ++

  • Kubernetes++ By Google

    ++

    ++

  • Rancher

  • Rancher++ web-

  • Rancher++ web-

    ++

  • Rancher++ web-

    ++

    ++

  • Swarm

  • Swarm++ Docker

  • Swarm++ Docker

    ++

  • Swarm++ Docker

    ++

    ++

  • ++

  • ++

    ++

  • ++

    ++

    ++

  • ++

    ++

    ++

    ++

  • ++

    ++

    ++

    ++

    ++ , ,

  • Docker !

  • Docker !++ docker build

  • Docker !++ docker build

    ++ docker-compose

  • Docker !++ docker build

    ++ docker-compose

    ++ BASH

  • Docker !++ docker build

    ++ docker-compose

    ++ profit!

    ++ BASH

  • Dockerfile

  • Dockerfile ++

  • Dockerfile ++

    ++ RUN

  • Dockerfile ++

    ++ RUN

    ++ COPY

  • Dockerfile ++

    ++ RUN

    ++ COPY

    ++ ENV

  • Dockerfile ++

    ++ RUN

    ++ COPY

    ++ ENV

    ++ To be continued...

  • ? !

  • ? !++ Root

  • ? !++ Root

    ++

  • ? !++ Root

    ++

    ++

  • ? !++ Root

    ++

    ++

    ++ COPY root

  • ? !++ Root

    ++

    ++

    ++ COPY root

    ++ COPY ,

  • root

  • root++ Root = root host-

  • ++ root

    root++ Root = root host-

  • ++ root

    root++ Root = root host-

    ++ - root

  • ++ root

    root++ Root = root host-

    ++ - root

    ++ Non-root

  • Linux capabilities

  • Linux capabilities ++ chown

    ++ dac_override

    ++ fowner

    ++ fsetid

    ++ kill

    ++ setgid

    ++ setuid

    ++ setpcap

    ++ net_bind_service

    ++ net_raw

    ++ sys_chroot

    ++ mknod

    ++ audit_write

    ++ setfcap

    ++ man capabilities(7)

  • ++ --cap-add & --cap-drop

  • ++ --cap-add & --cap-drop

    ++ --cap-drop=all,

  • ++ --cap-add & --cap-drop

    ++ --cap-drop=all,

    ++ --privileged

  • stateful?

  • stateful?++ stateless

  • stateful?++ stateless

    ++

  • stateful?++ stateless

    ++

    ++ etcd, consul, redis

  • stateful?++ stateless

    ++

    ++ etcd, consul, redis

    ++ -

  • ++ host-

  • ++ host-

    ++ VOLUME

  • ++

  • ++

    ++ ARG USER_ID=1000 for the rescue

  • ++

    ++ ARG USER_ID=1000 for the rescue

    ++ docker build --build-arg USER_ID="$(id -u)"

  • User namespaces

  • User namespaces++ 1.10

  • User namespaces++ 1.10++ Root = non uid-0 host-

  • User namespaces++ 1.10++ Root = non uid-0 host-++

  • User namespaces++ 1.10++ Root = non uid-0 host-++ ++ --userns-remap

  • ++ Bridge

  • ++ Bridge

    ++ Host

  • ++ Bridge

    ++ Host

    ++ Overlay

  • ++ Bridge

    ++ Host

    ++ Overlay

    ++ Macvlan

  • ++ Bridge

    ++ Host

    ++ Overlay

    ++ Macvlan

    ++ None

  • Network mode: bridge

  • Network mode: bridge++ NAT

  • Network mode: bridge++ NAT

    ++ IP

  • Network mode: bridge++ NAT

    ++ IP

    ++

  • Network mode: bridge++ NAT

    ++ IP

    ++

    ++ iptables

  • Network mode: host

  • Network mode: host++ -

  • Network mode: host++ -

    ++ Host port = container port

  • Network mode: host++ -

    ++ Host port = container port

    ++ iptables Docker

  • Network mode: host++ -

    ++ Host port = container port

    ++ iptables Docker

    ++ !

  • Network mode: overlay

  • Network mode: overlay++ Multi-host

  • Network mode: overlay++ Multi-host

    ++ Swarm mode (1.12+)

  • Network mode: overlay++ Multi-host

    ++ Swarm mode (1.12+)

    ++

  • Network mode: macvlan

  • Network mode: macvlan++ 1.12, kernel v3.93.19 and 4.0+

  • Network mode: macvlan++ 1.12, kernel v3.93.19 and 4.0+

    ++

  • Network mode: macvlan++ 1.12, kernel v3.93.19 and 4.0+

    ++

    ++ MAC

  • Network mode: macvlan++ 1.12, kernel v3.93.19 and 4.0+

    ++

    ++ MAC

    ++ , gateway

  • Network mode: macvlan++ 1.12, kernel v3.93.19 and 4.0+

    ++

    ++ - IP

    ++ MAC

    ++ , gateway

  • Network mode: none

  • Network mode: none++ !

  • Network mode: none++ !

    ++

  • Limit everything

  • Limit everything++ CPU: --cpu-shares, --cpuset-cpus

  • Limit everything++ CPU: --cpu-shares, --cpuset-cpus

    ++ Memory: --memory, --memory-reservation

  • Limit everything++ CPU: --cpu-shares, --cpuset-cpus

    ++ Memory: --memory, --memory-reservation

    ++ Swap: --memory-swap, --memory-swappiness

  • Limit everything++ CPU: --cpu-shares, --cpuset-cpus

    ++ Memory: --memory, --memory-reservation

    ++ Swap: --memory-swap, --memory-swappiness

    ++ Storage: --device-read-bps, --device-write-bps

  • Limit everything++ CPU: --cpu-shares, --cpuset-cpus

    ++ Memory: --memory, --memory-reservation

    ++ Swap: --memory-swap, --memory-swappiness

    ++ Storage: --device-read-bps, --device-write-bps

    ++

  • ++ docker-compose.yml

  • ++ docker-compose.yml

    ++ YAML template

  • ++ docker-compose.yml

    ++ YAML template

    ++

  • ++ docker-compose.yml

    ++ YAML template

    ++

    ++ scale bee=10 ant=1 goqueue=1

  • ++ docker-compose

  • ++ docker-compose

    ++ %%

  • ++ docker-compose

    ++ %%

    ++

  • ++ docker-compose

    ++ %%

    ++

    ++ env.sh

  • ++ docker-compose

    ++ %%

    ++

    ++ env.sh

    ++ BASH

  • ++ %SEQUENCE%

  • ++ %SEQUENCE%

    ++ ${VAR} env.sh

  • ++ %SEQUENCE%

    ++ ${VAR} env.sh

    ++ export BEE_PORT=${BEE_PORT:-7089}

  • ++ git

  • ++ git

    ++ docker-compose.yml

  • ++ git

    ++ docker-compose.yml

    ++

  • ++ git

    ++ docker-compose.yml

    ++

    ++ docker-compose --no-build up

  • ?

  • ?++ , , scale cluster.yml

  • ?++ , , scale cluster.yml

  • ++ grep production cluster.yml | cut -d: -f1

  • ++ grep production cluster.yml | cut -d: -f1

    ++ deploy-

  • ++ grep production cluster.yml | cut -d: -f1

    ++ deploy-

    ++

  • ++ grep production cluster.yml | cut -d: -f1

    ++ deploy-

    ++

    ++ BASH: background processes & wait

  • ++ grep production cluster.yml | cut -d: -f1

    ++ deploy-

    ++

    ++ BASH: background processes & wait

    ++ Success or fail? slack

  • ++ grep production cluster.yml | cut -d: -f1

    ++ deploy-

    ++

    ++ BASH: background processes & wait

    ++ Success or fail? slack

  • ++ 50 -

  • ++ 150 -

    ++ 50 -

  • ++ 150 -

    ++ 500 BASH' 5-

    ++ 50 -

  • ++ 150 -

    ++ 500 BASH' 5-

    ++

    ++ 50 -

  • ++ 150 -

    ++ 500 BASH' 5-

    ++

    ++ dev-

    ++ 50 -

  • ++ 8 ++ 24

  • ++ 8 ++ 24

    ++ Chef, monit, Centos 6 ++ Ubuntu 16 LTS (4.4)

  • ++ 8 ++ 24

    ++ Chef, monit, Centos 6 ++ Ubuntu 16 LTS (4.4)

  • ++ 8 ++ 24

    ++ Chef, monit, Centos 6 ++ Ubuntu 16 LTS (4.4)

  • ++ 8 ++ 24

    ++ Chef, monit, Centos 6 ++ Ubuntu 16 LTS (4.4)

  • dmitrii@kuzmenkov.me

    dmitrykuzmenkov

  • ? dmitrii@kuzmenkov.me

    dmitrykuzmenkov