zookeeper presentation
DESCRIPTION
Presentation from Dev TimeTRANSCRIPT
![Page 1: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/1.jpg)
Координация в распределенных системах с помощью Apache
Zookeeper
Маирбек Хадиков
![Page 2: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/2.jpg)
Координация
В распределенных системах не обойтись без
координации
![Page 3: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/3.jpg)
Координация
Система координации должна быть
отказоустойчивой
![Page 4: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/4.jpg)
Координация
Реализовать
синхронизацию в распределенных системах
сложно
![Page 5: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/5.jpg)
Zookeeper
Маштабируемый
Open Source
Fault Tolerant
Распределенный
![Page 6: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/6.jpg)
Задачи
Synchronization ConfigurationMaintenance
Groups Naming
![Page 7: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/7.jpg)
Структура
![Page 8: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/8.jpg)
ZNodeFile
DirectoryWatches
Atomic Operations
OptimisticLocking
![Page 9: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/9.jpg)
Флаги
Sequential
Ephemeral
![Page 10: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/10.jpg)
Sequentialsome-znode-00000001
some-znode-00000002
some-znode-00000003
...
![Page 11: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/11.jpg)
Ephemeral
Время жизни ограниченно временем сессии
![Page 12: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/12.jpg)
ГарантииSequential Consistency
AtomicitySingle System Image
Timelines Reliability
![Page 13: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/13.jpg)
API
create get data
delete
set data
get children
sync
![Page 14: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/14.jpg)
Cluster mode
![Page 15: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/15.jpg)
Cluster mode
Данные сохраняются в памяти на всех серверах
![Page 16: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/16.jpg)
Cluster mode
Лидер выбирается при старте
![Page 17: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/17.jpg)
Cluster mode
Клиенты подключенны к follower'ам
![Page 18: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/18.jpg)
Cluster mode
Все изменения проходят через лидера
![Page 19: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/19.jpg)
Cluster modeCommit после обновления данных на большинстве
серверов
![Page 20: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/20.jpg)
Group Membership
![Page 21: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/21.jpg)
Group MembershipГруппе соответствует определенный
znode
/
...
group
![Page 22: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/22.jpg)
Group MembershipЧлены группы создают ephemeral
znode внутри группы/
...
member
group
![Page 23: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/23.jpg)
Group MembershipЕсли клиент недоступен znode
автоматически удалится/
...
member
group
![Page 24: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/24.jpg)
Barier
![Page 25: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/25.jpg)
BarierКлиент вызывает exists+watcher
/
...
barier
![Page 26: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/26.jpg)
BarierЕсли znode отсутствует барьер
отпущен
/
...
barier
![Page 27: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/27.jpg)
BarierЕсли znode существует, то ожидаем
оповещение о его удалении
/
...
barier
![Page 28: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/28.jpg)
Leader Election
![Page 29: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/29.jpg)
Leader Election
/
...
election
![Page 30: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/30.jpg)
Leader ElectionКаждый желающий создает ephemeral|sequential znode
election
node_00000001
node_00000002
node_00000003
![Page 31: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/31.jpg)
Leader ElectionКлиент у которого znode с наименьшим
номером -- leader
election
node_00000001
node_00000002
node_00000003
![Page 32: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/32.jpg)
Leader ElectionОстальные клиенты ставят watch и следят за
leader
election
node_00000001
node_00000002
node_00000003
![Page 33: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/33.jpg)
Leader ElectionЕсли leader недоступен -- следующий
доступный клиент избирается лидером
election
node_00000001
node_00000004
node_00000007
![Page 34: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/34.jpg)
Lock
![Page 35: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/35.jpg)
LockДля общего ресурса создается znode
/
...
resource
![Page 36: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/36.jpg)
LockКаждый процесс создает
ephemeral|sequential znode
resource
lock_00000001
lock_00000003
lock_00000002
![Page 37: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/37.jpg)
LockПроцесс с наименьшим znode удерживает
lock
lock_00000001
lock_00000003
lock_00000002
resource
![Page 38: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/38.jpg)
LockОстальные процессы ждут удаления
предыдущей znode
lock_00000001
lock_00000003
lock_00000002
resource
![Page 39: Zookeeper presentation](https://reader033.vdocuments.mx/reader033/viewer/2022052601/558b3b32d8b42ac66a8b4626/html5/thumbnails/39.jpg)
LockДля освобождения lock процессом удаляется
znode
lock_00000001
lock_00000003
lock_00000002
resource