초보자를 위한 분산 캐시 이야기
TRANSCRIPT
![Page 2: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/2.jpg)
잉여서버개발자 In NHN
관심분야!!!
Cloud, Big Data
특기!!!
발표 날로 먹기!!!
![Page 3: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/3.jpg)
What is Cache?
![Page 4: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/4.jpg)
Cache is a component
that transparently stores data so
that future requests for that
data can be served faster.
In Wikipedia
![Page 5: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/5.jpg)
Cache 는 나중에 요청올 결과를 미리 저장해두었다가 빠르게 서비스해 주는 것
![Page 6: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/6.jpg)
Lots of Data
![Page 7: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/7.jpg)
Cache
![Page 8: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/8.jpg)
Cache
Lots of Data
![Page 9: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/9.jpg)
CPU Cache
![Page 10: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/10.jpg)
Browser Cache
![Page 11: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/11.jpg)
Why is Cache?
![Page 12: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/12.jpg)
![Page 13: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/13.jpg)
Use Case: Login
![Page 14: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/14.jpg)
Use Case: Login
Common Case
![Page 15: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/15.jpg)
Use Case: Login
Common Case Read From DB
Select * from Users where id=‘charsyam’;
![Page 16: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/16.jpg)
일반적인 DB구성
Master
Slave
REPLICATION/FailOver
![Page 17: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/17.jpg)
일반적인 DB구성
Master
Slave
REPLICATION/FailOver
모든 Traffic은 Master 가 처리 Slave는 장애 대비
![Page 18: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/18.jpg)
이러니깐 부하가!!!
![Page 19: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/19.jpg)
선택의 기로!!!
![Page 20: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/20.jpg)
Scale UP Vs
Scale OUT
![Page 21: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/21.jpg)
Scale UP
![Page 22: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/22.jpg)
초당 1000 TPS
![Page 23: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/23.jpg)
초당 3000 TPS
3배 처리 가능한 서버를 투입
![Page 24: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/24.jpg)
Scale OUT
![Page 25: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/25.jpg)
초당 1000 TPS
![Page 26: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/26.jpg)
초당 2000 TPS
![Page 27: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/27.jpg)
초당 3000 TPS
![Page 28: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/28.jpg)
Scale Out을 선택!!!
![Page 29: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/29.jpg)
Scale Out을 선택!!! 돈만 많으면 Scale Up도 좋습니다.
![Page 30: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/30.jpg)
Request 분석
읽기 70%, 쓰기 30%
![Page 31: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/31.jpg)
분석 결론
읽기 70%, 읽기를 분산하자!!!
![Page 32: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/32.jpg)
One Write Master
+ Multi Read Slave
![Page 33: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/33.jpg)
Client
Master
Slave
ONLY WRITE
Slave Slave
Only READ
REPLICATION
![Page 34: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/34.jpg)
Eventual Consistency
![Page 35: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/35.jpg)
Ex) Replication Master
Slave
REPLICATION
Slave에 부하가 있거나, 다른 이유로, 복제가 느려질 수 있다. 그러나 언젠가는 같아진다.
![Page 36: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/36.jpg)
Login 정보 중에 잠시라도 서로 다르면 안되는 경우는?
![Page 37: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/37.jpg)
Login 정보 중에 잠시라도 서로 다르면 안되는 경우는? => 다시 처음으로!!!
![Page 38: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/38.jpg)
초반에는 행복했습니다.
![Page 39: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/39.jpg)
Client
Master
Slave Slave Slave
REPLICATION
Slave Slave
![Page 40: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/40.jpg)
부하가 더 커지니!!!
![Page 41: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/41.jpg)
Client
Master
Slave
REPLICATION
Slave Slave Slave Slave Slave
Slave Slave Slave Slave Slave Slave
![Page 42: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/42.jpg)
성능 향상이 미비함!!! WHY?
![Page 43: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/43.jpg)
머신의 I/O는 Zero섬
![Page 44: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/44.jpg)
![Page 45: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/45.jpg)
![Page 46: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/46.jpg)
Partitioning
![Page 47: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/47.jpg)
Client
PART 1
Web Server
DBMS
PART 2
Web Server
DBMS
Scalable Partitioning
![Page 48: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/48.jpg)
Paritioning 성능
![Page 49: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/49.jpg)
Paritioning 성능 관리이슈
![Page 50: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/50.jpg)
Paritioning 성능 관리이슈 비용
![Page 51: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/51.jpg)
간단한 해결책
![Page 52: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/52.jpg)
DB 서버 Disk는 SSD 메모리도 데이터보다 많이
![Page 53: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/53.jpg)
DB 서버 Disk는 SSD 메모리도 데이터보다 많이
=> 돈돈돈!!!
![Page 54: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/54.jpg)
Why is Cache?
![Page 55: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/55.jpg)
Use Case: Login
![Page 56: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/56.jpg)
Use Case: Login
Read From Cache
Get charsyam
![Page 57: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/57.jpg)
Use Case: Login
Apply Cache For Read
![Page 58: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/58.jpg)
General Cache Layer
Cache
DBMS
Storage Layer
Application Server
READ
WRITE
WRITE UPDATE
![Page 59: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/59.jpg)
Type 1: 1 Key – N Items
KEY
Profile:User_ID
Value Profile - LastLoginTime - UserName - Host - Name
![Page 60: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/60.jpg)
Type 2: 1 Key – 1 Item
KEY name:User_ID
Value
UserName
LastLoginTime:User_ID LastLoginTime
![Page 61: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/61.jpg)
Pros And Cons Type 1: 1 Key – N Items
Pros: Just 1 get.
![Page 62: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/62.jpg)
Pros And Cons
Type 1: 1 Key – N Items
Pros: Just 1 get.
Cons: if 1 item is changed. Need Cache Update And Race Condition
![Page 63: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/63.jpg)
Pros And Cons
Type 2: 1 Key – 1 Item
Pros: if 1 item is changed, just change that item.
![Page 64: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/64.jpg)
Pros And Cons
Type 2: 1 Key – 1 Item
Pros: if 1 item is changed, just change that item. Cons: Some Items can be removed
![Page 65: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/65.jpg)
변화하지 않는 데이터
변화하는 데이터
![Page 66: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/66.jpg)
변화하지 않는 데이터
변화하는 데이터
Divide
![Page 67: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/67.jpg)
Don’t Try Update After didn’t Read DB
Value Profile - LastLoginTime - UserName - Host - Name
![Page 68: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/68.jpg)
Don’t Try Update After didn’t Read DB
Value Profile - LastLoginTime - UserName - Host - Name
EVENT: Update Only LastLoginTime
![Page 69: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/69.jpg)
Don’t Try Update After didn’t Read DB
Value Profile - LastLoginTime - UserName - Host - Name
EVENT: Update Only LastLoginTime
Read Cache: User Profile
![Page 70: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/70.jpg)
Don’t Try Update After didn’t Read DB
Value Profile - LastLoginTime - UserName - Host - Name
EVENT: Update Only LastLoginTime
Read Cache: User Profile
Update Data & DB
Just Save Cache
![Page 71: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/71.jpg)
Don’t Try Update After didn’t Read DB
Value Profile - LastLoginTime - UserName - Host - Name
EVENT: Update Only LastLoginTime
Read Cache: User Profile
Update Data & DB
Just Save Cache
It Makes Race Condition
![Page 72: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/72.jpg)
Need Global Lock
![Page 73: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/73.jpg)
Need Global Lock
=> 오늘은 PASS!!!
![Page 74: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/74.jpg)
How to Test
![Page 75: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/75.jpg)
Using Real Data
In 25 million User ID
100,000 Request
Reproduct From Log
![Page 76: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/76.jpg)
Test Result
136 vs 1613 seconds
Memcache VS Mysql
![Page 77: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/77.jpg)
Result of Applying Cache
![Page 78: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/78.jpg)
Select Query
![Page 79: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/79.jpg)
CPU utility
![Page 80: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/80.jpg)
균등한 속도!!!
부하 감소!!!
![Page 81: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/81.jpg)
What is Hard?
![Page 82: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/82.jpg)
Key and Value – SYNC Easy
1. Update To DB
2. Fail To Transaction
Fail!!!
![Page 83: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/83.jpg)
Key and Value – SYNC HARD
1. Update To DB
2. Update to Cache Fail!!!
![Page 84: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/84.jpg)
Key and Value – How
1. Update To DB
2. Update to Cache Fail!!!
RETRY BATCH DELETE
![Page 85: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/85.jpg)
Data! The most important thing
![Page 86: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/86.jpg)
If data is not important
Cache Updating is not important
Login Count
Last Login Time
ETC
![Page 87: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/87.jpg)
BUT!
![Page 88: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/88.jpg)
If data is important
Cache Updating is important
Server Address
Data Path
ETC
![Page 89: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/89.jpg)
HOW!
![Page 90: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/90.jpg)
RETRY! Retry, Retry, Retry Solve Over 9x%. Or Delete Cache!!!
![Page 91: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/91.jpg)
Batch! Queuing Service
![Page 92: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/92.jpg)
Error Log Queue
Error Log
Error Log
Error Log
Error Log
Batch Processor
Cache Server
![Page 93: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/93.jpg)
Error Log Queue
Error Log
Error Log
Error Log
Error Log
Batch Processor
Cache Server
![Page 94: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/94.jpg)
Error Log Queue
Error Log
Error Log
Error Log
Error Log
Batch Processor
Cache Server
UPDATE
![Page 95: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/95.jpg)
Caches
Memcache
Redis
![Page 96: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/96.jpg)
Memcache
Atomic Operation
![Page 97: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/97.jpg)
Memcache
Atomic Operation
Key:Value
![Page 98: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/98.jpg)
Memcache
Atomic Operation
Key:Value Single Thread
![Page 99: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/99.jpg)
Memcache
Over 100,000 TPS Processing
![Page 100: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/100.jpg)
Memcache
Max size of item 1 MB
![Page 101: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/101.jpg)
Memcache
LRU, ExpireTime
![Page 102: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/102.jpg)
Redis
Key:Value
![Page 103: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/103.jpg)
Redis
ExpireTime Replication Snapshot
![Page 104: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/104.jpg)
Redis
Key:Value
Collection
List Sorted Set Hash
![Page 105: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/105.jpg)
주의 사항
![Page 106: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/106.jpg)
Item Size
1~XXX KB, Item 사이즈는 적을수록 좋다.
![Page 107: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/107.jpg)
Cache In
Global Service
![Page 108: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/108.jpg)
Memcached In Facebook • Facebook and Google and Many Companies
• Facebook – 하루 Login 5억명(한달에 8억명)(최신, 밑에는 작년 2010/04자료)
– 활성 사용자 7,000만
– 사용자 증가 비율 4일에 100만명
– Web 서버 10,000 대, Web Request 초당 2000만번
– Memcached 서버 805대 -> 15TB, HitRate: 95%
– Mysq server 1,800 대 Master/Slave(각각, 900대)
• Mem: 25TB, SQL Query 초당 50만번
![Page 109: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/109.jpg)
Cache In Twitter(Old) API WEB
DB DB
Page Cache
![Page 110: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/110.jpg)
Cache In Twitter(NEW) API WEB
DB DB DB
Page Cache
Fragment Cache
Row Cache
Vector Cache
![Page 111: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/111.jpg)
Redis In Wonga
Using Redis for DataStore Write/Read
![Page 112: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/112.jpg)
Redis In Wonga
Flash Client Ruby Backend
![Page 113: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/113.jpg)
NoCache In Wonga
1 million daily Users
200 million daily HTTP Requests
![Page 114: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/114.jpg)
NoCache In Wonga
1 million daily Users
200 million daily HTTP Requests
100,000 DB Operation Per Sec
40,000 DB update Per Sec
![Page 115: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/115.jpg)
NoCache In Wonga
![Page 116: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/116.jpg)
First Scale Out
![Page 117: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/117.jpg)
First Setting – 3 Month
![Page 118: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/118.jpg)
More Traffic
![Page 119: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/119.jpg)
MySQL hiccups – DB Problems Began
![Page 120: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/120.jpg)
Analysis About DBMS
ActiveRecord’s status Check caused 20% extra DB
60% of All Updates were done on ‘tiles’ table
![Page 121: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/121.jpg)
Applying Sharding 2xD
![Page 122: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/122.jpg)
Result of Applying Sharding 2xD
![Page 123: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/123.jpg)
Doubling MySQL
![Page 124: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/124.jpg)
Result of Doubling MySQL
![Page 125: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/125.jpg)
50,000 TPS on EC2
![Page 126: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/126.jpg)
Wonga Choose Redis! But some failure
=> 오늘은 PASS!!!
![Page 127: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/127.jpg)
![Page 128: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/128.jpg)
Result!!!
![Page 129: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/129.jpg)
Consistent Hashing
![Page 130: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/130.jpg)
Proxy
Server
Server
Server
Server
Server
User Request
K = 10000 N = 5
Origin
![Page 131: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/131.jpg)
Proxy
Server
Server
Server
Server
Server
User Request
K = 10000 N = 4
FAIL : Redistribution about 2000 Users
![Page 132: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/132.jpg)
Proxy
Server
Server
Server
Server
Server
User Request
K = 10000 N = 5
RECOVER: Redistribution about 2500 Users
![Page 133: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/133.jpg)
A
Add A,B,C Server
![Page 134: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/134.jpg)
A
B
Add A,B,C Server
![Page 135: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/135.jpg)
A
B
C
Add A,B,C Server
![Page 136: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/136.jpg)
A
B
C
1
Add Item 1
![Page 137: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/137.jpg)
A
B
C
1
2
Add Item 2
![Page 138: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/138.jpg)
A
B
C
1
2
3
4
5
Add Item 3,4,5
![Page 139: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/139.jpg)
A
B
C 2
3
4
5
Fail!! B Server
![Page 140: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/140.jpg)
A
B
C
1
2
3
4
5
Add Item 1 Again -> Allocated C Server
![Page 141: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/141.jpg)
A
B
C
1
2
3
4
5
1
Recover B Server -> Add Item 1
![Page 142: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/142.jpg)
Thank You!
![Page 143: 초보자를 위한 분산 캐시 이야기](https://reader034.vdocuments.mx/reader034/viewer/2022042510/55568f65d8b42a182f8b4e50/html5/thumbnails/143.jpg)
A
B
C
Real Implementation
A+1
A+2
A+3 B+1
B+2
C+1
C+2
A+4
C+3
B+3