thread blocking is evil
DESCRIPTION
This is the simple version of the orinial one addressed at Deview2011.TRANSCRIPT
![Page 1: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/1.jpg)
Thread Blocking Is Evil
Game Network Component Team
Kim, J.H.
![Page 2: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/2.jpg)
CONCURRENCY
![Page 3: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/3.jpg)
Why is Thread Blocking
a Concern?
![Page 4: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/4.jpg)
“The Free Lunch is OVER.”
Herb SutterDecember 2004
![Page 5: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/5.jpg)
Clock speedExecution optimizationCache
![Page 6: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/6.jpg)
Cache
HyperthreadingMulticoreCache
![Page 7: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/7.jpg)
Write Multithreaded Application
![Page 8: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/8.jpg)
CONCURRENCY
![Page 9: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/9.jpg)
Waiting in Line
![Page 10: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/10.jpg)
Thread Blocking
![Page 11: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/11.jpg)
Starvation
Deadlock
![Page 12: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/12.jpg)
Starvation
![Page 13: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/13.jpg)
Task A Task B
1: Lock 1: Sleep 0 msec
2: Sleep 1msec
3: Unlock
![Page 14: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/14.jpg)
Pro
cess
ti
me(m
s)
# of Thread
1 2 3 4 5 6 7 8 9984986988990992994996998
1000
Task BTask A
![Page 15: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/15.jpg)
DeadlockHold & Wait
![Page 16: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/16.jpg)
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
![Page 17: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/17.jpg)
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
![Page 18: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/18.jpg)
My Prob-lem
![Page 19: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/19.jpg)
Framework for Lobby-Room style
Game
![Page 20: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/20.jpg)
Remote Procedure Call
![Page 21: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/21.jpg)
Easy to write code be-cause it’s all contained
inside one function.
![Page 22: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/22.jpg)
Game Client
LobbyRoom‘Enter Room’
RPC‘Authentication’
RPC
![Page 23: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/23.jpg)
Game Client
LobbyRoom
‘Enter Room’
RPC func-tion body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
![Page 24: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/24.jpg)
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
‘Enter Room’
RPC func-tion body
‘Authenti-cation RPC
‘Function
body
![Page 25: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/25.jpg)
Game Client
LobbyRoom
‘Enter Room’ RPC
function body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Response DelayFrom Lobby Server
![Page 26: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/26.jpg)
Game Client
LobbyRoom
‘Enter Room’ RPC
function body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Room Serverthroughput Decline
Response DelayFrom Lobby Server
![Page 27: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/27.jpg)
What is theAxis of Evil?
![Page 28: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/28.jpg)
Thread Block-
ing
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
![Page 29: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/29.jpg)
Inherent Limitation of RPC
![Page 30: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/30.jpg)
Easy to write code be-cause it’s all contained
inside one function.
![Page 31: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/31.jpg)
Can not split code be-cause it’s all contained
inside one function.
![Page 32: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/32.jpg)
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCall(“VerifyLobbyUserToken”, …); …. return result;}
![Page 33: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/33.jpg)
What We Want
![Page 34: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/34.jpg)
RPC Function Body
Non-Blocking
![Page 35: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/35.jpg)
What is the Essence of Problems?
![Page 36: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/36.jpg)
Request & Wait
![Page 37: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/37.jpg)
Solution
![Page 38: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/38.jpg)
Asynchronous
Programming
![Page 39: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/39.jpg)
Request(callback, …)
![Page 40: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/40.jpg)
Pros:Non-Blocking
![Page 41: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/41.jpg)
Cons:Can’t Write Sequential CodeCan’t Use Stack VariableCan’t Split Programming Construct…
![Page 42: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/42.jpg)
Too Difficult!!
![Page 43: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/43.jpg)
Coroutine
![Page 44: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/44.jpg)
Similar to Thread
![Page 45: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/45.jpg)
Line of Execution
![Page 46: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/46.jpg)
Stack&
Local Variable
![Page 47: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/47.jpg)
But
![Page 48: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/48.jpg)
Non-preemptive
![Page 49: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/49.jpg)
Instruction:YieldYield BreakResume
![Page 50: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/50.jpg)
Who does provide Coroutine?
![Page 51: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/51.jpg)
C#ErlangHaskell
JavaScript(since 1.7)LuaPerl
Python(since 2.5)Ruby
…
![Page 52: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/52.jpg)
What about C++?
![Page 53: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/53.jpg)
![Page 54: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/54.jpg)
On Your Own!
![Page 55: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/55.jpg)
Windows: Fiber
Linux: getcontext/setcontextmakecontextswapcontext
![Page 56: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/56.jpg)
Asynchronous Pro-gramming
+Coroutine
![Page 57: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/57.jpg)
Non-Blocking&
Writing Sequential Code
![Page 58: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/58.jpg)
Solution for Starva-
tion
![Page 59: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/59.jpg)
Task A Task B
1: Lock 1: Sleep 0msec
2: Sleep 2msec
3: Unlock
Before
![Page 60: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/60.jpg)
Task A Task B
Thread A 1: Sleep 0msec
1: Create & Resume Coroutine
2: Request Lock
3: Yield
Thread A’
4: Resume
5: Sleep 2msec
6: Yield Break
7: Unlock
Non-Blocking
Sequential Code
After
![Page 61: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/61.jpg)
BeforePro
cess
ti
me(m
s)
# of Thread
1 2 3 4 5 6 7 8 9984986988990992994996998
1000
Task BTask A
![Page 62: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/62.jpg)
AfterPro
cess
ti
me(m
s)
1 2 3 4 5 6 7 8 90
200
400
600
800
1000
1200
Task BTask A
# of Thread
![Page 63: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/63.jpg)
Solution for
DeadlockHold & Wait
![Page 64: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/64.jpg)
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
Before
![Page 65: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/65.jpg)
Thread A Thread B
1A: Lock 1B: Lock
2A: Create & Resume Coroutine 2B: Send Response
3A: Request 3B: Unlock
4A: Yield
5A: Unlock
Thread A’ (invoked by response)
6A’: Lock
7A’: Resume
8A’: Do something with response
9A’: Yield Break
10A’: Unlock
Non-Blocking
Sequential Code
After
![Page 66: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/66.jpg)
Solutionfor
My Problem
![Page 67: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/67.jpg)
Thread Block-
ing
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
![Page 68: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/68.jpg)
Room Thread Lobby Service
1R: Call ‘Authentication’ RPC function 1L: Process the request
2R: Wait for the response 2L: Send back the response
3R: Process
Before
![Page 69: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/69.jpg)
Room Thread Lobby Service
1R: Create & Resume Coroutine 1L: Process the request
2R: Request 2L: Send back the response
3R: Yield
Room Thread’(invoked by response)
4R’: Resume
5R’: Process
6R’: Yield Break
Non-Blocking
Sequential Code
After
![Page 70: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/70.jpg)
Request & Wait
Request & Yield
![Page 71: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/71.jpg)
Non-Blocking&
Writing Sequential Code
![Page 72: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/72.jpg)
Coroutine(Fiber)
Thread
CPU CPU CPU CPU
User-mode SchedulingKernel-mode Scheduling
![Page 73: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/73.jpg)
Corou-tine Pool
Network I/O Thread
Pool
RPC Task Queue
RPC Exe-cution
Thread Pool
![Page 74: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/74.jpg)
RPC Task Queue
RPC Execu-tion Thread
Network I/O Thread
enqueuetask
dequeuetask
acquire corou-tine
enqueue
re-sumetask
dequeue
re-sumetask
resume
yield break
release corou-tine
Network Event
Network Event
Coroutine Pool
pool-ing
corou-tine
RPC Execu-tion
yield return
start
![Page 75: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/75.jpg)
RPC Task Queue
RPC Execu-tion Thread
1
Coroutine Pool
RPC Execu-tion
enqueuetask
dequeuetask
acquire corou-tine corou-
tine
yield return
enqueue
re-sumetask
dequeue
re-sumetask
release corou-tine
Network Event
Network Event
pool-ing
RPC Execu-tion Thread
2
resume
yield break
resume
Network I/O Thread
![Page 76: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/76.jpg)
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCall(“VerifyLobbyUserToken”, …); …. return result;}
![Page 77: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/77.jpg)
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCallYield(“VerifyLobbyUserToken”, …); …. return result;}
![Page 78: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/78.jpg)
TPS
# of Thread
Blocking
Yield
![Page 79: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/79.jpg)
Wrap-up
![Page 80: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/80.jpg)
“The free lunch is OVER.”Write Multithreaded
Application
CONCURRENCY
![Page 81: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/81.jpg)
Thread Blocking is Evil
Starvation Deadlock
![Page 82: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/82.jpg)
Asynchronous Programming
+Coroutine
![Page 83: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/83.jpg)
Non-Blocking+
Writing Sequen-tial Code
![Page 84: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/84.jpg)
Request & Wait
Request & Yield
![Page 85: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/85.jpg)
Inspired by Jeffrey Richter
‘Simplified APM With The Asyn-cEnumerator’
![Page 86: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/86.jpg)
![Page 87: Thread Blocking is Evil](https://reader036.vdocuments.mx/reader036/viewer/2022062511/54c17d7b4a79597d388b45b0/html5/thumbnails/87.jpg)
Pictures fromwww.Istockphoto.comwww.Flickr.com