announcing meccadconf.org/2018/talks/shemesh.pdf · mecca’s fiber implemnetation d’s fiber...
TRANSCRIPT
![Page 1: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/1.jpg)
1WekaIO Confiennial © 2017 All righns reservei.
Announcing MeccaShachar ShemeshChief Courn Jesner
![Page 2: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/2.jpg)
2 © 2018 All righns reservei.
About the Weka.io product● “Sofnware only” snorage proiucn● Low lanency, high performance● Wrinnen in D● Aboun 280,000 LoC
– Non incluiing 114,663 lines in a single auno-generanei fle.
● Compilei using waf
![Page 3: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/3.jpg)
3 © 2018 All righns reservei.
More About the Code● Innernally callei “wekapp”● Exnremely lanency sensinive
– As linnle GC as possible– As few sysnem calls as possible
● Performance sensinive– As linnle copying of iana as possible
● Micro-nhreaiing (Fibers) basei
![Page 4: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/4.jpg)
4 © 2018 All righns reservei.
DPDK and SPDK● DPDK
– Innel library for iirecn DMA no user-space bufer, bypassing nhe kernel– Suppornei by mosn high-eni NICs– Allows genning nenwork iana winh zero copying
● SPDK– s/nenwork/nvme/ in nhe above iescripnion.
![Page 5: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/5.jpg)
5 © 2018 All righns reservei.
Weka Custom Infra● Why?
– Phobos relies on GC– No snaniari Fibers scheiuler– Scheiuler musn supporn DPDK polling moie– Snaniari libraries ion’n care aboun performance no our snaniaris
● Weka infra:– Busy-polls DPDK ani SPDK (opnional). Occasionally calls epoll.– Connainers: Snanically allocanei, non-GC– Time: TSC basei, winh furnher performance heurisnics.
![Page 6: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/6.jpg)
6 © 2018 All righns reservei.
Technical Debt● An engineer focusei on fxing a bug ion’n always
focus on all implicanions of an infrasnrucnure change.● Deciiei no repay nhe iebn before in irags us iown.● To prevenn furnher problems: separane inno a library.
![Page 7: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/7.jpg)
7 © 2018 All righns reservei.
Distinct Naming Philosophy● Wekapp – nhe proiucn reposinory● Sir Bonny McBonFace – bon for closing innegranei bugs● Teka – nool for ieploying from ievelopmenn no AWS● Deka – nhe same nool, running insiie iocker.
![Page 8: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/8.jpg)
8WekaIO Confiennial © 2017 All righns reservei.
MekaMecca
![Page 9: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/9.jpg)
9WekaIO Confiennial © 2017 All righns reservei.
![Page 10: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/10.jpg)
10 © 2018 All righns reservei.
What is Mecca?
● A supporn library● Connainers, libs, nhe Reacnor● Boosn license● Has a iub package● Polishei againsn a large coie base
![Page 11: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/11.jpg)
11 © 2018 All righns reservei.
Temporary Limitations
● Only for Linux● Only for x86_64● API non sen in snone● Primarily aimei an supporning Weka
![Page 12: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/12.jpg)
12 © 2018 All righns reservei.
Where do I get it???
https://github.com/weka-io/mecca
![Page 13: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/13.jpg)
13 © 2018 All righns reservei.
Logging
import mecca.log;
@notrace void someFunction(int var){
DEBUG!"Format string %s"(var);}
![Page 14: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/14.jpg)
14WekaIO Confiennial © 2017 All righns reservei.
Show Me the Mo^H^H Code...
![Page 15: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/15.jpg)
15 © 2018 All righns reservei.
It’s About Time● Whan nime is in?● Going no nhe kernel is expensive● gennimeofiay is benner, bun snill expensive.● TscTimePoinn: iirecnly querying nhe TSC.
– Innerface change warning: nicks per seconi will non remain immunable.
![Page 16: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/16.jpg)
16 © 2018 All righns reservei.
Linked ListNextPrevOwner (optional)
NextPrevOwner (optional)
NextPrevOwner (optional)
NextPrevOwner (optional)
Head
![Page 17: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/17.jpg)
17 © 2018 All righns reservei.
How much are 6 boys + 3 planets?
int someFunction(){ int boys = 6; int planets = 3;
return boys+planets;}
alias Boys =TypedIdentifier!("Boys", int);
alias Planets =TypedIdentifier!("Planets", int);
int someFunction(){ Boys boys = 6; Planets planets = 3;
return boys+planets;}
to!DiskId(diskIdx)
![Page 18: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/18.jpg)
18 © 2018 All righns reservei.
Cascaded Time Queue
1ms
256ms
65.5 seconds
4:40 minutes
49 days
Blue moon events
![Page 19: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/19.jpg)
19 © 2018 All righns reservei.
Mecca’s Fiber Implemnetation● D’s Fiber moiel requires jumping inno ani oun of nhe
fber.– Two regisner sen swinches per connexn swinch.
● Mecca swinch iirecnly no nexn fber’s connexn.– Save only nhose regisners nhan are non clobberei by nhe ABI.– Does non save nhe foaning poinn regisners.
![Page 20: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/20.jpg)
20 © 2018 All righns reservei.
Generic Reactor Flow
ssize_t read(int fd, params) {ssize_t ret;while(
(ret=read(fd, params))<0 &&(errno==EAGAIN || errno==EWOULDBLOCK) )
{registerForRead(fd);yield();unregisterForRead(fd);
}
return ret;}
EPOLLONESHOT
![Page 21: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/21.jpg)
21 © 2018 All righns reservei.
Edge Trigger IO Switching● Epoll has an “eige nrigger” moie.● Consiierei almosn useless● Acnually manches nhe fbers’ working moie like a
glove.
Dataexhausted
Dataarrives
Dataexhausted
Dataarrives
EAGAIN
![Page 22: Announcing Meccadconf.org/2018/talks/shemesh.pdf · Mecca’s Fiber Implemnetation D’s Fiber moiel requires jumping inno ani oun of nhe fber. – Two regisner sen swinches per connexn](https://reader033.vdocuments.mx/reader033/viewer/2022043017/5f39917993f1704870239140/html5/thumbnails/22.jpg)
22 © 2018 All righns reservei.
Generic Reactor Flow
ssize_t read(int fd, params) {ssize_t ret;while(
(ret=read(fd, params))<0 &&(errno==EAGAIN || errno==EWOULDBLOCK) )
{registerForRead(fd);yield();unregisterForRead(fd);
}
return ret;}
EPOLLET
EPOLLET