how to build system storage bigfile
DESCRIPTION
How to build system storage bigfileTRANSCRIPT
How to transfer a BigFile
Presenter: Nguyễn Văn TiếnBarcamp, RMIT, [email protected]
Architecture
Upload
Download
Applications
Architecture
SBFSession
SBFProfiles
SBFMeta
SBFUserItems
IDZen
Frontend
SBFUpload
FilePoolSBFWork
er
SBFChunks
SBFZenTok
SBFDownload
Java, C++LibEvent, Poco,KyotoCabinet
Upload
SBFSession
SBFProfiles
SBFMeta
SBFUserItems
IDZen
Frontend
SBFUpload
FilePoolSBFWork
er
SBFChunks
1: reqPageUp
SBFZenTok
2b:genTok 3:tok2a:checkUser
SBFDownload
4:pageUp4:up(tok)
5a:check(tok)
6a:genId()
6b:writeFile(id)
7: putMeta
9:notify
8: putUsersFile
10: readFile
12: putChunks
11: genIds
6c: fileId
Download
SBFSession
SBFProfiles
SBFMeta
SBFUserItems
IDZen
Frontend
SBFUpload
FilePoolSBFWork
er
SBFChunks
1: reqPageDown
SBFZenTok
SBFDownload
4:pageDown
2a:checkUser2b:genTok 3:tok
4:down(tok)
5:getMeta
6:getChunks
loop
sendChunks
One more thing
SBFSession
SBFProfiles
SBFMeta
SBFUserItems
IDZen
Frontend
SBFUpload
FilePoolSBFWork
er
SBFChunks
SBFZenTok
SBFDownload
SBFChunks
SBFDownload
Node 1
SBFUpload
FilePoolSBFWork
er
SBFDownload
SBFChunks
Node 2
SBFNodeInfo
SBFBroker
Applications
Issues
Disk Bottleneck
Chunk Size, Sector Size, Block Size
Speed process file slow when concurrent upload high
Bottleneck disk