how to build system storage bigfile
DESCRIPTION
How to build system storage bigfileTRANSCRIPT
Architecture
Upload
Download
Applications
Architecture
SBFSession
SBFProfiles
SBFMeta
SBFUserItems
IDZen
Frontend
SBFUpload
FilePoolSBFWork
er
SBFChunks
SBFZenTok
SBFDownload
Java, C++LibEvent, Poco,KyotoCabinet
Upload
SBFSession
SBFProfiles
SBFMeta
SBFUserItems
IDZen
Frontend
SBFUpload
FilePoolSBFWork
er
SBFChunks
1: reqPageUp
SBFZenTok
2b:genTok 3:tok2a:checkUser
SBFDownload
4:pageUp4:up(tok)
5a:check(tok)
6a:genId()
6b:writeFile(id)
7: putMeta
9:notify
8: putUsersFile
10: readFile
12: putChunks
11: genIds
6c: fileId
Download
SBFSession
SBFProfiles
SBFMeta
SBFUserItems
IDZen
Frontend
SBFUpload
FilePoolSBFWork
er
SBFChunks
1: reqPageDown
SBFZenTok
SBFDownload
4:pageDown
2a:checkUser2b:genTok 3:tok
4:down(tok)
5:getMeta
6:getChunks
loop
sendChunks
One more thing
SBFSession
SBFProfiles
SBFMeta
SBFUserItems
IDZen
Frontend
SBFUpload
FilePoolSBFWork
er
SBFChunks
SBFZenTok
SBFDownload
SBFChunks
SBFDownload
Node 1
SBFUpload
FilePoolSBFWork
er
SBFDownload
SBFChunks
Node 2
SBFNodeInfo
SBFBroker
Applications
Issues
Disk Bottleneck
Chunk Size, Sector Size, Block Size
Speed process file slow when concurrent upload high
Bottleneck disk