file fragmentation over an unreliable network - bachelor thesis · 2009-10-06 · bachelor thesis...
TRANSCRIPT
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
File Fragmentation over an UnreliableNetwork
Bachelor Thesis
Martin [email protected]
Royal Institute of Technology
September 7, 2009
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Imagine yourself playing an action game
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
There is always a risk of getting caught, which increasesby time
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
AcknowledgmentsIf you get caught, you have to start over where you lastsaved
But saving very frequently would cost too much time
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
AcknowledgmentsIf you get caught, you have to start over where you lastsaved
But saving very frequently would cost too much time
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Basic model
A file is sent over an unreliable network
Large files are more likely to be corrupted than small
If a file is corrupted, the whole file needs to be resent
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Basic model
A file is sent over an unreliable network
Large files are more likely to be corrupted than small
If a file is corrupted, the whole file needs to be resent
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Basic model
A file is sent over an unreliable network
Large files are more likely to be corrupted than small
If a file is corrupted, the whole file needs to be resent
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Large files are likely to be corrupted
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Basic model
To prevent file losses and heavy-tailes, large files can befragmented
Smaller fragments are less likely to be corrupted
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Basic model
To prevent file losses and heavy-tailes, large files can befragmented
Smaller fragments are less likely to be corrupted
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Fragmentation prevents corruption
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Packets however, carry an overhead
This makes too small packet sizes inefficient
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Thesis
What is the optimal fragment size?
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Mathematical model
xi + φ denotes the packet size (xi data size, φ overheadsize)
Transmission will succeed if Ai ≥ xi + φ (Ai randomvariable with distribution f)
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Mathematical model
xi + φ denotes the packet size (xi data size, φ overheadsize)
Transmission will succeed if Ai ≥ xi + φ (Ai randomvariable with distribution f)
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Expected transmission time
J =M−1∑i=0
xi + φ
F̄ (xi + φ)=
M−1∑i=0
h(xi ) F̄ (x) = P(Ai > x) (1)
Want to minimize J over M and xi
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Expected transmission time
J =M−1∑i=0
xi + φ
F̄ (xi + φ)=
M−1∑i=0
h(xi ) F̄ (x) = P(Ai > x) (1)
Want to minimize J over M and xi
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Minimizing J
The KKT condition for (1) is
h′(xi ) =1
F̄ (xi + φ)+ (xi + φ)
f (xi + φ)(F̄ (xi + φ)
)2 = λ
M−1∑i=0
xi = L (2)
Lemma
If the failure rate (failure risk) f (x)/F̄ (x) is increasing:
The optimization over xi is convex
The solution is x∗i = x∗ for all i (packet sizes are equal)
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Minimizing J
The KKT condition for (1) is
h′(xi ) =1
F̄ (xi + φ)+ (xi + φ)
f (xi + φ)(F̄ (xi + φ)
)2 = λ
M−1∑i=0
xi = L (2)
Lemma
If the failure rate (failure risk) f (x)/F̄ (x) is increasing:
The optimization over xi is convex
The solution is x∗i = x∗ for all i (packet sizes are equal)
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Minimizing J
The KKT condition for (1) is
h′(xi ) =1
F̄ (xi + φ)+ (xi + φ)
f (xi + φ)(F̄ (xi + φ)
)2 = λ
M−1∑i=0
xi = L (2)
Lemma
If the failure rate (failure risk) f (x)/F̄ (x) is increasing:
The optimization over xi is convex
The solution is x∗i = x∗ for all i (packet sizes are equal)
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Minimizing J
The KKT condition for (1) is
h′(xi ) =1
F̄ (xi + φ)+ (xi + φ)
f (xi + φ)(F̄ (xi + φ)
)2 = λ
M−1∑i=0
xi = L (2)
Lemma
If the failure rate (failure risk) f (x)/F̄ (x) is increasing:
The optimization over xi is convex
The solution is x∗i = x∗ for all i (packet sizes are equal)
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Minimizing J
Seek minimizer M∗ by inserting x∗ = L/M into (1):
J = MLM + φ
F̄(
LM + φ
) = LLM + φ
LM F̄
(LM + φ
) := L·g(
L
M
)(3)
Lemma
Under the assumption that the failure rate f (x)/F̄ (x) isincreasing, g(x) is unimodal
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Minimizing J
Seek minimizer M∗ by inserting x∗ = L/M into (1):
J = MLM + φ
F̄(
LM + φ
) = LLM + φ
LM F̄
(LM + φ
) := L·g(
L
M
)(3)
Lemma
Under the assumption that the failure rate f (x)/F̄ (x) isincreasing, g(x) is unimodal
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Minimizing J
Seek minimizer M∗ by inserting x∗ = L/M into (1):
J = MLM + φ
F̄(
LM + φ
) = LLM + φ
LM F̄
(LM + φ
) := L·g(
L
M
)(3)
Lemma
Under the assumption that the failure rate f (x)/F̄ (x) isincreasing, g(x) is unimodal
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Bounding x∗
Definition
a := argminx
(g(x))
Theorem
If L > a:
a
2≤ a
1 + a/L≤ x∗(L) ≤ min
(2a,
a
1− a/L
)
Corollary
limL→∞
x∗(L) = a
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Bounding x∗
Definition
a := argminx
(g(x))
Theorem
If L > a:
a
2≤ a
1 + a/L≤ x∗(L) ≤ min
(2a,
a
1− a/L
)
Corollary
limL→∞
x∗(L) = a
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Bounding x∗
Definition
a := argminx
(g(x))
Theorem
If L > a:
a
2≤ a
1 + a/L≤ x∗(L) ≤ min
(2a,
a
1− a/L
)
Corollary
limL→∞
x∗(L) = a
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Bounding x∗
Definition
a := argminx
(g(x))
Theorem
If L > a:
a
2≤ a
1 + a/L≤ x∗(L) ≤ min
(2a,
a
1− a/L
)
Corollary
limL→∞
x∗(L) = a
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Conclusion
Under the assumption that the failure rate f (x)/F̄ (x) isincreasing,
Constant fragmentation is optimal
The fragment sizes are unique
If the file size and distribution f are known, the file canbe pre-fragmented
For large filesizes L, the optimal fragment size isindependent of L
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Conclusion
Under the assumption that the failure rate f (x)/F̄ (x) isincreasing,
Constant fragmentation is optimal
The fragment sizes are unique
If the file size and distribution f are known, the file canbe pre-fragmented
For large filesizes L, the optimal fragment size isindependent of L
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Conclusion
Under the assumption that the failure rate f (x)/F̄ (x) isincreasing,
Constant fragmentation is optimal
The fragment sizes are unique
If the file size and distribution f are known, the file canbe pre-fragmented
For large filesizes L, the optimal fragment size isindependent of L
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Conclusion
Under the assumption that the failure rate f (x)/F̄ (x) isincreasing,
Constant fragmentation is optimal
The fragment sizes are unique
If the file size and distribution f are known, the file canbe pre-fragmented
For large filesizes L, the optimal fragment size isindependent of L
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Conclusion
Under the assumption that the failure rate f (x)/F̄ (x) isincreasing,
Constant fragmentation is optimal
The fragment sizes are unique
If the file size and distribution f are known, the file canbe pre-fragmented
For large filesizes L, the optimal fragment size isindependent of L
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Acknowledgments
Steven H. Low, Jayakrishnan Nair (Caltech) andLachlan Andrew (Swinburne)
Henrik Sandberg and Karl Henrik Johansson (KTH)
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Acknowledgments
Steven H. Low, Jayakrishnan Nair (Caltech) andLachlan Andrew (Swinburne)
Henrik Sandberg and Karl Henrik Johansson (KTH)
File Fragmentationover an Unreliable
Channel
Martin [email protected]
Introduction
Model and method
Conclusion
Acknowledgments
Acknowledgments
Steven H. Low, Jayakrishnan Nair (Caltech) andLachlan Andrew (Swinburne)
Henrik Sandberg and Karl Henrik Johansson (KTH)