Design of Optimal Short-Length LT Codes Using Evolution Strategies

John K. Zao*, Martin Hornansky, Pei-lun DiaoDesign of Optimal Short-Length LT Codes Using Evolution StrategiesWCCI 2012 IEEE World Congress on Computational Intelligence June, 2012 - Brisbane, AustraliaOutline IntroductionLT codesPerformance modelsOptimization problem Optimization methodsOptimization schemeExperiments and resultsIntroductionLT codes :With large symbol blocks ( ) : The asymptotic behaviors have been deduced analytically

With short symbol blocks ( ) : A proficient method for finding the optimal degree distributions is still absent

[8] E. Hyytia, T. Tirronen, and J. Virtamo, Optimal degree distribution for LT codes with smallmessage length,in the 26th IEEE INFOCOM 2007, pp.2576-2580.[9] E. A. Bodine and M. K. Cheng, Characterization of Luby transform codes with small messageSize for low-latency decoding,IEEE International Conference in Communications, 2008.

In this paperTo employ evolution strategies in designing optimal SL-LT codes with decoding performance that suit different applications.A new performance model : Coding overhead Failure ratio rFailure occurrence probability p

How to ensure proper use of the evolution strategiesThe selection of evolution strategiesThe choice of decision variables The specification of fitness functionsThe choice of initial populationThe criteria for selecting population samples in every generation

LT codesA. Encoding and Decoding Operations B. Degree DistributionC. Code PerformanceD. Code Applications

LT codesA. Encoding and Decoding Operations B. Degree Distribution

Ideal soliton distributionRobust soliton distributionC. Code PerformanceThere is a high chance that up to 70% of source symbols may not be recovered if only a small number of codewords were used for decoding.

Reducing the maximum failure rate or the probability of high failure instances.

LT codesLT codesD. Code ApplicationsErasure protection for lossless data transferFile downloadsPerfect data reception among the receiversAs few symbols as possibleData transfer with limited overhead allowanceVideo streamingCan tolerate small amount of decoding failuresCan not tolerate large increase in bandwidth or latencyPostcoding in rateless composite codes Adding precodingOnly need to bring the decoding failure rates below certain threshold

Performance models

Overhead The ratio between the number of extra encoded symbols received and the number of source symbols.

Failure RateThe fraction of the unrecovered source symbols during a decoding process

Failure probabilityThe probability of the decoding failure rate to be higher than a threshold value r while the code is decoded with an overhead .

LT codesD. Code ApplicationsErasure protection for lossless data transfer minimize File downloadsPerfect data reception among the receiversAs few symbols as possibleData transfer with limited overhead allowance reduce rVideo streamingCan tolerate small amount of decoding failuresCan not tolerate large increase in bandwidth or latencyPostcoding in rateless composite codes minimize pAdding precodingOnly need to bring the decoding failure rates below certain threshold

Optimal problemA. Design VariablesB . Fitness Function EvaluationC. Optimization Scenarios

Optimal problem : A. Design Variables

We created two M-tuples :Degree tupleProbability tuple

Each captures a non-trivial entry of the probability mass function:

Degree elements need to be rounded to the closest integers .Several degree elements may arrive at the same value .Some probability elements may become insignificant in the final results.

Optimal problem B . Fitness Function EvaluationEvaluating the performance of different SL-LT code samples in each generation by means of numerical simulation of the actual decoding process.Fitness function f(x)Fitness value

C. Optimization ScenariosFixed Degree Scenario : The components of the degree tuple are kept constant.

Variable Degree Scenario: The design variables consist of the components of both the degree and the probability tuples.

....14Optimization methods

Covariance Matrix Adaptation Evolution StrategyIteratively updating the covariance matrix of a multivariate normal distribution of mutated population. Especially successful in solving badly conditioned, multimodal and noisy problems with high-dimensional rugged search landscape.Natural Evolution StrategyA new numerical optimization method Performing gradient ascent along the natural gradient in the population parameter spacePreventing oscillations, premature convergence, and other undesired effects

Differential EvolutionAn evolution strategyCreating new candidate solutions by combining existing ones according to simple formulae based on vector differences.Good on noisy constrained optimization problems with multidimensional real-valued functions and problems that change over time.

Optimization methodsOptimization scheme : Decision Variables

Transformations between Decision and Design Variables:The design variables d and p , have bounded value rangesWe defined two decision variable tuples :

How to transform ?

CMA-ES and NES are stochastic strategies that use Gaussian distributions to produce random off-springs over an unbounded variable space

DE is a population-based evolution strategy that can evolve its off-springs in a bounded variable space.

Probability Transformation for CMA-ES and NES:

Probability Normalization for DE:

Degree Rounding and Constraining:

Optimization scheme : Decision Variables

Optimization scheme : Decision VariablesCMA-ES and NES can generate unbounded values for their variables affects both, degrees and probabilities.Adding monotonically increasing penalty :

Optimization scheme : Initial Degree DistributionChoices of Initial Degrees:We chose their initial degrees from the range of [1, K/5]Either the prime numbers or the powers of two

Optimization scheme : Specification of Initial ProbabilitiesSparse Robust Soliton Distribution:Gathering probability values under the adjacent degrees of the robust soliton distribution to those under the selected initial degrees .

is the probability mass function of robust soliton distribution

Optimization scheme : Specification of Initial ProbabilitiesSampled Ideal Soliton Distribution:The last component then absorbed the remaining probability [20]:

Uniform Distribution:Every component of the probability tuple was assigned the same value.

[20] G. G. Yan, H. C. Chang, Research on separable UEP-LT code, M.S. thesis, Dept. of Electron. Eng., National Chiao Tung University, Hsinchu, Taiwan, 2007, pp. 45-46.

Experiments and results

Specific Values of Performance MeasurementsWe tried to find optimal SL-LT codes based on

We fixed the values of two performance measurements while optimizing the third one.

Parameters of Optimization Methods CMA-ES and NES Cases:Users only need to specify the initial decision variable values and their standard deviations. For the standard deviations: [10, 30] for decision variables representing degrees [0.02, 0.2] for decision variables representing probabilities

DE Parameters:The optimization problem is continuous, noisy, and exhibit respective problem dimensionality.We assigned 0.5 for crossover probability 0.7 for scaling factorExperiments and results

[19] R. Storn, K. Price, "Differential evolution -- a simple and efficient heuristic for global optimization over continuous spaces". Journal of Global Optimization 11, 1997, pp. 341-359. [20] G. G. Yan, H. C. Chang, Research on separable UEP-LT code, M.S. thesis, Dept. of Electron. Eng., National Chiao Tung University, Hsinchu, Taiwan, 2007, pp. 45-46. [21] M. E. H. Pedersen, Good parameters for differential evolution, Technical Report no. HL1002, Hvass Laboratories, 2010. Comparison of Convergence Behaviors :We traced the fitness function values and average degrees of every generation throughout the evolution process.Experiments and results

Experiments and resultsConsistency among Optimized Performance :

pExperiments and results : Observations

Evolution strategies were shown to be a practical method for designing optimal SL-LT codes.All three strategies are managed to converge and produce degree distributions.CMA-ES and NES showed similar convergence NES appeared to be the most robust evolutionary strategy .NES > CMA-ES > DEThe optimization scenarios with fixed and variable degrees produce similar performance measurement values.