adaptive coding from a diffusion process on the integer line robert ellis october 26, 2009

Adaptive Coding from a DiffusionProcess on the Integer Line

Robert Ellis

October 26, 2009

Joint work with Joshua Cooper,University of South Carolina

Outline of Talk

Coding theory overview– Packing (error-correcting) & covering codes– Coding as a 2-player game– Liar game and pathological liar game

Diffusion processes on Z– Simple random walk (linear machine)– Liar machine– Pathological liar game, alternating question strategy

Improved pathological liar game bound– Reduction to liar machine– Discrepancy analysis of liar machine versus linear machine

Concluding remarks

2

Coding Theory Overview

Codewords:fixed-length strings from a finite alphabet

Primary uses: Error-correction for transmission in the presence of noiseCompression of data with or without loss

Viewpoints:Packings and coverings of Hamming balls in the hypercube2-player perfect information games

Applications:Cell phones, compact disks, Mars Reconnaissance Orbiter

3

Transmit blocks of length n

Noise changes≤ e bits per block(||||1 ≤ e)

Repetition code 111, 000– length: n = 3 – e = 1– information rate: 1/3

Coding Theory: (n,e)-Codes

x1…xn

(x1+1)…(xn+ n)

110 010 000

000

101

000 111111

Received:

Decoded:

blockwise majority vote

Richard Hamming

4

0010011

3 errors: incorrect decoding

Coding Theory – A Hamming (7,1)-Code

1 0 0 0 1 1 1 0 1 1 0 1 1 0

0 1 0 0 0 1 1 0 1 0 1 1 0 1

0 0 1 0 1 0 1 0 0 1 1 0 1 1

0 0 0 1 1 1 0 1 1 1 0 0 0 1

0 0 0 0 0 0 0 1 1 0 1 0 1 0

1 1 0 0 1 0 0 1 0 1 1 1 0 0

1 0 1 0 0 1 0 0 1 1 1 0 0 0

1 0 0 1 0 0 1 1 1 1 1 1 1 1

Length n=7, corrects e=1 error

1001011

received

decoded

1001001

1 error: correct decoding

5

A Repetition Code as a Packing

(3,1)-code: 111, 000

Pairwise distance = 3 1 error can be corrected

The M codewords of an(n,e)-code correspond toa packing of Hamming ballsof radius e in the n-cube

110 011101

111

000

010 001100

000

010 001100

110 011101

111

A packing of 2 radius-1 Hamming balls

in the 3-cube

6

A (5,1)-Packing Code as a 2-Player Game

(5,1)-code: 11111, 10100, 01010, 00001

0What is the 5th bit?1What is the 4th bit?0What is the 3rd bit?0What is the 2nd bit?0What is the 1st bit?

CarolePaul 11111

00001

1010001010

0 1 >1# errors

11111 0000110100 01010

01111 00100 00010 0001100100

01010

000100001000010

00001000010000111111 10100 01010 00001

7

Covering Codes

Covering is the companion problem to packing

Packing: (n,e)-code

Covering: (n,R)-code

lengthpacking radius

covering radius

110 011101

111

000

010 001100

000

010 001100

110 011101

111

(3,1)-packing code and(3,1)-covering code

“perfect code”11111

00001

1010001010

11111

11000

0111110111 00001

0010000010

(5,1)-packing code (5,1)-covering code

8

Optimal Length 5 Packing & Covering Codes

0100101100

01110 01101

00100

11100

01000

11110 11101 01111

00000

0101011000 10100 00110 00101

10110 10011

1000110010

11011

00011

10111

000010001010000

11111

10101 00111010111100111010

01110 01101

0100101100

00100

11100

01000

11110 11101 01111

00000

0101011000 10100 00110 00101

10110 10011

1000110010

11011

00011

10111

000010001010000

11111

10101 00111010111100111010

(5,1)-packing code

(5,1)-covering code

9

Sphere bound:

A (5,1)-Covering Code as a Football Pool

WLLLLBet 7

LWLLLBet 6

LLWLLBet 5

LLLWWBet 4

WWWLWBet 3

WWWWLBet 2

WWWWWBet 1

Round 5Round 4Round 3Round 2Round 1

Payoff: a bet with ≤ 1 bad predictionQuestion. Min # bets to guarantee a payoff? Ans.=7

00100

01111

11000

10111

00001

00010

11111

10

Codes with Feedback (Adaptive Codes)

FeedbackNoiseless, delay-less report of actual received bits

Improves the number of decodable messagesE.g., from 20 to 28 messages for an (8,1)-code

sender receiver

Noise

Noiseless FeedbackElwyn Berlekamp

1, 0, 1, 1, 0 1, 1, 1, 1, 0

1, 1, 1, 1, 0

11

A (5,1)-Adaptive Packing Code as a 2-Player Liar Game

A

D

BC

0 1 >1# liesYIs the message C?

NIs the message D?

NIs the message B?

NIs the message A or C?

YIs the message C or D?

CarolePaul

00101

Message

Originalencoding

Adaptedencoding

A B C D

01110 0101011000 10011

1**** 1****11*** 10*** 10*** 1000*101** 100**1000* 1000010001

Y $ 1, N $ 0

12

A (5,1)-Adaptive Covering Code as a Football Pool

LWLLWCarole

LBet 6

LBet 5

LBet 4

WBet 3 W

L

L

WWBet 2

L

W

W

W

W

W

L

L

WWBet 1

Round 5Round 4Round 3Round 2Round 1

Payoff: a bet with ≤ 1 bad predictionQuestion. Min # bets to guarantee a payoff?

Ans.=6

Bet 3

Bet 6

Bet 4Bet 5

0 1 >1# bad

predictions(# lies)

Bet 2Bet 1

13

Optimal (5,1)-Codes14

Code type Optimal size

(5,1)-code 4

(5,1)-adaptive code 4

Sphere bound 5 1/3

(5,1)-adaptive covering code 6

(5,1)-covering code 7

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

15

11

Linear Machine on Z

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

Linear Machine on Z16

5.5 5.5

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

Linear Machine on Z17

2.75 5.5 2.75

Time-evolution is proportional to rows of Pascal’s triangle

Liar Machine on Z18

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

Liar machine time-stepNumber chips left-to-right 1,2,3,…Move odd chips right, even chips left(Reassign numbers every time-step)

11 chips

t=0

Liar Machine on Z19


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=1

Liar Machine on Z20


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=2

Liar Machine on Z21


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=3

Liar Machine on Z22


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=4

Liar Machine on Z23


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=5

Liar Machine on Z24


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=6

Liar Machine on Z25


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

Height of linear machine at t=7

t=7

(6,1)-Liar Game26

Liar game time stepPaul bipartitions chips: green, purpleCarole moves one color to right

Paul’s goal: disqualify all but ≤1 chip after t=6 time steps

0 1 2

t=0

disqualified

Paul bipartitionsCarole moves purple9 chips

(6,1)-Liar Game27



0 1 2

t=1

disqualified

Paul bipartitionsCarole moves green

(6,1)-Liar Game28



0 1 2

t=2

disqualified


(6,1)-Liar Game29



0 1 2

t=3

disqualified

Paul bipartitionsCarole moves purple

t=4

(6,1)-Liar Game30



0 1 2

disqualified


t=5

(6,1)-Liar Game31



0 1 2

disqualified


t=6

(6,1)-Liar Game32



0 1 2

disqualified

Two chips survive: Paul loses

A Liar Game Strategy for Carole

Weight function for n rounds left; xi = #chips with i lies:

Lemma (Berlekamp)

Refined sphere boundLiar game. Carole keeps half of weight every step.Initial weight > 2n ) Final weight >1 ) Carole wins.

Pathological variant. Carole reduces half of weight every step.Initial weight < 2n ) Final weight <1 ) Carole wins.

33

(6,1)-Pathological Liar Game34

Paul’s goal: preserve ¸ 1 chip after t=6 time steps

0 1 2

t=0

disqualified

Paul bipartitionsCarole moves green9 chips

wt6-t(x)=wt6(x)=26-1

Carole moves green


0 1 2

t=1

disqualified

wt5(x)=25-3

Paul bipartitions


Paul bipartitions


0 1 2

t=2

disqualified

Carole moves green

wt4(x)=24-2




0 1 2

t=3

disqualified

wt3(x)=23-1


Paul bipartitions t=4


0 1 2

disqualified

wt2(x)=22-1

Carole moves purple


Carole moves greenPaul bipartitions t=5


0 1 2

disqualified

wt1(x)=21-1


t=6


0 1 2

disqualified

No chips survive: Paul loses

wt0(x)=20-1<1


Optimal (6,1)-Codes41

Code type Optimal #chips

(6,1)-code 8

(6,1)-adaptive code(Liar game)

8

Sphere bound 9 1/7

(6,1)-adaptive covering code(Pathological liar game)

10

(6,1)-covering code 12

New Approach to the Pathological Liar Game

Spencer and Winkler (`92) reduced the liar game to the liar machine, a discrete diffusion process on the integer line.

Ellis and Yan (`04) introduced the pathological liar game.

Cooper and Spencer (`06) use discrepancy analysis to compare the Propp-machine to simple random walk on Zd.

Here: (1) We reduce the pathological liar game to the liar machine, (2) Use discrepancy analysis to compare the liar machine to simple random walk on Z, and thereby (3) Improve the best known pathological liar game strategy when the number of lies is a constant fraction of the number of rounds.

42

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

Liar Machine vs. Pathological Liar Game43

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

9 chips

9 chips

t=0

disqualified

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=1

disqualified

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=2

disqualified

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=3

disqualified

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=4

disqualified

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=5

disqualified

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8


9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

t=6

disqualified

No chips survive: Paul loses

(6,1)-Pathological Liar Game, Liar Machine50

Code type Optimal #chips

Sphere bound 9 1/7

(6,1)-adaptive covering code(Pathological liar game)

10

(6,1)-liar machine 12

(6,1)-liar machine optimum: Minimum number of initial chips for ¸ 1 chip to be at position · -4 when t=6

9-9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8

(6,1)-Liar machine started with 12 chips after 6 rounds

disqualified

Reduction to Liar Machine51

Reduction to Liar Machine52

Liar Machine Versus Linear Machine53

Saving One Chip in the Liar Machine54

Pathological Liar Game Theorem55

Further Exploration

Tighten the discrepancy analysis for the special case of initial chip configuration f0(z)=M 0(z).

Generalize from binary questions to q-ary questions, q ¸ 2.

Improve analysis of the original liar game from Spencer and Winkler `92.

Prove general pointwise and interval discrepancy theorems for various discretizations of random walks.

56

Reading List

This paper: Linearly bounded liars, adaptive covering codes, and deterministic random walks, preprint (see homepage).

The liar machine– Joel Spencer and Peter Winkler. Three thresholds for a liar. Combin.

Probab. Comput.,1(1):81-93, 1992. The pathological liar game

– Robert Ellis, Vadim Ponomarenko, and Catherine Yan. The Renyi-Ulam pathological liar game with a fixed number of lies. J. Combin. Theory Ser. A, 112(2):328-336, 2005.

Discrepancy of deterministic random walks– Joshua Cooper and Joel Spencer, Simulating a Random Walk with

Constant Error, Combinatorics, Probability, and Computing, 15 (2006), no. 06, 815-822.

– Joshua Cooper, Benjamin Doerr, Joel Spencer, and Gabor Tardos. Deterministic random walks on the integers. European J. Combin., 28(8):2072-2090, 2007.

57

adaptive coding from a diffusion process on the integer line robert ellis october 26, 2009

Documents