Invertible Zero-Error Dispersers and Defective Memory with Stuck-At Errors

Ariel Gabizon Ronen Shaltiel

Defective memory with stuck-at errors

msg z

0 0 0 1 1 1

Enc

* 0 * * 1 *

0 1 0

Dec

0 1 0

Reading proc. Dec must retrieve z, doesn’t know which bits are stuck.

n-bit memory with ≤ p∙n “stuck bits” (can’t write)Goal: store msg z.

Give explicit schemes. p: store n-p∙n-logcn bits.Rate > 1-p-o(1).

Storage proc. Enc knows which bits are stuck.Earlier work [KuzTsy]

shows existence of non-explicit schemes.

Error-correcting codes where encoder knows in advance which bits will be corrupted.

In standard codes we expect* rate 1-h(2p)-o(1).

We can also handle few additional errs

0 0 0 1 1 1

Enc

Dec

0 1 0

0 0 1 1 1 1Noise

Give explicit schemes. Handle o(n½) unknown errs.Rate > 1-p-o(1).

Storage proc. Enc knows which bits are stuck.Reading proc. Dec must retrieve z, doesn’t know which bits are stuck.

n-bit memory with ≤ p∙n “stuck bits” (can’t write)Goal: store msg z.

msg z

* 0 * * 1 *

0 1 0

`Defect pattern’:Subset Sµ[n] of size ≤p∙n,

String a2{0,1}p∙n.

-Encoding proc. Enc, on message z 2{0,1}m and pattern S,a produces x2{0,1}n with x|S=a.

-Decoding proc. Dec such that Dec(x) = z

Dec doesn’t depend on S,a

m/n is the rate of the scheme.

Explicit: Enc expected poly-time*, Dec poly-time.

*Simply because that’s what we get.

Dfn of memory scheme (for parameter 0<p<1):

Write Once Memory (k-WOM)n-bit memory initialized to zeros.If we raise a bit, it is “stuck-at” 1.Goal: Reuse memory k times. (think k=2).

• Stuck-at scheme => 2-WOM, gives opt. rate ≈ h(p) + (1-p).

• Intuitively WOM is easier to achieve.– Defect pattern is not adversarial.– Can pass o(n) bits from enc to dec by reserving first

o(n) bits. (errorless channel from enc to dec).

• Starting point for this work: – Shpilka: Linear seeded extractors => 2-WOM.– We generalize: seedless zero-error disp. for bit-fixing

sources schemes for stuck-at errors.

0 0 0 0 0 01 1

Bit-fixing sources and dispersers:Sµ[n] – subset of size p∙n. a – string in {0,1}p∙n XS,a, set of strings in {0,1}n, s.t. x|S=a.Call such a set a bit-fixing source.|XS,a|=2k for k=n-p∙n (# unfixed bits).

Dfn: D:{0,1}n{0,1}m is a zero-error bit-fixing disperser for threshold k if S,a as above D(XS,a)={0,1}m. (Explicit: poly-time).

Goal: achieve large m. Obviously m ≤ k = n-p∙n.

Efficiently invertible: given S,a and output z, can find x2XS,a with D(x) = z in expected* poly(n)-time.*Simply because that’s what we get.

* 0 * * 1 *

* * *

D

Dispersers => memory schemes (and vice-versa)

Bit-fixing disperser D:{0,1}n{0,1}m.

• Encoding: Given message z2{0,1}m and defect pattern S,a, Enc finds x2XS,a with D(x) = z.

• Decoding: Given x, Dec computes D(x)=z.

• Output len. m=(1-o(1))∙k => optimal rate.

• Explicitness of scheme follows if D is explicit and efficiently invertibile.

This work: construct such dispersers (for k>logcn). Improve [GabSha08] where m=(k).

Construction of zero-error disperser

Seeded zero-error dispersers

Dfn: E(x,y) is a seeded bit-fixing disperser for threshold k, if z, and bit-fixing source XS,a with k unfixed bits, there exist:- a seed y, and - x2XS,a

such that E(x,y) = z.E is efficiently invertible if x,y can be found in expected poly(n)-time given z,S,a.

Known [RazReiVad]: m=(1-o(1))∙k, |y|=log3n.

Shpilka: 2-WOM, Enc and Dec can share o(n) bits. Given z,S,a, Enc finds y and shares it.

Linear seeded extractors => seeded bit-fixing disperser

Known: Explicit constructions of E(x,y) s.t.• bit-fixing source* XS,a with k unfixed bits,

for most y’s E(XS,a,y) is close to uniform.• y: T(x)=E(x,y) is linear over F2

n.

* In fact, distribution with min-entropy ≥ k.

XS,a is an affine subspace of F2n.

Þ E is a seeded bit-fixing source disperser that is efficiently invertible.

[Tre99,ShaUma01]: seed O(log n), output √k[RazReiVad00]: seed O(log3n), output (1-o(1))k

(Seedless) disperser construction:Step 1: (seedless) disperser with short

output m=O(log n) that is efficiently invertible.

Step 2: (Using approach of [GabSha08])Combine with seeded disperser with large output length to get (seedless) and efficiently invertible disperser with large output length.

Thm: Any explicit bit-fixing extractor* D:{0,1}n{0,1}m with m=O(log n) is efficiently invertible.

*Say, PrXXS,a [D(X)=z] ¸ 2-2m = 1/poly(n)

Known Explicit constructions: [KempZuck03,GabRazSha04,Rao09].

Proof: Sample random x2XS,a and check if D(x) =z. Will take expected poly(n)-time.

Step 1: Disperser with m=O(log n)

-D(x) is a b.f.d. with output length O(log n)-E(x,y) is a seeded b.f.d. with seed O(log n)

F(x) , E(x,D(x)) has large m. Disperser? [GabSha08]: Yes, if E is a `subsource hitter’:

S,a z y bit-fixing source XS’,a’ s.t.• {x XS,a: E(x,y)=z} contains XS’,a’

• XS’,a’ has k’ unfixed bits (for k’<<k not tiny).

Proof: x s.t. D(x)=y and then,F(x) = E(x,D(x)) = E(x,y) = z.

Efficiently invertible if y,S’,a’ can be found in expected poly(n) time given S,a,z.

Step 2: Increasing output lengthwith threshold k’ rather than k

=> F is efficiently invertible

-D(x) is a b.f.d. with output length O(log n)-E(x,y) is a seeded b.f.d. with seed O(log n)

F(x) , E(x,D(x)) has large m. Disperser? [GabSha08]: Yes, if E is a `subsource hitter’:

S,a z y bit-fixing source XS’,a’ s.t.• {x XS,a: E(x,y)=z} contains XS’,a’

• XS’,a’ has k’ unfixed bits (for k’<<k not tiny).

E linear seeded => z y : {x XS,a: E(x,y)=z} is an affine subspace of dim k’ := k-m.

Works if D is an extractor for affine sources [Bou07,Yeh10,Li11] for k=n/(log log n)½.

Step 2: Increasing output lengthwith threshold k’ rather than k

affine subspace

Dispersers for affine sourcesGet: zero-error disperser for affine sources.• Used in memory scheme w/ additional

errs.• Have explicit D only for k=n/(log log

n)½.Þ Final disperser has k > n/(log log n)½.Þ Output length m = k - O(n/(log log

n)½).• Suffices for achieving rate 1-p-o(1).

In paper: Subsource-hitter for bit-fixing.=> Final disperser for bit-fixing sources.Works for any k > logcn.Output length m = k - O(logcn).Gives schemes with m = n-p∙n - O(logcn).

Linear seeded extractors

Subsource hitter for bit-fixing

sources.

Subsource hitter (Inspired by [GabRazSha])

* 0 * * 1 *XS,a

* * *

seed y

* 0 * * 1 *XS,a

* * *

seed yseed y’

y’ - seed for sampler, selects subset of [n]. y’ that partitions unfixed bits:• Most unfixed bits are selected.• At least k’ bits are not selected.Apply E(∙,y) only on selected portion of x.

Handling stuck-at memory with noise

msg z

0 0 0 1 1 1

Enc

* 0 * * 1 *

0 1 0

Dec

0 1 0

0 0 1 1 1 1Noise

Idea: Force Enc to output codeword in code for noise.

Need code: defect pattern many consistent codewords.

Linear code w/ dual dist p∙n.

Code ∩ XS,a is affine space.dim n-p∙n-o(n) (for n½ errs)

Use affine source disp.

k=n-p∙n-o(n) => m=n-p∙n-o(n).=> rate 1-p-o(1).

Given msg z and defect pattern S,a: Find x Code ∩ XS,a s.t. D(x)=z.

Given corrupted x’ that is close to x:Decode x’ to find x.Compute D(x) to get msg z.

Open questions

• Construct invertible dispersers for `Hamming sources’: Hamming balls of radius p∙n around some point x {0,1}n.Want: output len m approaching h(p)∙n.Motivated by k-WOMs.Issues with definition of explicitness.

• Our constructions should give 2-WOMs that are `more robust’ than known.• Don’t rely on channel from enc to

dec.• Can handle additional errs.

Thanks

That’s it…

Standrad coding theory (unknown errors)

msg z

0 1 0 1 0 1

Enc

0 1 0

Dec

0 1 0

0 0 0 1 1 1

Channel

If p∙n bits are corrupted, rate can be* ≈ 1-h(p)

• Achieved by random codes.

• Not known to be tight.• Explicit constructions

achieve poorer rate.

C(z)

Suppose Enc\Dec know where errors are:

msg z

0 0 1 0 1

Enc

* 0 * * 1

0 1 0

Dec

0 1 0

Can trivially get get rate 1-p, by using `unstuck’ places

Channel replaces ≤p∙n bits by fixed bits.

What if only Enc knows errors?

msg z

0 0 0 1 1 1

Enc

* 0 * * 1 *

0 1 0

Dec

0 1 0

Dec must retrieve z without knowing the error pattern

Channel replaces ≤p∙n bits by fixed bits.

[Kuznetsov-Tsybakov:] possible to get any rate R < 1-p.

We give first explicit schemes.rate R > 1-p-o(1).

Dfn: A subsource of XS,a is a set

XS’,a’ ½ XS,a

for- S’¾S with |S’|·2p¢n.

* 0 * * 1 *XS,a:

0 0 * * 1 *XS’,a’:

Dfn:E(x,y) is a subsource hitter if given z, there exists - a seed y- a subsource XS’,a’ of XS,a such that: E(XS’,a’,y) ´ z

E is eff. invertible if y, XS’,a’ can be found in poly(n)-time.

Theorem of [GabSha08]

-D is a b.f.d (for sets |S’|·2p¢n).-E is a subsource hitter. F(x) , E(x,D(x)) is a bit-fixing disperser.

Proof: -Given z, 9XS’,a’ with E(XS’,a’,y)´z. -Also, 9x2XS’,a’ with D(x)=y.

For this x: E(x,D(x))=E(x,y)=z.

Efficient Invertibilty of FProof: -Given z, 9XS’,a’ with E(XS’,a’,y)´z. -Also, 9x2XS’,a’ with D(x)=y.

For this x: E(x,D(x))=E(x,y)=z.

E e.i. can efficiently find XS’,a’,y.D e.i. can eff. find x2XS’,a’ with D(x)=y.

Seeded dispersers subsource hitters:

First, sample a random subset of the input bits.Apply seeded disperser only on this subset.

* 0 * * 1 *

Preimage set now includes all settings of non-fixed bits outside of the subset.

apply E here

Similar results for `mixed’ scenario

‘stuck errors’ known to encoder+noise added after encoding

uses affine dispersers and error correcting codes rather than bit-fixing dispersers.

We get similar improvements for output length of affine dispersers.

Open questions

-Invertible dispersers for `Hamming sources’:Find mapping D:{0,1}n{0,1}m such that for all large enough hamming balls BD(B) = {0,1}m ,and there is efficient way of finding preimage of z in B.

Thanks

Dispersers imply memory schemes

Bit-fixing disperser D:{0,1}n{0,1}m.

- Encoding: Given message z2{0,1}m and defect pattern S,a, Enc finds x2XS,a

with D(x) = z.- Decoding: Given x, Dec computes D(x)=z.

Output length m = (1-o(1))∙k => optimal rate.Explicitness of scheme follows if D is explicit and efficiently invertibile.

Fixed constant 0<p<1.

Encoding function C:{0,1}m{0,1}n ,n>m.

-Message z2 {0,1}m

-Codeword C(z) 2 {0,1}n

Decoder must retrieve z from C(z) after p∙n bits of C(z) have been corrupted. (will look at different dfns next)

m/n , the rate of C.

Coding Theory

What if only Enc knows errors?

msg z

0 0 0 1 1 1

Enc

* 0 * * 1 *

0 1 0

Dec

0 1 0

[Kuznetsov-Tsybakov:] possible to get any rate R < 1-p. We give first explicit schemes.

`defect pattern’- p∙n `stuck’ bits

Memory with n cells. 0<p<1, some constant.

p∙n cells are `stuck’ either at 0 or 1.

Person writing in memory knows which cells are stuck and to what values.

Person reading memory does not.

Memory with stuck-at defects:

`Defect pattern’:Subset Sµ[n] of size p∙n,

String a2{0,1}p∙n.

Message z 2{0,1}m.

-Encoding function E, given S,a,z, produces x2{0,1}n with x|S=a.

-Decoding function D such that D(x) = z

D doesn’t depend on S,a

m/n is the rate of the scheme

Formal dfn of memory scheme:

[Kuznetsov-Tsybakov]: Introduced model. Non-explicit schemes with any rate R<1-p

[Tsybakov]: Explicit schemes using linear codes with rate <<1-p

Our result: Explicit* schemes with rate 1-p-o(1).

*-Our encoding is runs in expected polynomial time.

Previous results and Ours

Bit-fixing dispersers:

* 0 * * 1 *

* * *

Given all strings with a certain `defect pattern’ a bit-fixing disperser produces all possible outputs

D

Bit-fixing dispersers:

* 0 * * 1 *

* * *

Given all strings with a certain `defect pattern’ a bit-fixing disperser produces all possible outputs

D

Example:D(x1,…,xn) , i xi (mod 2)

Bit-fixing dispersers:

Sµ[n] – subset of size p∙n. a – string in {0,1}p∙n XS,a, set of strings in {0,1}n, with x|

S=a.Call such a set a bit-fixing source.

Dfn: D:{0,1}n{0,1}m is a bit-fixing disperser if for every S,a as above D(XS,a)={0,1}m

(this is actually a zero-error disperser)

Bit-fixing dispersers:

* 0 * * 1 *

* * *

Another example: Suppose k=n-p∙n. Take the sum of input bits (mod k)..gives us logk output bits.

D

Dispersers to memory schemes:

Bit-fixing disperser D(x).Message z

-Given defect pattern S,a, Enc finds x2XS,a with D(x) = z.

-Dec simply computes D.

Efficiency of scheme requires `efficient invertibility’ of D.

Dfn: We say D is efficiently invertible if given S,a and output z,we can find in expected* poly(n)-time x2XS,a with D(x) = z.

*Simply because that’s what we get

Our results:Bit-fixing dispersers with output length m=k-o(k) , where k = n-p¢n.

Previous result m=(k).[GabSha08]

Also, our dispersers are efficiently invertible – (previous constructions seem not to be)

Disperser Construction:

Step 1: Disperser with short output.

Step 2: Seeded disperser with large output length.

Step 3: Combining the above to get (seedless) disperser with large output length. (Follows [GabSha08])

Thm: Any explicit bit-fixing extractor* D:{0,1}n{0,1}m with m=O(log n) is efficiently invertible.

*Say, PrxXS,a [D(x)=z]¸2-2m=1/poly(n)

Known Explicit constructions: [KempZuck03,GabRazSha04,Rao09].

Proof: Sample random x2XS,a and check if D(x) =z. Will take expected poly(n)-time.

Step 1: Disperser with m=O(log n)

2nd step: we construct efficiently invertible seeded bit-fixing dispersers with large output length.

Seeded Bit-fixing dispersers:

* 0 * * 1 *

* * *

We `invest’ a short seed to obtain a longer output.

E *auxiliary short random input a.k.a `seed’

Dfn:E(x,y) is a seeded bit-fixing disperser if given z, there exist:- a seed y , and - x2XS,a such that: E(x,y) = z.

E is efficiently invertible if x,y can be found in poly(n)-time.

..idea is that |y|<<|z|.

Observation:If T(x)=E(x,y) is linear function for every fixed seed y,then E is efficiently invertible:

For every y: (assume |y|=O(log n) )-Check if exists x with- x2XS,a

- E(x,y) =z

(This is a set of affine equations and can be check efficiently)

`Linear seeded extractors’ exactlygive us such an E with large output length

Such constructions by [Trevisan\RazReingoldVadhan\ShaltielUmans,..]

Step 3: Get rid of seed:-D(x) is a b.f.d. with output length d-E(x,y) is a seeded b.f.d. with seed length d

[GabSha08]: F(x) = E(x,D(x)) is a bit-fixing disperser assuming E isalso a `subsource hitter’.

Also: If D and E are efficiently invertible so is F.

Subsource hitters[GabSha08]

`a seeded bit-fixing disperser where the set of preimages of z contains a bit-fixing source’

Dfn: A subsource of XS,a is a set

XS’,a’ ½ XS,a

for- S’¾S with |S’|·2p¢n.

* 0 * * 1 *XS,a:

0 0 * * 1 *XS’,a’:

Dfn:E(x,y) is a subsource hitter if given z, there exists - a seed y- a subsource XS’,a’ of XS,a such that: E(XS’,a’,y) ´ z

E is eff. invertible if y, XS’,a’ can be found in poly(n)-time.

Theorem of [GabSha08]

-D is a b.f.d (for sets |S’|·2p¢n).-E is a subsource hitter. F(x) , E(x,D(x)) is a bit-fixing disperser.

Proof: -Given z, 9XS’,a’ with E(XS’,a’,y)´z. -Also, 9x2XS’,a’ with D(x)=y.

For this x: E(x,D(x))=E(x,y)=z.

Efficient Invertibilty of FProof: -Given z, 9XS’,a’ with E(XS’,a’,y)´z. -Also, 9x2XS’,a’ with D(x)=y.

For this x: E(x,D(x))=E(x,y)=z.

E e.i. can efficiently find XS’,a’,y.D e.i. can eff. find x2XS’,a’ with D(x)=y.

Seeded dispersers subsource hitters:

First, sample a random subset of the input bits.Apply seeded disperser only on this subset.

* 0 * * 1 *

Preimage set now includes all settings of non-fixed bits outside of the subset.

apply E here

Similar results for `mixed’ scenario

‘stuck errors’ known to encoder+noise added after encoding

uses affine dispersers and error correcting codes rather than bit-fixing dispersers.

We get similar improvements for output length of affine dispersers.

Open questions

-Invertible dispersers for `Hamming sources’:Find mapping D:{0,1}n{0,1}m such that for all large enough hamming balls BD(B) = {0,1}m ,and there is efficient way of finding preimage of z in B.

Thanks

Pseudorandomness Course

Dispersers are related to Extractors that have applications in TCS, e.g.:-Running randomized algorithms with weak randomness-Generating pseudorandom bits for small space algorithms.

Increasing the output length of dispersers[GabSha08]

D – a bit-fixing disperser with output length d

E – a subsource hitter with seed length d, output length m

F(x) = E(x,D(x)) is a bit-fixing disperser with output length m.

Efficient invertibility

F(x) = E(x,D(x))D has output length O(log n)

Finding preimage of z:-choose random y – find subspace X’ of XS,a with E(X’,y) ´z (system of affine equations)

Find x2X’ with D(x) = y.We get F(x) = E(x,D(x)) = E(x,y) = z.(Last equality because x2X’)

Efficient invertibility

When D has output length O(logn)random sampling will succeed w.h.pin poly(n)-time.

Dispersers to memory schemes:

Message z

0 0 0 1 1 1

Encoder finds x2XS,a

with D(x) = z

* 0 * * 1 *

0 1 0

Decoder computes

D(x) to retrieve z

0 1 0

must always decode correctly without knowing the defect pattern

`defect pattern’- p∙n `stuck’ bits

Standard coding:Message z

0 1 0 1 0 1

Encoder

0 1 0

Decoder

0 1 0

0 0 0 1 1 1

Noise

Codeword C(z)