debugging floating-point math in racket · finite approximation • actual flonum fields are...

Debugging Floating-PointDebugging Floating-PointDebugging Floating-Point

Math in RacketMath in RacketMath in Racket

Neil Toronto

RacketCon 2013

Racket Floating-Point SupportRacket Floating-Point SupportRacket Floating-Point Support

• Fast (JIT-ed) and compliant (IEEE 754 and C99)

1111111111111111



• More flonum (i.e. 64-bit float) functions:

math/special-functions: gamma, beta, psi,zeta, erf, etc.

math/distributions: Gamma, Normal, etc.

1111111111111111






• Other floating-point modules:

racket/extflonum: basic 80-bit operations

math/bigfloat: arbitrary-precision floats

1111111111111111






• Other floating-point modules:

racket/extflonum: basic 80-bit operations

math/bigfloat: arbitrary-precision floats

• math/flonum: a bunch of weird things like fl,flnext, +max.0, flonum->ordinal, fllog1p,flsqrt1pm1, flcospix

1111111111111111

You Could Have Invented Floating-PointYou Could Have Invented Floating-PointYou Could Have Invented Floating-Point

Need to represent or ...

2222222222222222



(struct: float ([sign : (U -1 1)][sig : Natural][exp : Integer]))

2222222222222222




(: float->real (float -> Real))(define (float->real x) (match-define (float s n m) x) (* s n (expt 2 m)))

2222222222222222





> (float->real (float -1 10 0))-10

2222222222222222







2222222222222222

You Could Have Invented Floating-Point MultiplicationYou Could Have Invented Floating-Point MultiplicationYou Could Have Invented Floating-Point Multiplication


3333333333333333



(: float* (float float -> float))(define (float* x1 x2) (match-define (float s1 n1 m1) x1) (match-define (float s2 n2 m2) x2) (float (* s1 s2) (* n1 n2) (+ m1 m2)))

3333333333333333



(: float* (float float -> float))(define (float* x1 x2) (match-define (float s1 n1 m1) x1) (match-define (float s2 n2 m2) x2) (float (* s1 s2) (* n1 n2) (+ m1 m2)))

> (float->real (float* (float -1 10 0)(float -1 10 3)))

800

3333333333333333

Finite ApproximationFinite ApproximationFinite Approximation

• Actual flonum fields are fixed-size, requiring

Rounding least significant bit after operations

Representations for overflow (i.e. +inf.0 and-inf.0) and underflow (i.e. +0.0 and -0.0)

4444444444444444





• Consequence: a natural well-order over flonums

> (flnext 0.0) ; from math/flonum4.9406564584125e-324

4444444444444444







> (list (flonum->ordinal 0.0)(flonum->ordinal +max.0))

'(0 9218868437227405311)

4444444444444444








'(0 9218868437227405311)

> (flonums-between 0.0 1.0)4607182418800017408 ; 4.6 *quintillion*

4444444444444444








'(0 9218868437227405311)

> (flonums-between 0.0 1.0)4607182418800017408 ; 4.6 *quintillion*

• Consequence: most flonum functions aren’t exact 4444444444444444

Correctness Means Minimizing ErrorCorrectness Means Minimizing ErrorCorrectness Means Minimizing Error

• A flonum’s unit in last place (ulp) is the distancebetween it and the next flonum

> (flulp #i355/113) ; from math/flonum4.440892098500626e-16

5555555555555555




• Error is distance from the true value, in ulps

> #i355/113 pi3.14159292035398253.141592653589793

5555555555555555





> #i355/113 pi3.14159292035398253.141592653589793

> (flulp-error #i355/113 pi)600699552.0 ; 600.7 million ulps

5555555555555555





> #i355/113 pi3.14159292035398253.141592653589793


• A flonum function is correctly rounded* when itsoutputs’ maximum error is no more than 0.5 ulps

5555555555555555





> #i355/113 pi3.14159292035398253.141592653589793


• A flonum function is correctly rounded* when itsoutputs’ maximum error is no more than 0.5 ulps

* assuming inputs are exact; i.e. no guarantees aregiven for arguments with error 5555555555555555

Correctness Example: SubtractionCorrectness Example: SubtractionCorrectness Example: Subtraction

> (plot3d (contour-intervals3d(λ (x y) (let ([x (fl x)] [y (fl y)])

(define z* (- (inexact->exact x)(inexact->exact y)))

(flulp-error (fl- x y) z*)))-1 1 -1 1))

6666666666666666

Correctness Example: SubtractionCorrectness Example: SubtractionCorrectness Example: Subtraction

> (plot3d (contour-intervals3d(λ (x y) (let ([x (fl x)] [y (fl y)])

(define z* (- (inexact->exact x)(inexact->exact y)))

(flulp-error (fl- x y) z*)))-1 1 -1 1))

000000000

.1.1.1.1.1.1.1.1.1

.2.2.2.2.2.2.2.2.2

.3.3.3.3.3.3.3.3.3

.4.4.4.4.4.4.4.4.4

.5.5.5.5.5.5.5.5.5

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1

x axisx axisx axisx axisx axisx axisx axisx axisx axis

y axis

y axis

y axisy axis

y axisy axis

y axis

y axis

y axis

6666666666666666

Correctness Example: LogarithmCorrectness Example: LogarithmCorrectness Example: Logarithm

> (require math/bigfloat) ; default sig. size: 128 bits> (plot (function

(λ (x) (let ([x (fl x)]) (define z* (bigfloat->real (bflog (bf x)))) (flulp-error (fllog x) z*)))

0 10))

7777777777777777

Correctness Example: LogarithmCorrectness Example: LogarithmCorrectness Example: Logarithm

> (require math/bigfloat) ; default sig. size: 128 bits> (plot (function

(λ (x) (let ([x (fl x)]) (define z* (bigfloat->real (bflog (bf x)))) (flulp-error (fllog x) z*)))

0 10))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

000000000 222222222 444444444 666666666 888888888 101010101010101010000000000

.1.1.1.1.1.1.1.1.1

.2.2.2.2.2.2.2.2.2

.3.3.3.3.3.3.3.3.3

.4.4.4.4.4.4.4.4.4

7777777777777777

Correctness is NoncompositionalCorrectness is NoncompositionalCorrectness is Noncompositional

> (plot (function(λ (x) (define z* (bigfloat->real (bflog (bf x)))) (flulp-error (fllog (fl x)) z*)))

0 10)

8888888888888888

Correctness is NoncompositionalCorrectness is NoncompositionalCorrectness is Noncompositional


0 10)


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

000000000 222222222 444444444 666666666 888888888 101010101010101010000000000

505050505050505050

100100100100100100100100100

150150150150150150150150150

200200200200200200200200200

250250250250250250250250250

8888888888888888

Debugging: Geometric Inverse CDFDebugging: Geometric Inverse CDFDebugging: Geometric Inverse CDF

Implement for

9999999999999999


Implement for

• First stab:

(define (geom p u)(fl/ (fllog u) (fllog (fl- 1.0 p))))

9999999999999999


Implement for

• First stab:

(define (geom p u)(fl/ (fllog u) (fllog (fl- 1.0 p))))

• Reference implementation:

(define (geom* p u) (let ([p (bf p)] [u (bf u)]) (bigfloat->real

(bf/ (bflog u) (bflog (bf- 1.bf p))))))

9999999999999999


• Error plot for geom for p <= 1:

000000000

222222222

444444444

666666666

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

ppppppppp

uuuuuuuuu

ulps errorulps errorulps errorulps errorulps errorulps errorulps errorulps errorulps error

10101010101010101010101010101010


• Error plot for geom for p <= 0.1:

000000000

505050505050505050

100100100100100100100100100

150150150150150150150150150

.1.1.1.1.1.1.1.1.1

.08.08.08.08.08.08.08.08.08

.06.06.06.06.06.06.06.06.06

.04.04.04.04.04.04.04.04.04

.02.02.02.02.02.02.02.02.02

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

ppppppppp

uuuuuuuuu


10101010101010101010101010101010


• Error plot for geom for p <= 1e-05:

000000000

250000250000250000250000250000250000250000250000250000

5×10⁵5×10⁵5×10⁵5×10⁵5×10⁵5×10⁵5×10⁵5×10⁵5×10⁵

750000750000750000750000750000750000750000750000750000

1×10⁶1×10⁶1×10⁶1×10⁶1×10⁶1×10⁶1×10⁶1×10⁶1×10⁶

.00001.00001.00001.00001.00001.00001.00001.00001.00001

8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶

6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶

4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶

2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000ppppppppp

uuuuuuuuu


10101010101010101010101010101010



000000000

2×10⁸2×10⁸2×10⁸2×10⁸2×10⁸2×10⁸2×10⁸2×10⁸2×10⁸

4×10⁸4×10⁸4×10⁸4×10⁸4×10⁸4×10⁸4×10⁸4×10⁸4×10⁸

6×10⁸6×10⁸6×10⁸6×10⁸6×10⁸6×10⁸6×10⁸6×10⁸6×10⁸

1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰

8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹

6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹

4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹

2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000ppppppppp

uuuuuuuuu


10101010101010101010101010101010

Argh!Argh!Argh!

Q. Is this normal???

A. Yes. Most straightforward flonum functionimplementations have low error most places andhigh error (often unbounded) in others.

11111111111111111111111111111111

Argh!Argh!Argh!



The good news:You can usually fix them using just flonum ops.

11111111111111111111111111111111

Argh!Argh!Argh!



The good news:You can usually fix them using just flonum ops.

Q. How do I fix them???

A. Most functions—not implementations, butfunctions themselves—have ill-conditioned placeswhere they turn low input error into high outputerror. Avoid these badlands.

11111111111111111111111111111111

The Floating-Point Priest Says...The Floating-Point Priest Says...The Floating-Point Priest Says...

“The condition number of afunction is the absolutevalue of the ratio of itsderivative and its value,multiplied by the blah blahblah blah blah blah blahblah blah blah blah blahblah blah blah blah blahblah blah blah blah blah.”

12121212121212121212121212121212

The Badlands: SubtractionThe Badlands: SubtractionThe Badlands: Subtraction

> (plot3d (contour-intervals3d(λ (x y) (define z* (- x y)) (flulp-error (fl- (fl x) (fl y)) z*))-1 1 -1 1))

13131313131313131313131313131313

The Badlands: SubtractionThe Badlands: SubtractionThe Badlands: Subtraction

> (plot3d (contour-intervals3d(λ (x y) (define z* (- x y)) (flulp-error (fl- (fl x) (fl y)) z*))-1 1 -1 1))

000000000

222222222

444444444

666666666

888888888

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1


y axis

y axis

y axisy axis

y axisy axis

y axis

y axis

y axis

13131313131313131313131313131313

The Badlands: LogarithmThe Badlands: LogarithmThe Badlands: Logarithm


0 10)

14141414141414141414141414141414

The Badlands: LogarithmThe Badlands: LogarithmThe Badlands: Logarithm


0 10)


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

000000000 222222222 444444444 666666666 888888888 101010101010101010000000000

505050505050505050

100100100100100100100100100

150150150150150150150150150

200200200200200200200200200

250250250250250250250250250

14141414141414141414141414141414

The Badlands: DivisionThe Badlands: DivisionThe Badlands: Division

000000000

.25.25.25.25.25.25.25.25.25

.5.5.5.5.5.5.5.5.5

.75.75.75.75.75.75.75.75.75

111111111

1.251.251.251.251.251.251.251.251.25

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1


y axis

y axis

y axisy axis

y axisy axis

y axis

y axis

y axis

15151515151515151515151515151515


000000000

.25.25.25.25.25.25.25.25.25

.5.5.5.5.5.5.5.5.5

.75.75.75.75.75.75.75.75.75

111111111

1.251.251.251.251.251.251.251.251.25

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1


y axis

y axis

y axisy axis

y axisy axis

y axis

y axis

y axis

• No badlands: except for flonum rounding error,division error doesn’t depend on inputs

15151515151515151515151515151515


000000000

.25.25.25.25.25.25.25.25.25

.5.5.5.5.5.5.5.5.5

.75.75.75.75.75.75.75.75.75

111111111

1.251.251.251.251.251.251.251.251.25

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1

111111111

.5.5.5.5.5.5.5.5.5

000000000

-.5-.5-.5-.5-.5-.5-.5-.5-.5

-1-1-1-1-1-1-1-1-1


y axis

y axis

y axisy axis

y axisy axis

y axis

y axis

y axis

• No badlands: except for flonum rounding error,division error doesn’t depend on inputs

• Multiplication error is the same way

15151515151515151515151515151515

Informal Error AnalysisInformal Error AnalysisInformal Error Analysis

• Recursively reason about the body of geom:

(define (geom p u) (fl/ (fllog u) (fllog (fl- 1.0 p))))

16161616161616161616161616161616




• Can’t do anything about fl/ except make sure itsarguments are as accurate as possible

16161616161616161616161616161616





• If u is exact, (fllog u) has <= 0.5 ulps error

16161616161616161616161616161616






• If p is exact, (fl- 1.0 p) has <= 0.5 ulps error

16161616161616161616161616161616







• If p is exact and near 0.0...

16161616161616161616161616161616








Then (fl- 1.0 p) is inexact and near 1.0...

16161616161616161616161616161616









So (fllog (fl- 1.0 p)) may have high error

16161616161616161616161616161616









So (fllog (fl- 1.0 p)) may have high error

• Let’s check math/flonum for another incantation...16161616161616161616161616161616

log(1+x)log(1+x)log(1+x)

• Looks interesting: fllog1p

xxxxxxxxx

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

000000000 2.52.52.52.52.52.52.52.52.5 555555555 7.57.57.57.57.57.57.57.57.5

-2-2-2-2-2-2-2-2-2

000000000

222222222

17171717171717171717171717171717

log(1+x)log(1+x)log(1+x)

• Looks interesting: fllog1p

xxxxxxxxx

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

log

(1+

x)

000000000 2.52.52.52.52.52.52.52.52.5 555555555 7.57.57.57.57.57.57.57.57.5

-2-2-2-2-2-2-2-2-2

000000000

222222222

• We can use it almost directly—mathematically,

17171717171717171717171717171717

Debugging: Geometric Inverse CDF (Second Stab)Debugging: Geometric Inverse CDF (Second Stab)Debugging: Geometric Inverse CDF (Second Stab)

(define (geom p u) (fl/ (fllog u) (fllog1p (- p))))

18181818181818181818181818181818



• Error plot for geom for p <= 1:

000000000

.5.5.5.5.5.5.5.5.5

111111111

1.51.51.51.51.51.51.51.51.5

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

pppppppppuuuuuuuuu


18181818181818181818181818181818



• Error plot for geom for p <= 0.1:

000000000

.5.5.5.5.5.5.5.5.5

111111111

1.51.51.51.51.51.51.51.51.5

222222222

.1.1.1.1.1.1.1.1.1

.08.08.08.08.08.08.08.08.08

.06.06.06.06.06.06.06.06.06

.04.04.04.04.04.04.04.04.04

.02.02.02.02.02.02.02.02.02

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

pppppppppuuuuuuuuu


18181818181818181818181818181818




000000000

.5.5.5.5.5.5.5.5.5

111111111

1.51.51.51.51.51.51.51.51.5

222222222

.00001.00001.00001.00001.00001.00001.00001.00001.00001

8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶8×10⁻⁶

6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶6×10⁻⁶

4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶4×10⁻⁶

2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶2×10⁻⁶

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000pppppppppuuuuuuuuu


18181818181818181818181818181818




000000000

.5.5.5.5.5.5.5.5.5

111111111

1.51.51.51.51.51.51.51.51.5

1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰1×10⁻¹⁰

8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹8×10⁻¹¹

6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹6×10⁻¹¹

4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹4×10⁻¹¹

2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹2×10⁻¹¹

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000pppppppppuuuuuuuuu


18181818181818181818181818181818

Argh It Is Not Perfect!Argh It Is Not Perfect!Argh It Is Not Perfect!

• But < 3 ulps error is very accurate

19191919191919191919191919191919



• Does geom have badlands?

19191919191919191919191919191919




000000000

222222222

444444444

666666666

888888888

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

ppppppppp

uuuuuuuuu


19191919191919191919191919191919




000000000

222222222

444444444

666666666

888888888

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

111111111

.8.8.8.8.8.8.8.8.8

.6.6.6.6.6.6.6.6.6

.4.4.4.4.4.4.4.4.4

.2.2.2.2.2.2.2.2.2

000000000

ppppppppp

uuuuuuuuu


• This is a property of the function, so we can’t doanything about it 19191919191919191919191919191919

Debugging SummaryDebugging SummaryDebugging Summary

• Make direct and reference implementations

20202020202020202020202020202020



• Repeat:

Plot error, identify high-error regions

Find badlands inputs, replace computations

20202020202020202020202020202020



• Repeat:



• Avoid:

Subtracting nearby values

Taking logs of values near 1.0

Other badlands (most zero crossings,exponential growth)

20202020202020202020202020202020



• Repeat:



• Avoid:

Subtracting nearby values

Taking logs of values near 1.0

Other badlands (most zero crossings,exponential growth)

• Move multiplication and division outward20202020202020202020202020202020

What If I Need Moar Bits?What If I Need Moar Bits?What If I Need Moar Bits?

• racket/extflonum: 80-bit extended flonums

Requires (extflonum-available?) = #t

64-bit significand, 15-bit exponent

(extfl->exact (extflexp (real->extfl 1/7)))

21212121212121212121212121212121


• racket/extflonum: 80-bit extended flonums

Requires (extflonum-available?) = #t

64-bit significand, 15-bit exponent

(extfl->exact (extflexp (real->extfl 1/7)))

• double-double flonums: sum of twononoverlapping flonums represent a number

Requires correctly rounded arithmetic

~105-bit significand, 11-bit exponent

(let*-values ([(x2 x1) (fl2 1/7)][(y2 y1) (fl2exp x2 x1)])

(fl2->real y2 y1))

21212121212121212121212121212121


• math/bigfloat: arbitrary precision floats

Requires MPFR (Racket ships with libraries)

Any size significand, 32-bit exponent

22222222222222222222222222222222





• Exact rationals

22222222222222222222222222222222





• Exact rationals

• AN IMPORTANT DATA POINT:

22222222222222222222222222222222





• Exact rationals


It takes 1074-bit bigfloats to fix geomIt takes 1074-bit bigfloats to fix geomIt takes 1074-bit bigfloats to fix geom

using (bflog (bf- 1.bf p))using (bflog (bf- 1.bf p))using (bflog (bf- 1.bf p))

22222222222222222222222222222222





• Exact rationals


It takes 1074-bit bigfloats to fix geomIt takes 1074-bit bigfloats to fix geomIt takes 1074-bit bigfloats to fix geom

using (bflog (bf- 1.bf p))using (bflog (bf- 1.bf p))using (bflog (bf- 1.bf p))• Conclusion: “moar bits” is not a general solution

22222222222222222222222222222222

The Badlands: Square OotThe Badlands: Square OotThe Badlands: Square Oot

> (plot (function(λ (x) (define z* (bigfloat->real (bfsqrt (bf x)))) (flulp-error (flsqrt (fl x)) z*))0 1e-320))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

000000000 2×10⁻³²¹2×10⁻³²¹2×10⁻³²¹2×10⁻³²¹2×10⁻³²¹2×10⁻³²¹2×10⁻³²¹2×10⁻³²¹2×10⁻³²¹ 4×10⁻³²¹4×10⁻³²¹4×10⁻³²¹4×10⁻³²¹4×10⁻³²¹4×10⁻³²¹4×10⁻³²¹4×10⁻³²¹4×10⁻³²¹ 6×10⁻³²¹6×10⁻³²¹6×10⁻³²¹6×10⁻³²¹6×10⁻³²¹6×10⁻³²¹6×10⁻³²¹6×10⁻³²¹6×10⁻³²¹ 8×10⁻³²¹8×10⁻³²¹8×10⁻³²¹8×10⁻³²¹8×10⁻³²¹8×10⁻³²¹8×10⁻³²¹8×10⁻³²¹8×10⁻³²¹000000000

1×10¹³1×10¹³1×10¹³1×10¹³1×10¹³1×10¹³1×10¹³1×10¹³1×10¹³

2×10¹³2×10¹³2×10¹³2×10¹³2×10¹³2×10¹³2×10¹³2×10¹³2×10¹³

3×10¹³3×10¹³3×10¹³3×10¹³3×10¹³3×10¹³3×10¹³3×10¹³3×10¹³

4×10¹³4×10¹³4×10¹³4×10¹³4×10¹³4×10¹³4×10¹³4×10¹³4×10¹³

5×10¹³5×10¹³5×10¹³5×10¹³5×10¹³5×10¹³5×10¹³5×10¹³5×10¹³

The Badlands: SineThe Badlands: SineThe Badlands: Sine

> (plot (function(λ (x) (define z* (bigfloat->real (bfsin (bf x)))) (flulp-error (flsin (fl x)) z*))0 (* 2 pi)))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

000000000 222222222 444444444 666666666000000000

100100100100100100100100100

200200200200200200200200200

300300300300300300300300300

The Badlands: CosineThe Badlands: CosineThe Badlands: Cosine

> (plot (function(λ (x) (define z* (bigfloat->real (bfcos (bf x)))) (flulp-error (flcos (fl x)) z*))0 (* 2 pi)))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

000000000 222222222 444444444 666666666000000000

200200200200200200200200200

400400400400400400400400400

600600600600600600600600600

800800800800800800800800800

The Badlands: TangentThe Badlands: TangentThe Badlands: Tangent

> (plot (function(λ (x) (define z* (bigfloat->real (bftan (bf x)))) (flulp-error (fltan (fl x)) z*))0 (* 2 pi)))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

000000000 222222222 444444444 666666666000000000

200200200200200200200200200

400400400400400400400400400

600600600600600600600600600

The Badlands: ArcsineThe Badlands: ArcsineThe Badlands: Arcsine

> (plot (function(λ (x) (define z* (bigfloat->real (bfasin (bf x)))) (flulp-error (flasin (fl x)) z*))-1 1))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

-1-1-1-1-1-1-1-1-1 -.5-.5-.5-.5-.5-.5-.5-.5-.5 000000000 .5.5.5.5.5.5.5.5.5 111111111

.25.25.25.25.25.25.25.25.25

.5.5.5.5.5.5.5.5.5

.75.75.75.75.75.75.75.75.75

111111111

1.251.251.251.251.251.251.251.251.25

The Badlands: ArccosineThe Badlands: ArccosineThe Badlands: Arccosine

> (plot (function(λ (x) (define z* (bigfloat->real (bfacos (bf x)))) (flulp-error (flacos (fl x)) z*))0.8 1))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

.85.85.85.85.85.85.85.85.85 .9.9.9.9.9.9.9.9.9 .95.95.95.95.95.95.95.95.95 111111111000000000

505050505050505050

100100100100100100100100100

150150150150150150150150150

200200200200200200200200200

The Badlands: ArctangentThe Badlands: ArctangentThe Badlands: Arctangent

> (plot (function(λ (x) (define z* (bigfloat->real (bfatan (bf x)))) (flulp-error (flatan (fl x)) z*))-10 10))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

-10-10-10-10-10-10-10-10-10 -5-5-5-5-5-5-5-5-5 000000000 555555555 101010101010101010

.2.2.2.2.2.2.2.2.2

.4.4.4.4.4.4.4.4.4

.6.6.6.6.6.6.6.6.6

The Badlands: ExponentialThe Badlands: ExponentialThe Badlands: Exponential

> (plot (function(λ (x) (define z* (bigfloat->real (bfexp (bf x)))) (flulp-error (flexp (fl x)) z*))-100 100))


y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

y a

xis

-100-100-100-100-100-100-100-100-100 -50-50-50-50-50-50-50-50-50 000000000 505050505050505050 100100100100100100100100100

101010101010101010

202020202020202020

303030303030303030

404040404040404040

505050505050505050

The Badlands: Exponential With BaseThe Badlands: Exponential With BaseThe Badlands: Exponential With Base

> (plot3d (contour-intervals3d(λ (x y) (define z* (bigfloat->real

(bfexpt (bf x) (bf y)))) (flulp-error (flexpt (fl x) (fl y)) z*))0 101 -101 101))

000000000

505050505050505050

100100100100100100100100100

150150150150150150150150150

200200200200200200200200200

250250250250250250250250250

100100100100100100100100100

757575757575757575

505050505050505050

252525252525252525

000000000

100100100100100100100100100

505050505050505050

000000000

-50-50-50-50-50-50-50-50-50

-100-100-100-100-100-100-100-100-100x axisx axisx axisx axisx axisx axisx axisx axisx axis

y axis

y axis

y axisy axis

y axisy axis

y axis

y axis

y axis

Condition NumberCondition NumberCondition Number

(: condition ((Flonum -> Flonum)(Flonum -> Flonum)-> (Flonum -> Flonum)))

(define ((condition f df) x) (abs (/ (* x (df x)) (f x)))) (: condition2d ((Flonum Flonum -> Flonum)

(Flonum Flonum -> (Values Flonum Flonum))-> (Flonum Flonum -> Flonum)))

(define ((condition2d f df) x y) (define-values (dx dy) (df x y)) (define z (f x y)) (max (abs (/ (* x dx) z))

(abs (/ (* y dy) z))))

debugging floating-point math in racket · finite approximation • actual flonum fields are...

Documents