paper fingerprinting using alpha-masked image matching

Copyright CISRA Slide 1 Printed 30 November 2009

Australian R&D with global impact

Paper fingerprinting using alpha-masked image matching

Tuan Pham, Stuart Perry, and Peter FletcherCanon Information Systems Research Australia

Paper 55 Session3B 12:10PM-12:30PM on Thu 3 Dec



Overview

1. Paper FingerPrint (PFP)Random structure of paperUniqueness

2. Alpha-masked image matchingNormalized correlationImage inpainting

3. PFP robustnessEvaluationPossible improvements



Paper fingerprintIntrinsic characteristic of a piece of paper that uniquely describes itself

The coupon

Paper Fingerprint SystemPaper Fingerprint System

The original fingerprint

Match Strength

ThresholdComparison

Not Original Original

Document authentication application



Paper FingerPrinting (PFP) using a scanner

Officepaper



PFP matching using cross-correlationPaper FingerPrint is a 256x256 8-bit grayscale scan at 600dpi

PFP match is determined based on correlation peak strength:Peak strength > 10 → matchPeak strength < 10 → non-match

Using extreme value theory, the false alarm rate is about 1x10-50

___ Fisher-Tippettdistribution fit

Non-Match PFP strengths Matching PFP strengths



Correlation is not robust against changeWe swap 7.5% of pixels around → match strength drops below 10

Printing also decreases match strength & increases false matches

Some printed textcreates a false negative

Low correlation due to the influence of the printed

areas

Same sheet of paper

Same printed textcreates a false positive

Same printed textcreates a false positive

High correlation due to the influence of the

printed areas

Different sheets of paper



Solution 1: alpha-masked correlation [Fitch et al]

Use weight to mask out change areas during least-square matching:

( )20 0 1 2 0 0 1 2 0 0

,

2 21 1 2 1 1 2 2 1 2 2

( , ) ( , ) ( , ) ( , ) ( , )

2x y

E x y f x y f x x y y x y x x y y

f f f f

α α

α α α α α α

= − − − − −

= ⊗ − ⊗ + ⊗

∑α

1 1 2 2

1 2

2Nf fE α αα α⊗

= −⊗

If images f1 & f2 have zero mean, alpha-masked correlation can be simplified to normalized correlation:

α

Alpha-masked correlation

Peak strength = 4.04

Paper 1

Same printed texts

Paper 2

Cross-correlation

Peak strength = 12.44



Solution 2: inpainting followed by correlation

How well do these different solutions compare to each other?

Scanned paper with printed texts Filled-in with mean value Smooth inpainting

Normalized correlation is most discriminative, followed by mean-filled correlation

5.287.597.635.67Ratio

5.365.885.848.16Non-match

28.3444.6244.6446.32Match

InpaintingMean-filledNormalizedAlpha-masked



Fill factor experiment: synthetic maskMask is successively eroded to reduce the fill-factor

…

fill = 0.77

fill = 0.58

fill = 0.45

fill = 0.05

PFP 1

Paper 1 after printing

PFP 2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.80

5

10

15

20

25

30

35

40

45

50

mask fill factor

mat

ch s

treng

th

alpha-masked correlation [8]normalized correlation (section 2.3)mean-filled correlation (section 3.3)inpainting correlation (section 3.3)correlation of non-matching pairs

100

90

80

70

60

50

40

30

20

10

0



The document is scanned twice, 256x256 image patches are matched

Match strength vs. fill factor from a real document

…

fill = 0.94

fill = 0.86

fill = 0.73

fill = 0.16 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

5

10

15

20

25

mask fill factor

mat

ch s

treng

th

alpha-masked correlation [8]

50

40

30

20

10

0



Improve PFP by multiple orientation scans

Reflection from paper consists of diffuse and specular components

These components can be separated by photometric stereography

_+++

paper scanned at 0º paper scanned at 180º

diffuse reflectance specular reflectance



Improve PFP robustness by double-sided scan

Verify the Paper FingerPrint on both sides.

Front and back side must correlate

The displacement between front and back side is fixed

Front side scanned at 0°

Back side scanned at 0°

Correlation of front and back



ConclusionsConventional scanners capture internal structure of paper: Paper FingerPrint (PFP)

PFPs are very unique and can be used for authentication

PFPs can be matched even after printing

PFPs can be made more robust using more than one scans



Thank you

[email protected]

paper fingerprinting using alpha-masked image matching

Technology