EE 6850: Visual Information System

Lecture # 12: Multimedia Security II

Professor Shih-Fu ChangGuest Speaker: Dr. Ching-Yung Lin

Columbia University /IBM T. J. Watson Research Center

cylin@watson.ibm.com

Nov 27, 2002

Outline§ Watermarking

-- Introduction-- Digital Communication and Spread Spectrum

Techniques-- Robustness of Watermarking Techniques -- Other Issues and References

Watermarking

Tx Rx

Watermark Verify thewatermark

PIL or content- based feature codes

• Embedding Visible/Invisible Codes in Multimedia Data for (or not for) Security Purpose

Visible Watermark

♦ Purpose: – Claim the ownership and prevent content piracy.

♦ Properties:– Robust: Watermarks must be very difficult, if not impossible,

to be removed.

– Non-obtrusive: Watermarks must not affect the audiovisual contents too much.

– Visible: It must be visible, but it had better to be insensible.

Visible Watermark Example: IBM Method

♦ Description:– Pixel brightness were altered pixel by pixel.

Depending on the brightness of each mask pixel, if it is larger than a threshold, then we add a value to the corresponding pixel of the original image to make it brighter. Otherwise, if it is smaller than a threshold, then we subtract a value to make the corresponding pixel darker.

♦ Alternation Function:

Watermark mask

♦ Robustness: – Randomly set the adjustment factor.– Randomly locate the mask.– The mask pixels may depend on the

image contents.

Invisible Watermark

♦ Purpose: – Protect ownership and trace illegal use.

♦ Properties -- Transmit a bitstream through a very noisy channel, i.e. the

original picture.

– Robust: The watermark must be very difficult, if not impossible, to

remove. It must be able to survive manipulations to the images, such

as: lossy compression, format transformation, shifting, scaling,

cropping, quantization, filtering, xeroxing, printing, and scanning.

– Invisible: The watermark should not visually affect the image/video

content.

original image

add watermark

manipulation

image after crop-and-replacement and JPEG lossy compression

watermarked SARI image

authentication

authentication& recovery

Invisible Watermark Example: Self Authentication-and-Recovery Images (SARI)

What is Watermarking ? –Multimedia as a Communication Channel

Information

W Encoder Decoder W

§ Analog Communication --§ Encoder/ Decoder:

§ Amplitude Modulation (AM), § Frequency Modulation (FM). èMultiplexing: use different carrier frequencies.

§ Channel: air, wire, water, space, …

Channel

Information

W Encoder Decoder WImage/Video/Audio

§ Watermarking:

§ Basic communication system:

Digital Communication

Information

W Encoder

S

Decoder W

Z ≤ N

SwX ≤ P Sw

Encoding Keys Decoding Keys

§ Encoder may include two stages: Coding and Modulation.§ Coding:§ Error Correction Codes, Scrambling (use cryptographic keys).

§ Modulation: § Time Division Multiple Access (TDMA), Frequency Division Multiple Access

(FDMA), Code Division Multiple Access (CDMA).§ Spread Spectrum is a CDMA technique, which needs modulation keys for

Frequency Hopping or other specific codes.

Spread Spectrum Communication

Information

W Encoder

S

Decoder W

Z ≤ N

SwX ≤ P Sw

Encoding Keys Decoding Keys

• Spread Spectrum Communication:• Orthogonal codebooks, E [ fi · fj] = 0• e.g.:

• f1 = 1 1 –1 1 1 –1 –1 –1 –1 1• f2 = -1 1 –1 1 –1 1 1 –1 1 1

• Detection:• arg (n) max correlation coefficient ( Sw – S , f n )

• Examples

Watermarking on Multimedia Content

Information

S: Source Image (Side Information)W: Embedded InformationX: Watermark (Power/Magnitude Constraint: P)Z: Noise (Power/Magnitude Constraint: N )

W Encoder

S

Decoder W

Z ≤ N

Source Image

SwX ≤ P Sw

PerceptualModel

Private/ Public

DistortionModel

Frequency-Domain Analysis: DCT and DFT

Spectrum

Original Image

DCT: (Discrete Cosine Transform)

DFT: (Discrete Fourier Transform)DFT

Spread Spectrum Watermarking (Cox et. al. 1997)

• Spread Spectrum: T( Sw ) = T( S ) + T(X)• T can be any spatial-frequency transforms. • E.g. Fourier Transforms (DFT, DCT), Wavelet Transforms

• Objectives:• Detect the existence of a specific code, which is served as thecopyright information.• Watermark detection needs the original source.

• Implementation:• Add a specific code on the 1000 largest or the 1000 lowest frequency DCT coefficients of the image.• E.g. T(X) = 1 1 -1 1 1 –1 –1 –1 –1 1 …..

• Detection:• correlation coefficient ( T(Sw ) – T (S) , T(X) )

Considerations for Watermarking System

§ Security – Kerchoff’s assumption

§ Robustness§ Hiding Payload§ Application Scenario

Three Metrics of Watermarking

Amount of embedded information

Robustness

Invisibility

RobustWatermark

FragileWatermark

Semi-FragileWatermark

Robustness of Watermarking

§ What spread spectrum methods can do -- Pixel Value Distortion

§ What spread spectrum methods cannot do -- Geometric Distortion:-- Rotation, Scale,-- Translation, Cropping

Change of Boundary, Padding

original printing printed documentscanning

rescannedimage

• An example of application scenario: Print-and-Scan

Designing Robust Watermarking against Geometric Distortion

rotation

cropping

scaling

R+S+C

S+C

General CroppingStrict Cropping

♦ Some solutions:– 2nd watermark (self-registration template): Univ. of Geneva, 1998.– Recognizable structure: Kutter, 1998. – Invariant coefficients: O’Ruanaidh, 1998; Lin et. al., 2000

♦ Example of Rotation, Scaling, and Cropping

Continuous Fourier coefficients of continuous images after RSC

♦ Rotation in time => Rotation in frequency

♦ Scaling in time => Scaling in frequency

♦ Translation in time => Phase shift in frequency

♦ Information Loss in time => Noise in frequency

♦ Change of Image Size in time => No definition in frequency

),()cossin,sincos(

)cossin,sincos(),(

212121

212121

ffXffffX

ttttxttx

R

FR

=+−→←+−=

θθθθθθθθ

),(),(),(),( 21221121 2

2

1

1 ffXffXxttx SFtt

S =→←= λλλλ

Difference between continuous Fourier Transform and Discrete Fourier Transform

♦ DFT: Samples of the Fourier coefficients of the repeated discrete image.

FT Sampling DFT coefficients

O p e r a t i o n s i n t h e d i s c r e t e i m a g ed o m a i n

D F T S i z e S c a l i n g C r o p p i n g R o t a t i o nI m a g e s i z e A l m o s t n o

e f f e c t *S c a l i n g +P h a s e s h i f t +( I n f o r m a t i o nl o s s )

R o t a t i o n

F i x e d l a r g e s i z e S c a l i n g P h a s e s h i f t +( I n f o r m a t i o nl o s s )

R o t a t i o n

S m a l l e s tr e c t a n g l e w i t hr a d i x - 2w i d t h / h e i g h t

S c a l i n g P h a s e s h i f t +( I n f o r m a t i o nl o s s ) +( S c a l i n g )

R o t a t i o n

S m a l l e s t s q u a r ei n c l u d i n g t h ew h o l e i m a g e

S c a l i n g i n o n ed i m e n s i o n a n dn o e f f e c t * i nt h e o t h e rd i m e n s i o n

S c a l i n g +P h a s e s h i f t +( I n f o r m a t i o nl o s s )

R o t a t i o n

♦ Compare to the continuous domain

DFT coefficients after scaling or zero-padding

t

F.T.f

F.T.(c)

TS/2 f0

t

F.T.f

t

F.T.f

The continuous original signal

(b)T0

TS

TB

f0 fS

fB

2fST0

t f(d)

2T0 f0/2fSTS

discretization

up-sampled (or scaled)

zero-padded

DFT coefficients after rotation

♦ Characteristics: “cross” effect, Cartesian sampling points=> Solutions: Estimate the cross positions from boundary/

larger values

Spectrum

After Rotation After Rotation and CroppingOriginal Image

The Log-Polar Map of Fourier Coefficients

♦ Log-Polar Map

θ

log r

For RST (uniform scaling)

§ Rotation: shift in the θ axis§ Scale without boundary change:

shift in the log r axis. § Translation: no effect on the

magnitudes.§ Scale with boundary change,

cropping: noise.

Projection along the log r axis:§ Cyclic shift for rotation,§ Invariant to scaling.

Algorithm for generating feature vector

Calculate the magnitudes of log-polar coefficients, |Fm|, from DFT magnitudes

Summation of the log of the Fourier-Mellin magnitudes along log r axis

Combine values in orthogonal directionsg1(θ) = g0(θ) + g0(θ+90°)

Subtract g(θ) by its global mean, (whitening filter)

The Feature Vectorfv = g(θl, …, θu)

Zero-padded to double image size

feature vector watermark vectormodified feature vector

Embedding Public Watermark : Feature Vector Shaping

§ Extract a Noise-Like Feature Vector and iteratively shape it to a watermark pattern

§ Estimation differences in the log-polar domain and distribute them in the 2-D DFT domain.

feature vector watermark vectormodified feature vector

(mixed signal)

Spread Spectrum:T( Sw ) = T( S ) + T(X)

Feature Vector Shaping:T( Sw ) ≈ T(X)

Test Example -- Print-and-Scan

After PS, Crop to 360x240 & JPEG CR: 95:1 => ρ=0.64, Z=4.30

After Print & Scan, Crop to 402x266 => ρ=0.80, Z=6.46

Original Image [384x256] Watermarked Image, PSNR 43.8dB, ρ=0.84, Z=7.02

Measure Metric I: False Positive (10,000 images from Corel Image Library, 10 different watermarks)

§ What are False Positive (false alarm) and False Negative (miss)?§ What are ROC curves?

Detection Result

Fact

A BC

Measure Metric II: Robustness (ROC curves of 2,000 images)

Rotation(4°, 8 °,

30 °, 45 °)

Scaledown

(5%, 10%, 15%, 20%)

Scale Up

(5%, 10%15%, 30%)

Trans-Lation

(5%, 10%,15%, 20%)

Information Hiding Capacity

Information

S: Source Image (Side Information)W: Embedded InformationX: Watermark (Power Constraint: P)Z: Noise (Power Constraint: N )

• Channel Capacity: Shannon (1948) (private watermarking) , Costa (1983) (public watermarking)

C = ½ log2 ( 1+ P/N) bit/sample

W Encoder

S

Decoder W

Z ≤ N

Source Image

SwX ≤ P Sw

PerceptualModel

Private/ Public

DistortionModel

-- Is this the final answer or just a beginning?

• Assumptions: • Uniform Power Constraints on Watermark and Noise• An image is a channel

Masking Effects on Human Vision System Models

e.g. JND modelPSNR = 32 dB

Specific Domains: •Watson’s DCT masking (1993)• Watson’s Wavelet masking (1997)• Chou and Li’s JND (1995)• JPEG, QF =50

General models:• Lubin’s HVS model (1993)• Daly’s HVS model (1993)

Some properties of HVS models:

-- Amplitude nonlinearity, Inta-eye blurring, Re-sampling, Contrast sensitivity function, Subband decomposition, Masking, Pooling

How many channels within an image?

Previous Propositions:• Image as parallel channels?

• Parallel Gaussian Channels – Akansu (1999), Servetto (1998), Kundur(2000) -- Possible Drawback: A channel needs infinite codeword length !!

• Image as one channel•Arbitrary Varying Channel (AVC, Csiszar 1989 ) – Possible Drawback: arbitrary varying

New research directions on watermarking capacity:• Image as a variant-state channel• Image coefficient values are discrete• Human vision models

If not power-constraint, but amplitude constraints on noises,è Zero-Error Watermarking Capacity

Watermarking capacity of power-constrained noisy environments

σnoise = 5

WW: 102490 bitsWD: 84675 bitsJPG: 37086 bitsChou: 33542 bits

Reference:Zero-error capacity

JPG: 28672 bits

Resources: Copyright Protection Forum§ The Copyright Protection Technical Working Group

(CPTWG) : http://cptwg.org

§ The Intellectual Property Management and Protection Group of the Moving Picture Experts Group (ISO/IEC JTC1/SC/29/WG11, IPMP group at MPEG) : http://www.cselt.it/mpeg and http://www.mpeg.org

§ TV Anytime Forum: http://www.tv-anytime.org

§ Digital Versatile Disk Forum: http://www.dvdforum.org

§ Secure Digital Music Initiative’s charter: http://www.sdmi.org

§ Open Platform Initiative for Multimedia Access: http://www.cselt.it/ufv/leonardo/opima

§ Digital Audio-Visual Council: http://www.davic.org

Resources: Books

§ Digital Watermarkingby Ingemar Cox, Jeffrey Bloom, Matthew Miller (Oct. 2001)

§ Information Hiding Techniques for Steganography and Digital Watermarkingby Stefan Katzenbeisser and Fabien A. P. Petitcolas (Jan. 2000)

§ Image and Video Database: Restoration, Watermarking and Retrieval

by Hanjalic, Langelaar, Van Roosmalen and Biemond (July 2000)

Resources: Papers§ H. Yu, D. Kundur, and C.-Y. Lin, “Spies, Thieves, and Lies: The Battle for

Multimedia in the Digital Era,” IEEE Multimedia, Vol.8, No. 3, July 2001.

§ I. J. Cox, J. Kilian, F. T. Leighton and T. Shamoon, “Secure Spread Spectrum Watermarking for Multimedia,” IEEE Trans. on Image Processing, Vol. 6, No. 12, Dec. 1997.

§ G. W. Braudaway, K. A. Magerlein, and F. Mintzer, “Protecting Publicly Available Images with a Visible Image Watermark,” SPIE Optical Security and Counterfeit Deterrence Techniques, Vol. 2659, Jan. 1996.

§ C.-Y. Lin and S.-F. Chang, “Semi-Fragile Watermarking for Authenticating JPEG Visual Content,” SPIE Security and Watermarking in Multimedia Contents II, Vol. 3971, Jan. 2000.

§ F. A. P. Petitcolas, R. J. Anderson, and M. G. Kuhn, “Information Hiding: A Survey,” Proceedings of the IEEE, Vol. 87, No. 7, July 1999.

§ J. A. Bloom, I. J. Cox, T. Kalker, J.-P. Linnartz, M. L. Miller and B. Traw, “Copy Protection for DVD Video,” Proceedings of the IEEE, Vol. 87, No. 7, July 1999.

Resources: Links

§ Multimedia Authentication Resources: http://www.ctr.columbia.edu/~cylin/auth

§ Watermarking mailing list: http://www.watermarkingworld.org

§ Information Hiding and Digital Watermarking:

http://www.cl.cam.ac.uk/~fapp2/steganography

Issues in Multimedia Security§ copyright protection, authentication, fingerprinting: system,

theory and techniques § public watermarking techniques, watermarking attacks, quality

evaluations and benchmarks § perceptual models, noise models, information theoretical models § conditional access § legal aspects § watermarking protocols§ security in JPEG2000, MPEG-4, MPEG-7 or MPEG21 § biometrics and multimedia security § watermarking/ information hiding applications