278 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 40, NO. 3, MAY 2010

Geometric Distortion Insensitive ImageWatermarking in Affine Covariant Regions

Xinbo Gao, Senior Member, IEEE, Cheng Deng, Xuelong Li, Senior Member, IEEE,and Dacheng Tao, Member, IEEE

Abstract—Feature-based image watermarking schemes, whichaim to survive various geometric distortions, have attracted greatattention in recent years. Existing schemes have shown robustnessagainst rotation, scaling, and translation, but few are resistant tocropping, nonisotropic scaling, random bending attacks (RBAs),and affine transformations. Seo and Yoo present a geometricallyinvariant image watermarking based on affine covariant regions(ACRs) that provide a certain degree of robustness. To furtherenhance the robustness, we propose a new image watermarkingscheme on the basis of Seo’s work, which is insensitive to geometricdistortions as well as common image processing operations. Ourscheme is mainly composed of three components: 1) feature se-lection procedure based on graph theoretical clustering algorithmis applied to obtain a set of stable and nonoverlapped ACRs; 2)for each chosen ACR, local normalization, and orientation align-ment are performed to generate a geometrically invariant region,which can obviously improve the robustness of the proposed wa-termarking scheme; and 3) in order to prevent the degradation inimage quality caused by the normalization and inverse normaliza-tion, indirect inverse normalization is adopted to achieve a goodcompromise between the imperceptibility and robustness. Experi-ments are carried out on an image set of 100 images collected fromInternet, and the preliminary results demonstrate that the devel-oped method improves the performance over some representativeimage watermarking approaches in terms of robustness.

Index Terms—Affine covariant region (ACR), dominant gra-dient, image normalization, image watermarking, orientationalignment.

I. INTRODUCTION

W ITH the rapid developments of information technolo-gies, images and videos can be accessed, distributed, and

copied in many convenient ways, but this also brings many prac-

Manuscript received May 25, 2009; revised August 22, 2009 and November13, 2009. First published January 12, 2010; current version published April14, 2010. This work was supported by National Science Foundation of China(60771068, 60702061, 60832005), by the Open-End Fund of National Labo-ratory of Pattern Recognition in China and National Laboratory of AutomaticTarget Recognition, China, by the 100 Talents Program of The Chinese Academyof Sciences, by the Program for Changjiang Scholars and innovative ResearchTeam in University of China (IRT0645), by the Nanyang Technological Uni-versity Nanyang SUG Grant (under project number M58020010), by the Mi-crosoft Operations PTE LTD-NTU Joint R&D (M48020065), and by the K. C.Wong Education Foundation Award. This paper was recommended by AssociateEditor L. Zhang.

X. Gao and C. Deng are with the School of Electronic Engineering,Xidian University, Xi’an 710071, China (e-mail: xbgao@mail.xidian.edu.cn;chdeng.xd@gmail.com).

X. Li is with the State Key Laboratory of Transient Optics and Photonics,Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sci-ences, Xi’an 710119, China (e-mail: xuelong_li@opt.ac.cn).

D. Tao is with the School of Computer Engineering, Nanyang TechnologicalUniversity, Singapore 639798, Singapore (e-mail: dctao@ntu.edu.sg).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TSMCC.2009.2037512

tical problems, e.g., illegal transmissions and manipulations. Asa consequence, protecting the copyright of images and videoshas been a challenging issue, and image watermarking and videowatermarking techniques have been widely studied [1], [2].

Image watermarking has been widely studied as a popularand powerful technique of copyright protection in the imagetransmission and processing. It embeds imperceptible copy-right information, termed watermark, in digital images. Owner-ship of the contents can be verified by detecting the embeddedinformation.

A desired image watermarking scheme must resist a widevariety of possible attacks. Usually, attacks on a watermarkingscheme can be classified as common image processing oper-ations and geometric distortions [1]. The common image pro-cessing operations include median filtering, noise contaminat-ing, and compression, e.g., Joint Photographic Experts Group(JPEG) and JPEG-2000. In the past few years, many schemes[23], [24] have been demonstrated to be effective against com-mon image processing operations. However, it is still an openproblem to deal with geometric distortions, which bring syn-chronization errors and thus disable detectors to detect water-marks preserved in distorted images. Representative geometricdistortions include rotation, scaling, translation, cropping, andaffine transformations.

The existing watermarking methods that addressed the issueagainst geometric distortions can be divided into three cate-gories. The first category includes those that embed a water-mark in the geometric invariant domain. In [3], the watermarkis embedded in the magnitude of Fourier–Mellin transform toachieve robustness to affine transforms. However, watermarkingtechniques involving invariant domain are inherently vulnera-ble to cropping and local geometric distortions, and are usuallydifficult to implement.

The methods belonging to the second category is to embeda template along with the watermark or insert a periodic wa-termark pattern. In [4], a template is embedded in the discreteFourier transform (DFT) domain to generate the shape of lo-cal peaks, which are utilized to recover geometric parametersin the detection end. The major limitation is that the template-based methods are vulnerable to template estimation attacks andincompetent to estimate the attack parameters of some compli-cated geometric distortions.

The third category is feature-based watermarking techniques.By binding the watermark with the geometrically invariant fea-tures, the watermark synchronization error can be avoided.Moreover, since the watermark is embedded in a number of localregions formed by feature points, such watermarking methods

1094-6977/$26.00 © 2010 IEEE

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.

GAO et al.: GEOMETRIC DISTORTION INSENSITIVE IMAGE WATERMARKING IN AFFINE COVARIANT REGIONS 279

can resist cropping. Bas et al. [5] adopted Harris detector toextract feature points, then divided the image into a set of dis-joint triangles by using the Delaunay tessellation, and finally,both the watermark embedding and detection were conducted inthese triangles. Since the Harris detector is not robust enough,this method performs poorly for many attacks. In [6], Mexi-can Hat wavelet filtering was used for feature points extraction,and watermark was embedded in the normalized local regionscentered at the feature points. However, the size of local re-gions remains fixed, so that this scheme is vulnerable to rotationand scaling. To further enhance the robustness of the feature-based image watermarking, scale-space theory is applied forfeature points extraction. In [7], feature points were extractedby Harris–Laplace detector, and several copies of the water-marks were thereafter embedded in the circular disk centeredat the feature points. However, the drawback of this scheme isthat the circular embedding region is highly sensitive to non-isotropic scaling and projective transformations. To this end,Seo and Yoo [8] exploited the affine invariant point detector [9]to extract feature points. For a chosen feature point, an ellipticalfeature region (viz., affine covariant region) is formed and usedfor embedding in spatial domain. Before embedding, the wa-termark is geometrically transformed into an elliptical patternaccording to the shape of the region. This approach possessesa certain degree of robustness and provides a potential idea forresistance against complicated geometric distortions. However,the overlapping between feature regions has not been resolvedin this approach.

After comparing the previous geometrically robust image wa-termarking methods, we find that the feature-based watermark-ing approaches are better than others in terms of robustness.Therefore, the goal of this paper is to propose a feature-basedimage watermarking scheme that can resist geometric distor-tions as well as common image processing operations. Inspiredby the work of Seo and Yoo [8], the proposed scheme alsoutilizes affine covariant regions (ACRs) for watermark embed-ding and detection. However, as described previously, someproblems that need solution still exist in [8]. First, the featureselection process used in [8] does not completely solve the is-sue of the overlapping between the feature regions, but, in fact,embedding watermark into a number of nonoverlapped featureregions is the first prerequisite of designing a robust image wa-termarking. In view of the multicharacteristic of scale-spacefeature points, feature selection process combined with cluster-ing algorithm should be a promising solution to this problem.Second, although the feature regions in [8] have been calledinvariant regions, in principle, they should be termed covariantregions since they deform covariantly with the transformation.The confusion probably arises from the fact that, even thoughthe regions themselves are covariant, the normalized image pat-tern they covered is typically invariant [10]. So, an ACR directlyused as the embedding unit inherently restricts the obtainablerobustness against geometric distortions. As to this, combiningACR extraction with geometrically invariant region construc-tion will evidently improve the robustness of the watermarkingsystem. Third, to achieve the invariance against rotation, thedominant orientation of the ACR in [8] is calculated after trans-

forming the region to a circular region of a fixed size, and thenthe elliptical watermark after orientation normalization is em-bedded additively to the ACR. However, there is no guaranteethat [8] can resist affine transformations only by normalizingthe shape of the watermark. In this case, we observe that theaffine ambiguity can be reduced to a rotational one by normaliz-ing each elliptical region into a circle, and full affine invarianceis achieved by eliminating rotation effect over normalized re-gions [11]. So, embedding watermarks in the normalized ACRscan achieve more robustness against affine transformations.

In this paper, we propose a new image watermarking schemeby incorporating the advantages of the affine invariant point de-tector, the image normalization, and the orientation alignmentseamlessly. The affine invariant point detector is adopted to ex-tract ACRs. The graph theoretical clustering algorithm is thenemployed to select a set of nonoverlapped ACRs for watermarkembedding. In order to achieve affine invariance, each region islocally normalized by transforming an ellipse into a circle androtated to align with its dominant gradient orientation. In water-mark embedding, circular watermark pattern is embedded in thenormalized patch. For the purpose of imperceptibility after wa-termarking, an image-dependent visual model is utilized to ad-just the embedding strength. Finally, a thorough evaluation usingthe standard benchmark, Stirmark, is conduct to demonstrate theperformance of the proposed scheme in improving robustness.

Although the proposed scheme follows the framework of [8],there are a number of significant contributions that the pro-posed method presents. First, graph theoretical clustering al-gorithm is integrated into the feature selection process to dealwith the overlapping between the ACRs. By using this new se-lection process, a set of nonoverlapped ACRs can be explicitlychosen, which can evidently improve the performance of thisproposed scheme. Second, the selected ACRs have good spa-tial localization, orientation selection, distinctiveness, and moreimportantly, their covariance records not only the geometric2-D transformations, but also nonrigid deformations [11], [22].Therefore, these regions are robust to the geometric distortions,especially cropping, nonisotropic scaling, and random bend-ing attacks (RBAs). Third, although the analysis indicates thatthe embedding method between [8] and the proposed schemeseems to be mathematically equivalent, the implements are dif-ferent. In [8], the normalized ACRs are only used to calculatethe dominant orientation to achieve resilience against rotation.By comparison, the normalized ACRs in the proposed schemeare applied not only to compute the dominant orientation, butalso to reduce the affine ambiguity to the rotational one. There-fore, the proposed scheme is more robust than [8] against ge-ometric distortions. Fourth, considering that normalization andinverse normalization could degrade the imperceptibility of thewatermarked images, Seo and Yoo [8] prefer to transform wa-termark than to normalize feature regions. Unlike [8], we adoptan indirect inverse normalization to achieve a good compromisebetween imperceptibility and robustness.

The rest of this paper is organized as follows. Section II firstbriefs the exploited affine invariant point detector and ACRsselection procedure. In Section III, the proposed scheme is de-tailed in two sections: embedding and detection, respectively.

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.

280 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 40, NO. 3, MAY 2010

Experimental results and detailed analysis are given inSection IV, and Section V summarizes the proposed image wa-termarking scheme.

II. AFFINE COVARIANT REGION DETECTION

Two issues concerning the proposed watermarking method,feature points extraction, and ACRs selection play importantroles in achieving the desire goal. Therefore, they will be dis-cussed in this section.

A. Affine Invariant Point Detector

Lindeberg and Garding [12] extended the notion of the scalespace to the affine Gaussian scale space, which is generated byconvolution with nonuniform Gaussian kernel as

g (x,Σ) =1

2π√

det Σexp

(−xT Σ−1x

2

)(1)

where Σ is a symmetric positive definite 2 × 2 matrix. Theaffine Gaussian scale space is given by

L (x,Σ) = g (x,Σ) ⊗ f (x) (2)

where f (x) is an input image and ⊗ denotes linear convolution.For a given point x, the second moment matrix µ in the affineGaussian scale space is defined by

µ (x,Σs ,Σt) = g (x,Σs) ⊗[∇L (x,Σt)∇LT (x,Σt)

](3)

where∇ is the gradient operator, and Σs and Σt are, respectively,the covariance matrices corresponding to the integration scaleand local scale.

The affine invariant point detector is based on the affine nor-malization around multiscale Harris points. After a set of featurepoints are detected by multiscale Harris detector, an iterativeprocedure is utilized to estimate elliptical ACRs around thefeature points, wherein these regions are relatively invariant toarbitrary affine transformations. Consider an image f(x) and itscorresponding linearly transformed image f(x) = f(Bx), theaffine scale-space second moment matrices µ and µ of f and f ,respectively, satisfy

µ (x,Σs ,Σt) = BT µ(Bx,BΣsBT ,BΣtBT

)B. (4)

It can be proven that if the following property holds for thesecond moment matrix ML at a fixed point xL of the image f ,i.e.,

µ (xL ,Σs,L ,Σt,L ) =ML ,Σs,L = s · M−1L ,

and Σt,L = t · M−1L (5)

then the second moment matrix at the point xR (xR = BxL ) ofthe image f satisfies the following property:

µ (xR ,Σs,R ,Σt,R ) =MR ,Σs,R=s · M−1R ,

and Σt,R = t · M−1R (6)

where M−1 is the inverse of the second moment matrix M ands, t ∈ + .

Baumberg [13] employed the square root of the second mo-ment matrix to transform an image to a normalized one. The

transformed images of f(x) and f(x) are, respectively, definedas

f ′(M−1/2L x

)= f (x) and f ′(M−1/2

R x)

= f (x) (7)

where M−1/2 is the square root matrix of M. By using thetransformation property, the normalized image can be shown as

µ′(M−1/2L xL , sI, tI

)= µ′(M−1/2

R xR , sI, tI)

= I (8)

where I is the 2 × 2 identity matrix. Thus, the affine transfor-mation can be expressed as

B = M−1/2R RM1/2

L (9)

where R is a rotational matrix.If the neighborhood of points xL and xR are normal-

ized by transformations x′L = M1/2

L xL and x′R = M1/2

R xR ,respectively, the normalized regions are related according tox′

L = Rx′R , which is simply a rotational transformation.

We further simplify the affine invariant point detector withfour steps: 1) detect initial feature points by using multiscaleHarris detector; 2) select the characteristic scale for each of thelocated feature points; 3) determine the shape of a point by theeigenvalues and eigenvectors of the second moment matrix; and4) normalize ACRs according to x′ = R1/2x.

B. Sparse Representation of ACRs

Affine invariant point detector is potential for object recog-nition and image retrieval. Feature points detected by an affineinvariant point detector are highly distinctive and matched witha relatively high repeatability against various image processing.For each feature point, the affine invariant point detector can beused to obtain a set of properties including location (x, y), scaleσ, and strength s, in which definitions of scale and strength canbe referred to [9]. However, ACRs centered at feature pointsare not directly applicable to watermarking, because not all theoriginal regions are stable and overlapped seriously with eachother usually.

In order to embed watermark in a host image, appropriateACRs should be first selected. Usually, features with small scaleshave a low repeatability, and features with large scales meanthat regions will seriously overlap with each other. As a conse-quence, a set of features whose scale is in the middle-scale band(4 ≤ σ ≤ 9), denoted as Ω, are selected for further analysis.

In addition, it is well known that the affine invariant pointdetector extracts important parts of an image, and it indicatesnot only the local information, but also large structures implic-itly via the overlap. Thus, the distribution of features should beconsidered carefully. If the distance between any two regions issmall, they will overlap in large areas. If the distance is large, thenumber of regions will be not enough for watermark embedding.Similar to that discussed in [14]–[16], a distance-related con-straint is used to control the distribution of extracted features.In the proposed scheme, we utilize the minimum spanning tree(MST) [17] based clustering algorithm to group these features.

Fig. 1 illustrates the MST obtained from 2-D feature pointsfor Baboon image. Given a set consisting of 47 feature pointsin Ω, χn = (x1 , x2 , . . . , x47), a spanning tree is a connected

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.

GAO et al.: GEOMETRIC DISTORTION INSENSITIVE IMAGE WATERMARKING IN AFFINE COVARIANT REGIONS 281

Fig. 1. Example of feature selection. (a) Minimum spanning tree constructedon the feature points detected from Baboon image and (b) the result of MSTclustering.

Fig. 2. Illustration of ACRs selection for Baboon, Lena, and Peppers, respec-tively. (a) Original ACRs. (b) ACRs in middle-scale band. (c) Final chosenACRs for watermarking.

acyclic graph that passes through all coordinates associated withthe point set χn . As shown in Fig. 1(a), for a given weightedgraph, the minimum spanning tree is the spanning tree thatminimizes the total weight of all the edges in the graph. Givena distance threshold D, 15 clusters are obtained by removingedges whose weights are larger than D. In other words, featureswhose adjacent distance is less than D will be assigned into anidentical cluster. Fig. 1(b) shows the final clustering result. Forthe same cluster, feature whose strength is the largest is used toconstruct ACR. Moreover, the distance constraint D can also betreated as an implicit parameter to enhance the security of theproposed watermarking scheme. This means that the receiverwill not be able to generate the same regions if he or she doesnot know this parameter.

Specifically, Fig. 2 illustrates the procedures in selecting theACRs for Baboon, Lena, and Peppers. Fig. 2(a) illustrates theACRs extracted by the affine invariant point detector. Fig. 2(b)shows the selected ACRs in the middle-scale band. The finallydetermined ACRs for watermarking are displayed in Fig. 2(c).

Fig. 3. Extracted ACR in (a) the original image, (b) the median-filtered image,(c) the 40% JPEG image, (d) the Gaussian-filtered image, (e) the 5 rotatedimage, and (f) the 1.5× scaled image.

As shown in Fig. 2, the elliptical shapes associated with affineinvariant points are determined by the second moment matrixM. For an affine transformation, scaling is different in everydirection. Nonisotropic scaling has an influence on the spatiallocalization, the scale, and the shape of a local structure. There-fore, elliptical regions are more adaptive to an anisotropic localstructure.

Fig. 3 shows an ACR extracted from an original image andcorresponding attacked images by using the proposed featureselection procedure. For the convenience of performance eval-uation, we only use one ellipse to illustrate an identical im-age region, which is usually formulated robustly, even whenthe image is distorted by JPEG compression, median filtering,Gaussian filtering, rotation, and scaling. Therefore, watermarkembedding and detection are implemented on such local ACRsfor resisting against both geometric distortions and commonimage processing operations.

III. PROPOSED WATERMARKING SCHEME

This section presents the watermark embedding and detectionprocedures of the proposed watermarking scheme. The water-mark sequence C = (c1 , c2 , . . . , cN ) is generated by a secretkey, and then mapped into circular watermark pattern W .

A. Watermark Embedding

For watermark embedding, we first select suitable ACRsaccording to the procedure mentioned in Section II, whereinthe selected regions are possible for watermark embedding.Second, these regions are transformed into circular patchesby normalization and dominant orientation alignment. Finally,the circular watermark pattern is embedded repeatedly in allnormalized patches. This process is visualized in Fig. 4 indetail.Step 1: The affine invariant point detector is used to extract

ACRs according to the procedure mentioned in SectionII-B. In order to embed watermarks, a set of stableand nonoverlapped ACRs are selected according to thescale range, repeatability, and distribution.

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.

282 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 40, NO. 3, MAY 2010

Fig. 4. Framework of the proposed watermark embedding scheme. The detailed procedures can refer to step 1–step 5 in watermark embedding.

Step 2: Each selected ACR Pi is normalized to a circularpatch NPi with a fixed radius r. When all ACRs arenormalized, corresponding regions are defined only bya simple rotation.The size of the normalized patch needs to be properlyconsidered. Our research has shown that if a largerpatch is warped into a small patch, which means thatthe warping process is a multiple-to-one pixel mapping,then one pixel in NPi represents several pixels in Pi .Under this circumstance, a small normalized patch isbeneficial for achieving robustness. In this study, thesize of normalized patch is empirically found to be39 × 39 (r = 19) for achieving a tradeoff betweenimperceptibility and robustness.

Step 3: For each normalized patch, we can calculate its domi-nant gradient orientation and align the patch accordingto this orientation by rotating. Thus, the rotation invari-ance of the patch can be obtained. Let RPi denote therotated circular patches.In [18], a window centered at these feature points isdefined. The gradients of all pixels in the window arecalculated by using the first-order derivative. Then thehistogram of gradients is computed and the peak of thehistogram is assigned as the dominant orientation ofthe feature point. The histogram has 90 bins to coverall orientations. The dominant orientation is usuallyrobust against noise, small local distortions, and somedisplacement of the feature point position.The gradient of the pixel (x0 , y0) in the image f iscomputed as follows:

∇f (x0 , y0) = [(∂f/∂x) , (∂f/∂y)]|(x0 ,y0 ) . (10)

The magnitude of this gradient is√(∂f/∂x)2 + (∂f/∂y)2 and its orientation is

given by tan−1 [(∂f/∂y)/(∂f/∂x)].

Step 4: To make the embedded watermark imperceptible, weadopt the following image-dependent visual model[19]:

Λ = (1 − NVF) · α + NVF · β (11)

where α and β are the watermark strength. For mostreal-world images, β is set to 3, and α can be adjustedto keep the peak SNR (PSNR) higher than a certainvalue (in our case, α is set to 15).

Step 5: Thereafter, the watermarked circular patch is obtainedby the additive rule, i.e.,

RPwi (x) = RPi(x) + Λ(x) · W (x). (12)

Once the watermarked normalized patch RPwi is ob-

tained, the inverse orientation alignment and the in-verse normalization are used to yield a watermarkedACR. Although “direct inverse normalization” is in-tuitive, blocking effects caused by the one-to-multiplepixel mapping may degrade the imperceptibility [20].To deal with this problem, the difference between RPi

and RPwi , which is caused by watermarking in the nor-

malized domain, is inversely aligned and then inverselynormalized to yield the difference P diff

i in the spatialdomain. Hence, the watermarked ACR in the spatialdomain can be described as follows:

Pwi = Pi + P diff

i . (13)

Finally, by integrating all watermarked ACRs, the wa-termarked image can be obtained.

B. Watermark Detection

The watermark detection stage uses the same feature selec-tion process as it is in the watermark embedding procedure.Even though the watermarked image undergoes specific affinetransformations, the ACRs with the locally largest repeatability

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.

GAO et al.: GEOMETRIC DISTORTION INSENSITIVE IMAGE WATERMARKING IN AFFINE COVARIANT REGIONS 283

Fig. 5. False positive probability versus watermark detection threshold T .

can be conserved. The procedure for watermark detection isdescribed in detail shortly.

Step 1–3: These three steps are the same as steps 1–3 in thewatermark embedding procedure.

Step 4: Since the Wiener filter can separate the image com-ponents from the watermark components, we use itto estimate embedded watermark patterns [4]. Theestimated watermark is converted into a sequenceV = (v1 , v2 , . . . , vN ). The extracted watermark se-quence is then compared with the original water-mark sequence to determine whether a watermarkexists in an ACR.

C. Threshold Selection

For each ACR, the number of its matching bits between Cand V is calculated. If the number is greater than the prede-fined threshold T , a watermark will be declared in an ACR. Todetermine the threshold T , we consider the false-positive prob-ability and the false-negative probability. Usually, it is difficultto analyze the false-negative probability for a wide variety ofdistortions in the procedure of watermark embedding and de-tection. Therefore, it is usual to select a threshold T based on afixed false-positive probability.

For an unwatermarked image, the extracted bits are treated asindependent random variables with probability 0.5. Accordingto Bernoulli trials, the false-positive probability of an ACR is

PFP−region =N∑

i=T

(0.5)N

(N !

i!(N − i)!

)(14)

where T is the predefined threshold, i is the number of thematching bits, and N is the length of the watermark sequence.Furthermore, an image is claimed to be watermarked if at leastm regions are detected. Therefore, the false-positive probabilityof one image is

PFP-image =M∑

j=m

(Mj

)(PFP-region)j (1 − PFP-region)M −j

(15)where M is the total number of ACRs in an image.

We can plot PFP-image against various T values, as shownin Fig. 5, using (15). When the aforementioned parametersare set to T = 160, M = 20, and N = 256, the false-positive

Fig. 6. (a) Original image, (b) watermarked image, and (c) difference imagein Baboon and Lena.

probability of an image for m = 1, 2, and 3 are 4.0 × 10−4 ,9.3 × 10−8 , and 1.2 × 10−11 , respectively.

IV. EXPERIMENTAL RESULTS AND ANALYSIS

We collect 100 512 × 512 gray level images from Internetthat include benchmark images Baboon, Boat, Lena, and Pep-pers to build a dataset. The performance of the proposed wa-termarking scheme is evaluated in two aspects: imperceptibilityand robustness. The watermark is a 256-length (N = 256) bipo-lar zero-mean pseudorandom sequence generated by a secretkey.

A. Imperceptibility Test

The original images and the watermarked images (Baboonand Lena) obtained by the proposed algorithm are shown inFig. 6(a) and (b), respectively. The difference between the orig-inal images and the watermarked versions are magnified bya factor 100 and is shown in Fig. 6(c). Based on (11), it isclear that, for a region with high local variance (NVF→ 0), astrong watermark is embedded, while for a homogenous region(NVF→ 1), the watermark strength is low. Through NVF, itcan modulate the watermark strength adaptively based on lo-cal image characteristics and meanwhile make the embeddedwatermark hardly perceptible. In addition, the overall PSNRvalues of all images in our testing set between the original andwatermarking versions are greater than 40 dB.

B. Robustness Test

Apart from the imperceptibility test, two experiments areconducted to measure: 1) the stability of feature points extractedfrom the affine invariant point detector and 2) the robustness ofthe proposed watermarking scheme.

Since the affine invariant point detector cannot provide sucha high accuracy, local search is adopted to compensate forsynchronization errors induced by geometric distortions inwatermark detection end. In [8], search space consists of po-sition, characteristic scale, and orientation. Taking the compu-tational cost into consideration, we do not use the same com-pensation method as in [8]. Similar to the scheme used in [14],

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.

284 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 40, NO. 3, MAY 2010

Fig. 7. Corresponding ratio under (a) common image processing operations and (b) geometric distortions for Baboon, Boat, Lena, and Peppers.

TABLE IWATERMARK DETECTION RESULTS UNDER COMMON IMAGE PROCESSING OPERATIONS

we compensate for the synchronization errors by changing theposition of each ACR in the amount of ±2 pixel.

1) Stability of the Affine Invariant Points: In our scheme,only a set of feature points chosen by our feature selectionprocess are used to form ACRs. The stability of these featurepoints is important for robustness. To measure the stability,we first extract feature points from the original images andthe attacked images, and then compute the repeatability ratiobetween the number of point-to-point correspondences and thenumber of points detected in original images.

By utilizing Stirmark 4.0 [21], we apply various attacks to fourbenchmark images, Baboon, Boat, Lena, and Peppers. Theseattacks include median filtering (3 × 3), Gaussian filtering(3 × 3), JPEG compression (QF 40 and 50), cropping (5% and10% off), rotation (5 and 10), scaling (0.9×,1.1×), and rota-tion + cropping (1 and 5).

Fig. 7(a) and (b) illustrates the results under common imageprocessing operations and geometric distortions, respectively.Corresponding ratio refers to the ratio of the number of cor-responding feature points between the original images and theattacked versions to the number of the feature points extractedfrom the original images. It can be found that the correspondingratio is more than 60% on average for common image process-ing operations, and nearly 50% feature points can be redetectedin geometric distortions. Therefore, these selected feature pointsare stable, and can resist both common image processing oper-ations and geometric distortions.

2) Robustness of the Proposed Scheme: We use Stirmark4.0 to evaluate the robustness of the proposed scheme. Tables I

and II present the detection results of the proposed watermark-ing scheme in comparison with three representative schemes,[6]–[8], under common image processing operations and geo-metric distortions, respectively. The values in main table unitesindicate the ratio of the number of regions where watermarksare successfully detected from attacked images to the numberof original watermarked regions. Table III presents the averagedetection results of all 100 images. Detection ratio indicates thefraction of correctly detected watermarked ACRs, and detectionfailure refers to the number of images where the embedded wa-termark is not detected from any ACRs, and hence, it fails toprove ownership.

For most of the attacks, the proposed watermarking schemecan detect the embedded watermark from a considerable numberof ACRs and the ownership can be proven with high confidence.Since a watermark sequence is mapped into a circular pattern,small alignment errors can be compensated. The affine invari-ant point detector can provide a set of distinctive and localizedACRs that are covariant to significant affine transformation.In the procedure of watermark embedding, normalization anddominant gradient orientation alignment are applied to theseACRs, which can achieve rotation and scaling invariance. Con-sequently, the proposed watermarking scheme performs wellin common image processing operations, geometric distortions,and even the combined complex attacks.

As presented in Tables I and II, the newly proposed schemeoutperforms the conventional schemes [6]–[8]. Among thesethree existing schemes, scheme [8] shows the worst perfor-mance. According to the detection results of [6], this scheme

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.

GAO et al.: GEOMETRIC DISTORTION INSENSITIVE IMAGE WATERMARKING IN AFFINE COVARIANT REGIONS 285

TABLE IIWATERMARK DETECTION RESULTS UNDER GEOMETRIC DISTORTIONS

TABLE IIIWATERMARK DETECTION RESULTS UNDER VARIOUS ATTACKS

can withstand many common image processing operations, e.g.,Gaussian filtering, JPEG compression, and noise contaminating,but fails to rotation, scaling, and particularly nonisotropic scal-ing. For [7], it shows better performance than [6], but, as afore-

mentioned, this scheme is susceptible to large scaling attacksand nonisotropic scaling.

Obviously, the proposed scheme is computationally more de-manding than other watermarking methods that do not considerfeature points extraction. The watermark embedding and detec-tion of this proposed scheme can be done within a minute ona 3.4-GHz Pentium D processor for 512 × 512 images, whichis acceptable for most applications. Furthermore, the detectionresults of the new scheme still strongly depend on the repeata-bility of the affine invariant point detector. The repeatability inimages with strong edge information is better than in texturedimages. Our experiments also revealed this fact. In the future,we will consider the use more robust watermarking schemesfor large-scale visual information secure retrieval and manage-ment [25]–[27].

V. CONCLUSION

In this paper, we developed a new scheme for feature-basedimage watermarking to resist geometric distortions and commonimage processing operations. The problem of synchronizationerror in image watermarking can also be solved by applyingthe affine invariant points of an image. In addition, the robust-ness of the proposed scheme, especially to nonisotropic scal-ing, RBAs, and other affine transformations, is improved byincorporating ACRs with normalization and orientation align-ment steps. Extensive experimental results showed the newlyproposed scheme performed well under various geometric dis-tortions and common image processing operations, and outper-formed several representative schemes in terms of robustness togeometric distortions.

ACKNOWLEDGMENT

The authors would like to thank the helpful comments andsuggestions from the anonymous reviewers.

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.

286 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 40, NO. 3, MAY 2010

REFERENCES

[1] D. Zheng, Y. Liu, and J. Zhao, “A survey of RST invariant image water-marking algorithms,” ACM Comput. Surv., vol. 39, no. 2, pp. 1–91, Jun.2007.

[2] Z. Zhao, N. Yu, and X. Li, “A novel video watermarking scheme incompression domain based on fast motion estimation,” in Proc. IEEE Int.Conf. Commun. Technol., 2003, pp. 1878–1882.

[3] J. K. O’Ruanaidh and T. Pun, “Rotation, scale and translation invariantspread spectrum digital image watermarking,” Signal Process., vol. 66,no. 3, pp. 303–317, Aug. 1998.

[4] S. Pereira and T. Pun, “Robust template matching for affine resistant imagewatermarks,” IEEE Trans. Image Process., vol. 9, no. 6, pp. 1123–1129,Jun. 2000.

[5] P. Bas, J. M. Chassery, and B. Macq, “Geometrically invariant water-marking using feature points,” IEEE Trans. Image Process., vol. 11,no. 9, pp. 1014–1028, Sep. 2002.

[6] C. W. Tang and H. M. Hang, “A feature-based robust digital image water-marking scheme,” IEEE Trans. Signal Process., vol. 51, no. 4, pp. 950–959, Apr. 2003.

[7] J. S. Seo and C. D. Yoo, “Localized image watermarking based on featurepoints of scale-space representation,” Pattern Recogn., vol. 37, no. 7,pp. 1365–1375, Jul. 2004.

[8] J. S. Seo and C. D. Yoo, “Image watermarking based on invariant regionsof scale-space representation,” IEEE Trans. Signal Process., vol. 54, no. 4,pp. 1537–1549, Apr. 2006.

[9] K. Mikolajczyk and C. Schmid, “Scale & affine-invariant interest pointdetectors,” Int. J. Comput. Vis., vol. 60, no. 1, pp. 63–86, Oct. 2004.

[10] K. Mikolajczyk and C. Schmid, “A comparison of affine region detectors,”Int. J. Comput. Vis., vol. 65, no. 1/2, pp. 43–72, Nov. 2005.

[11] S. Lazebnik, C. Schmid,, and J. Ponce, “A sparse texture representationusing local affine regions,” IEEE Trans. Pattern Anal. Mach. Intell.,vol. 27, no. 8, pp. 1265–1278, Aug. 2005.

[12] T. Lindeberg and J. Garding, “Shaped-adapted smoothing in estimation of3-D shape cues from affine deformation of local 2-D brightness structure,”Image Vis. Comput., vol. 15, no. 6, pp. 415–434, Jun. 1997.

[13] A. Baumberg, “Reliable feature matching across widely separated views,”in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recogn., Jun. 2000, vol. 1,pp. 774–781.

[14] H. Y. Lee, H. S. Kim, and H. K. Lee, “Robust image watermarking us-ing local invariant features,” Opt. Eng., vol. 45, no. 3, pp. 1–11, Mar.2006.

[15] C. Deng, X. Gao, X. Li, and D. Tao, “A local Tchebichef moments-basedrobust image watermarking,” Signal Process., vol. 89, no. 8, pp. 1531–1539, Aug. 2009.

[16] C. Deng, X. Gao, X. Li, and D. Tao, “Geometrically invariant water-marking using affine covariant regions,” in Proc. IEEE Int. Conf. ImageProcess., San Diego, CA, Oct. 2008, vol. 4, pp. 413–416.

[17] J. C. Gower and G. J. S. Ross, “Minimum spanning tree and single linkagecluster analysis,” Appl. Statist., vol. 18, no. 1, pp. 54–64, Jan. 1969.

[18] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,”Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, Nov. 2004.

[19] S. Voloshynovskiy, A. Herrigel, N. Baumgrtner, and T. Pun, “A stochasticapproach to content adaptive digital image watermarking,” in Proc. 3rdInt. Workshop Inf. Hiding, Dresden, Germany, 1999, pp. 212–236.

[20] C. S. Lu, S. W. Sun, C. Y. Hsu, and P. C. Chang, “Media hash-dependentimage watermarking resilient against both geometric attacks and esti-mation attacks based on false positive-oriented detection,” IEEE Trans.Multimedia, vol. 8, no. 4, pp. 668–685, Aug. 2006.

[21] F. A. P. Petitcolas, “Watermarking schemes evaluation,” IEEE SignalProcess. Mag., vol. 17, no. 5, pp. 58–64, Sep. 2000.

[22] Y. Pang, Y. Yuan,, and X. Li, “Gabor-based region covariance matricesfor face recognition,” IEEE Trans. Circuits Syst. Video Technol., vol. 18,no. 7, pp. 989–993, Jul. 2008.

[23] S. C. Chu, H. C. Huang, Y. Shi, S. Y. Wu, and C. S. Shieh, “Geneticwatermarking for zero tree-based application,” Circuits, Syst., SignalProcess., vol. 27, no. 2, pp. 171–182, 2008.

[24] H. C. Huang, C. M. Chu, and J. S. Pan, “The optimized copyright pro-tection system with genetic watermarking,” Soft Comput., vol. 13, no. 4,pp. 333–343, 2009.

[25] X. Li, “Watermarking in secure image retrieval,” Pattern Recogn. Lett.,vol. 24, no. 14, pp. 2431–2434, 2003.

[26] D. Tao, X. Li, and S. Maybank, “Negative samples analysis in relevancefeedback,” IEEE Trans. Knowl. Data Eng., vol. 19, no. 4, pp. 568–580,Apr. 2007.

[27] J. Li, N. Allinson, D. Tao, and X. Li, “Multitraining support vector ma-chine for image retrieval,” IEEE Trans. Image Process., vol. 15, no. 11,pp. 3597–3601, Nov. 2006.

Xinbo Gao (M’02–SM’07) received the B.Sc.,M.Sc., and Ph.D. degrees in signal and informationprocessing from Xidian University, Xi’an, China, in1994, 1997, and 1999, respectively.

From 1997 to 1998, he was a Research Fellow withthe Department of Computer Science, Shizuoka Uni-versity, Shizuoka, Japan. From 2000 to 2001, he wasa Postdoctoral Research Fellow with the Departmentof Information Engineering, The Chinese Universityof Hong Kong, Shatin, Hong Kong. Since 2001, hehas been with the School of Electronic Engineering,

Xidian University, where he is currently a Professor of pattern recognition andintelligent system and the Director of the Video and Image Processing System(VIPS) Laboratory. He has published four books and more than 100 scientificpapers in refereed journals and proceedings, including the IEEE TRANSACTIONS

ON IMAGE PROCESSING, the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS

FOR VIDEO TECHNOLOGY, the IEEE TRANSACTIONS ON NEURAL NETWORKS,the IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C:APPLICATIONS AND REVIEWS, etc. His research interests include computationalintelligence, machine learning, computer vision, pattern recognition, and artifi-cial intelligence.

Prof. Gao is a Fellow of the Institution of Engineering and Technology(IET). He is on the Editorial Boards of many journals, including EURASIPSignal Processing (Elsevier) and Neurocomputing (Elsevier). He served as Gen-eral Chair/Co-Chair or Program Committee Chair/Co-Chair or PC member foraround 30 major international conferences. He is a member of the IEEE Xi’anSection Executive Committee, the Membership Development Committee Chair,and the Vice Chairman of IET Xi’an Network.

Cheng Deng received the B.Sc., M.Sc., and Ph.D.degrees in signal and information processing fromXidian University, Xi’an, China.

Currently, he is a Lecturer at the School of Elec-tronic Engineering, Xidian University. His researchinterests include information hiding, digital water-marking, and image quality assessment.

Xuelong Li (M’02–SM’07) is currently a Researcher (Full Professor) with theState Key Laboratory of Transient Optics and Photonics, Xi’an Institute ofOptics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, China.

Dacheng Tao (M’07) received the B.Eng. degreefrom the University of Science and Technology ofChina, Hefei, China, the M.Phil. degree from the Chi-nese University of Hong Kong, Shatin, Hong Kong,and the Ph.D. degree from the University of London,London, U.K.

Currently, he is the Nanyang Assistant Professorwith the School of Computer Engineering, NanyangTechnological University, Singapore, and also holdsa visiting position in London. He is a Visiting Profes-sor at Xi Dian University, Xi’an, China, and a Guest

Professor at Wu Han University, Wuhan, China. His research interests includeapplying statistics and mathematics for data analysis problems in computer vi-sion, multimedia, machine learning, data mining, and video surveillance. He haspublished more than 100 scientific papers in top venues with best paper runnerup awards and finalists. He has authored or coauthored and edited six books andeight journal special issues.

Dr. Tao is an Associate Editor of the IEEE TRANSACTIONS ON KNOWLEDGE

AND DATA ENGINEERING and the Computational Statistics and Data Analysis(Elsevier). He was the (Co-)Chaired for special sessions, invited sessions, work-shops, panels, and conferences.

Authorized licensed use limited to: XIDIAN UNIVERSITY. Downloaded on May 08,2010 at 08:20:56 UTC from IEEE Xplore. Restrictions apply.