Playing detective with digital imagesPlaying detective with digital images
Two issuesTwo issues
• Can we fool casual observers?– How realistic is photorealistic?
• What about experts with lots of time and resources?– Digital image forensics
A history of photo manipulationA history of photo manipulation
• 1860: the portrait of Lincoln is a composite of Lincoln's head and John Calhoun's body
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 1917: “Cottingley fairies”: five pictures taken by Elsie Wright (16) and Frances Griffiths (10) in England
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 1917: “Cottingley fairies”: five pictures taken by Elsie Wright (16) and Frances Griffiths (10) in England
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 1917: “Cottingley fairies”: five pictures taken by Elsie Wright (16) and Frances Griffiths (10) in England
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 1930s: Stalin had disgraced comrades airbrushed out of his pictures
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
http://www.newseum.org/berlinwall/commissar_vanishes/index.htm
A history of photo manipulationA history of photo manipulation
• 1936: same story with Mao
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation• May 1970: This Pulitzer Prize winning photo by John Filo shows Mary
Ann Vecchio screaming as she kneels over the body of student Jeffrey Miller at Kent State University, where National Guardsmen had fired into a crowd of demonstrators, killing four and wounding nine. When this photo was published in LIFE Magazine, the fence post directly behind Vecchiowas removed.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 1989: Oprah’s head on somebody else’s body
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 1994: O.J. Simpson’s mug shot modified to appear more menacing
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 2000: black student’s face inserted into UW brochure
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 2003: Beatles album cover modified to remove cigarette from McCartney’s hand
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation• 2003: This digital composite of a British soldier in Basra, gesturing to Iraqi
civilians urging them to seek cover, appeared on the front page of the Los Angeles Times shortly after the U.S. led invasion of Iraq. Brian Walski, a staff photographer for the Los Angeles Times and a 30-year veteran of the news business, was fired after his editors discovered that he had combined two of his photographs to "improve" the composition.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation• 2003: This digital composite of a British soldier in Basra, gesturing to Iraqi
civilians urging them to seek cover, appeared on the front page of the Los Angeles Times shortly after the U.S. led invasion of Iraq. Brian Walski, a staff photographer for the Los Angeles Times and a 30-year veteran of the news business, was fired after his editors discovered that he had combined two of his photographs to "improve" the composition.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 2004: Composite of John Kerry and Jane Fonda
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 2004: Composite of John Kerry and Jane Fonda
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation• 2006: This photograph by Adnan Hajj, a Lebanese photographer, showed
thick black smoke rising above buildings in the Lebanese capital after an Israeli air raid. The Reuters news agency initially published this photograph on their web site and then withdrew it when it became evident that the original image had been manipulated.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation
• 2007: French Magazine Paris Match altered this photograph of French President Nicolas Sarkozy
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
A history of photo manipulationA history of photo manipulation• 2008: This photograph, by Liu Weiqiang of the Daqing Evening News,
won an award for "one of the ten most impressive news photos of 2006". This photograph was recently revealed to be a composite of two separate photographs: the antelopes and the train.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
Detecting digital tampering: cloningDetecting digital tampering: cloning
• Exposing Digital Forgeries by Detecting Duplicated Image Regions– A.C. Popescu and H. Farid– Technical Report, TR2004-515, Dartmouth College, Computer Science
Detecting digital tampering: retouchingDetecting digital tampering: retouching
• Exposing Digital Forgeries in Color Filter Array Interpolated Images – A.C. Popescu and H. Farid– IEEE Transactions on Signal Processing, 53(10):3948-3959, 2005
Detecting digital tampering: LightingDetecting digital tampering: Lighting
• Exposing Digital Forgeries by Detecting Inconsistencies in Lighting – M.K. Johnson and H. Farid– ACM Multimedia and Security Workshop, New York, NY, 2005
Detecting digital tampering: LightingDetecting digital tampering: Lighting
• Exposing Digital Forgeries by Detecting Inconsistencies in Lighting – M.K. Johnson and H. Farid– ACM Multimedia and Security Workshop, New York, NY, 2005
Detecting inconsistencies in lightingDetecting inconsistencies in lightingI(x,y) = N(x,y)·L + A
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
),(
),(),(
1),(),(),(
1),(),(),(1),(),(),(
22
11
222222
111111
nn
z
y
x
nnznnynnx
zyx
zyx
yxI
yxIyxI
ALLL
yxNyxNyxN
yxNyxNyxNyxNyxNyxN
MMMMM
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
),(
),(),(
1),(),(
1),(),(1),(),(
22
11
2222
1111
nn
y
x
nnynnx
yx
yx
yxI
yxIyxI
A
LL
yxNyxN
yxNyxNyxNyxN
MMMM
I(x,y) = N(x,y) ·L(x,y) + A
Full 3D case:
For points on the occluding contour:
P. Nillius and J.-O. Eklundh, “Automatic estimation of the projected light source direction,” CVPR 2001
Detecting inconsistencies in lightingDetecting inconsistencies in lighting
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
),(
),(),(
1),(),(
1),(),(1),(),(
22
11
2222
1111
nn
y
x
nnynnx
yx
yx
yxI
yxIyxI
A
LL
yxNyxN
yxNyxNyxNyxN
MMMM
Detecting inconsistencies in lightingDetecting inconsistencies in lighting
Fake photo
Real photo
Lighting: Lighting: SpecularSpecular highlights in the eyehighlights in the eye
M.K. Johnson and H. Farid, “Exposing Digital Forgeries Through SpecularHighlights on the Eye,” 9th International Workshop on Information Hiding,
2007
Analyzing illumination in paintingsAnalyzing illumination in paintings• Inferring Illumination Direction Estimated from Disparate Sources in
Paintings: An Investigation into Jan Vermeer's Girl With a Pearl Earring– M.K. Johnson, D. Stork, S. Biswas and Y. Furuichi– SPIE Symposium on Electronic Imaging, San Jose, CA, 2008
Analyzing illumination in paintingsAnalyzing illumination in paintings• Inferring Illumination Direction Estimated from Disparate Sources in
Paintings: An Investigation into Jan Vermeer's Girl With a Pearl Earring– M.K. Johnson, D. Stork, S. Biswas and Y. Furuichi– SPIE Symposium on Electronic Imaging, San Jose, CA, 2008
Analyzing illumination in paintingsAnalyzing illumination in paintings
• Computer Vision, Image Analysis, and Master Art: Part 2 – D. Stork and M.K. Johnson– IEEE Multimedia, October-December 2006
Georges de la Tour
Christ in the Carpenter’s Studio (1645)
Analyzing illumination in paintingsAnalyzing illumination in paintings
• Computer Vision, Image Analysis, and Master Art: Part 2 – D. Stork and M.K. Johnson– IEEE Multimedia, October-December 2006
Georges de la Tour
Christ in the Carpenter’s Studio (1645)
A historical detective story: Fenton CannonballsA historical detective story: Fenton Cannonballs
• Errol Morris, NY Times, 2007
Roger Fenton, “Valley of the Shadow of Death”, April 23, 1855
A historical detective story: Fenton CannonballsA historical detective story: Fenton Cannonballs
• Errol Morris, NY Times, 2007
Roger Fenton, “Valley of the Shadow of Death”, April 23, 1855
Fenton cannonballs: Which came first?Fenton cannonballs: Which came first?
“Off” “On”
HighHigh--resolution photosresolution photos
HighHigh--resolution photosresolution photos
Errol Morris on locationErrol Morris on location
Some observationsSome observations
• 19th century photographic emulsions are blue-sensitive and hence cannot record the sky – overcast, partially cloudy and sunny skies are all overexposed
• If the photos were taken about an hour apart, the change in illumination angle would be about 10 degrees – within the margin of error
• The interpretation of the change in illumination direction would depend on which way the photographer was facing
• The illumination in one of the images (“On”) appears to be more diffuse than in the other (“Off”)
The solution (hopefully)The solution (hopefully)
• Compare positions of distinctive rocks
“Off” “On”
The solution (hopefully)The solution (hopefully)
• Compare positions of distinctive rocks