playing detective with digital images - welcome to the …lazebnik/research/fall08/lec11...a history...
TRANSCRIPT
![Page 1: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/1.jpg)
Playing detective with digital imagesPlaying detective with digital images
![Page 2: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/2.jpg)
Two issuesTwo issues
• Can we fool casual observers?– How realistic is photorealistic?
• What about experts with lots of time and resources?– Digital image forensics
![Page 3: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/3.jpg)
A history of photo manipulationA history of photo manipulation
• 1860: the portrait of Lincoln is a composite of Lincoln's head and John Calhoun's body
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 4: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/4.jpg)
A history of photo manipulationA history of photo manipulation
• 1917: “Cottingley fairies”: five pictures taken by Elsie Wright (16) and Frances Griffiths (10) in England
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 5: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/5.jpg)
A history of photo manipulationA history of photo manipulation
• 1917: “Cottingley fairies”: five pictures taken by Elsie Wright (16) and Frances Griffiths (10) in England
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 6: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/6.jpg)
A history of photo manipulationA history of photo manipulation
• 1917: “Cottingley fairies”: five pictures taken by Elsie Wright (16) and Frances Griffiths (10) in England
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 7: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/7.jpg)
A history of photo manipulationA history of photo manipulation
• 1930s: Stalin had disgraced comrades airbrushed out of his pictures
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
http://www.newseum.org/berlinwall/commissar_vanishes/index.htm
![Page 8: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/8.jpg)
A history of photo manipulationA history of photo manipulation
• 1936: same story with Mao
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 9: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/9.jpg)
A history of photo manipulationA history of photo manipulation• May 1970: This Pulitzer Prize winning photo by John Filo shows Mary
Ann Vecchio screaming as she kneels over the body of student Jeffrey Miller at Kent State University, where National Guardsmen had fired into a crowd of demonstrators, killing four and wounding nine. When this photo was published in LIFE Magazine, the fence post directly behind Vecchiowas removed.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 10: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/10.jpg)
A history of photo manipulationA history of photo manipulation
• 1989: Oprah’s head on somebody else’s body
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 11: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/11.jpg)
A history of photo manipulationA history of photo manipulation
• 1994: O.J. Simpson’s mug shot modified to appear more menacing
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 12: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/12.jpg)
A history of photo manipulationA history of photo manipulation
• 2000: black student’s face inserted into UW brochure
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 13: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/13.jpg)
A history of photo manipulationA history of photo manipulation
• 2003: Beatles album cover modified to remove cigarette from McCartney’s hand
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 14: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/14.jpg)
A history of photo manipulationA history of photo manipulation• 2003: This digital composite of a British soldier in Basra, gesturing to Iraqi
civilians urging them to seek cover, appeared on the front page of the Los Angeles Times shortly after the U.S. led invasion of Iraq. Brian Walski, a staff photographer for the Los Angeles Times and a 30-year veteran of the news business, was fired after his editors discovered that he had combined two of his photographs to "improve" the composition.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 15: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/15.jpg)
A history of photo manipulationA history of photo manipulation• 2003: This digital composite of a British soldier in Basra, gesturing to Iraqi
civilians urging them to seek cover, appeared on the front page of the Los Angeles Times shortly after the U.S. led invasion of Iraq. Brian Walski, a staff photographer for the Los Angeles Times and a 30-year veteran of the news business, was fired after his editors discovered that he had combined two of his photographs to "improve" the composition.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 16: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/16.jpg)
A history of photo manipulationA history of photo manipulation
• 2004: Composite of John Kerry and Jane Fonda
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 17: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/17.jpg)
A history of photo manipulationA history of photo manipulation
• 2004: Composite of John Kerry and Jane Fonda
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 18: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/18.jpg)
A history of photo manipulationA history of photo manipulation• 2006: This photograph by Adnan Hajj, a Lebanese photographer, showed
thick black smoke rising above buildings in the Lebanese capital after an Israeli air raid. The Reuters news agency initially published this photograph on their web site and then withdrew it when it became evident that the original image had been manipulated.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 19: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/19.jpg)
A history of photo manipulationA history of photo manipulation
• 2007: French Magazine Paris Match altered this photograph of French President Nicolas Sarkozy
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 20: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/20.jpg)
A history of photo manipulationA history of photo manipulation• 2008: This photograph, by Liu Weiqiang of the Daqing Evening News,
won an award for "one of the ten most impressive news photos of 2006". This photograph was recently revealed to be a composite of two separate photographs: the antelopes and the train.
http://www.cs.dartmouth.edu/farid/research/digitaltampering/
![Page 21: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/21.jpg)
Detecting digital tampering: cloningDetecting digital tampering: cloning
• Exposing Digital Forgeries by Detecting Duplicated Image Regions– A.C. Popescu and H. Farid– Technical Report, TR2004-515, Dartmouth College, Computer Science
![Page 22: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/22.jpg)
Detecting digital tampering: retouchingDetecting digital tampering: retouching
• Exposing Digital Forgeries in Color Filter Array Interpolated Images – A.C. Popescu and H. Farid– IEEE Transactions on Signal Processing, 53(10):3948-3959, 2005
![Page 23: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/23.jpg)
Detecting digital tampering: LightingDetecting digital tampering: Lighting
• Exposing Digital Forgeries by Detecting Inconsistencies in Lighting – M.K. Johnson and H. Farid– ACM Multimedia and Security Workshop, New York, NY, 2005
![Page 24: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/24.jpg)
Detecting digital tampering: LightingDetecting digital tampering: Lighting
• Exposing Digital Forgeries by Detecting Inconsistencies in Lighting – M.K. Johnson and H. Farid– ACM Multimedia and Security Workshop, New York, NY, 2005
![Page 25: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/25.jpg)
Detecting inconsistencies in lightingDetecting inconsistencies in lightingI(x,y) = N(x,y)·L + A
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
),(
),(),(
1),(),(),(
1),(),(),(1),(),(),(
22
11
222222
111111
nn
z
y
x
nnznnynnx
zyx
zyx
yxI
yxIyxI
ALLL
yxNyxNyxN
yxNyxNyxNyxNyxNyxN
MMMMM
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
),(
),(),(
1),(),(
1),(),(1),(),(
22
11
2222
1111
nn
y
x
nnynnx
yx
yx
yxI
yxIyxI
A
LL
yxNyxN
yxNyxNyxNyxN
MMMM
I(x,y) = N(x,y) ·L(x,y) + A
Full 3D case:
For points on the occluding contour:
P. Nillius and J.-O. Eklundh, “Automatic estimation of the projected light source direction,” CVPR 2001
![Page 26: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/26.jpg)
Detecting inconsistencies in lightingDetecting inconsistencies in lighting
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
),(
),(),(
1),(),(
1),(),(1),(),(
22
11
2222
1111
nn
y
x
nnynnx
yx
yx
yxI
yxIyxI
A
LL
yxNyxN
yxNyxNyxNyxN
MMMM
![Page 27: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/27.jpg)
Detecting inconsistencies in lightingDetecting inconsistencies in lighting
Fake photo
Real photo
![Page 28: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/28.jpg)
Lighting: Lighting: SpecularSpecular highlights in the eyehighlights in the eye
M.K. Johnson and H. Farid, “Exposing Digital Forgeries Through SpecularHighlights on the Eye,” 9th International Workshop on Information Hiding,
2007
![Page 29: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/29.jpg)
Analyzing illumination in paintingsAnalyzing illumination in paintings• Inferring Illumination Direction Estimated from Disparate Sources in
Paintings: An Investigation into Jan Vermeer's Girl With a Pearl Earring– M.K. Johnson, D. Stork, S. Biswas and Y. Furuichi– SPIE Symposium on Electronic Imaging, San Jose, CA, 2008
![Page 30: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/30.jpg)
Analyzing illumination in paintingsAnalyzing illumination in paintings• Inferring Illumination Direction Estimated from Disparate Sources in
Paintings: An Investigation into Jan Vermeer's Girl With a Pearl Earring– M.K. Johnson, D. Stork, S. Biswas and Y. Furuichi– SPIE Symposium on Electronic Imaging, San Jose, CA, 2008
![Page 31: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/31.jpg)
Analyzing illumination in paintingsAnalyzing illumination in paintings
• Computer Vision, Image Analysis, and Master Art: Part 2 – D. Stork and M.K. Johnson– IEEE Multimedia, October-December 2006
Georges de la Tour
Christ in the Carpenter’s Studio (1645)
![Page 32: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/32.jpg)
Analyzing illumination in paintingsAnalyzing illumination in paintings
• Computer Vision, Image Analysis, and Master Art: Part 2 – D. Stork and M.K. Johnson– IEEE Multimedia, October-December 2006
Georges de la Tour
Christ in the Carpenter’s Studio (1645)
![Page 33: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/33.jpg)
A historical detective story: Fenton CannonballsA historical detective story: Fenton Cannonballs
• Errol Morris, NY Times, 2007
Roger Fenton, “Valley of the Shadow of Death”, April 23, 1855
![Page 34: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/34.jpg)
A historical detective story: Fenton CannonballsA historical detective story: Fenton Cannonballs
• Errol Morris, NY Times, 2007
Roger Fenton, “Valley of the Shadow of Death”, April 23, 1855
![Page 35: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/35.jpg)
Fenton cannonballs: Which came first?Fenton cannonballs: Which came first?
“Off” “On”
![Page 36: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/36.jpg)
HighHigh--resolution photosresolution photos
![Page 37: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/37.jpg)
HighHigh--resolution photosresolution photos
![Page 38: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/38.jpg)
Errol Morris on locationErrol Morris on location
![Page 39: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/39.jpg)
Some observationsSome observations
• 19th century photographic emulsions are blue-sensitive and hence cannot record the sky – overcast, partially cloudy and sunny skies are all overexposed
• If the photos were taken about an hour apart, the change in illumination angle would be about 10 degrees – within the margin of error
• The interpretation of the change in illumination direction would depend on which way the photographer was facing
• The illumination in one of the images (“On”) appears to be more diffuse than in the other (“Off”)
![Page 40: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/40.jpg)
The solution (hopefully)The solution (hopefully)
• Compare positions of distinctive rocks
“Off” “On”
![Page 41: Playing detective with digital images - Welcome to the …lazebnik/research/fall08/lec11...A history of photo manipulation • May 1970: This Pulitzer Prize winning photo by John Filo](https://reader034.vdocuments.mx/reader034/viewer/2022051800/5acfba487f8b9a4e7a8d0356/html5/thumbnails/41.jpg)
The solution (hopefully)The solution (hopefully)
• Compare positions of distinctive rocks