lyu 0602 automatic photohunt generation automatic photohunt generation shum hei lung to wan chi...

LYU 0602 Automatic PhotoHunt Generation

Automatic Automatic PhotoHunt PhotoHunt GenerationGenerationShum Hei Lung To Wan Chi

Supervisor: Prof. Michael R. Lyu


• Background

• Objectives

• System Implementation

• Segmentation Module

• Elimination Module

• Other Module

• Web-based PhotoHunt game

• Conclusion

OverviewOverview


BackgroundBackground

• PhotoHunt is …– a Spot-the-difference game

– Classic yet evergreen

– Popular in electronic game centers all over the world

However…It is limited by man power


Basic techniques to edit image Basic techniques to edit image • Elimination

• Color modification

• Cloning

• Transformation


ObjectivesObjectives

• To implement the PhotoHunt Generation Engine – Focus on eliminating object

• Develop an engine to support real-time image generation for PhotoHunt game


System ImplementationSystem Implementation

Image Generation

Engine

Edited Photo

Game Engine


Image Generation EngineImage Generation Engine

• To generate an image for PhotoHunt game– Effects that may be applied:

• Elimination

• Color Change

• Duplication

Definition of well generated image: • The effects should be

“NOT OBVIOUS YET DISCOVERABLE”


Image Generation Engine – The ProcessImage Generation Engine – The Process

Segmentation Module

Modification Modules

Smooth Image

GAME Game Engine


SegmentationSegmentationModuleModule


Segmentation ModuleSegmentation Module• To detect and extract segment from the input image

• Three Phases:1. Segmentation

2. Constraint CheckingArea of Input Image/10000<Area of Segment < Area of Input Image/500

3. Area of Segment Reference image building

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

1

1


Segmentation AlgorithmSegmentation Algorithm

The Segmentation algorithms we have examined:

• Pixel-based Algorithm

• Edge Detection Algorithm

• Region-based Algorithm


Gaussian Pyramid

Pyramid SegmentationPyramid Segmentation

• The Pyramid Segmentation Algorithm• Step 1 Generation of the Gaussian pyramid

• Step 2 Segmentation by pyramid-linking and averaging linked-pixel


Step 1- Generation of the Gaussian pyramidStep 1- Generation of the Gaussian pyramid

)2()2()2()1()2()0()2()1()2()2()( 11111 migwmigwmigwmigwmigwig llllll

W(-2)

W(-1)

W(0) W(1) W(2)

5 5 8 7 6 5 4

+

g(i)Father node

Level l-1

Level l


Step 1- Generation of the Gaussian pyramidStep 1- Generation of the Gaussian pyramid

m n

ll njmignmwjig )2,2(),(),( 1


Step 2 Segmentation by pyramid-linking Step 2 Segmentation by pyramid-linking and averaging linked-pixel and averaging linked-pixel

• ct(i,j,l) : local characteristics

• ft(i,j,l) : The potential father node

• st(i,j,l) : segment property

• areat(i,j,l) : area of the segment


1. Set ft(i,j,l)

2. Initializeareat(i,j,0) =1

ct(i,j,l) =c0(i,j,0)

3. for l = 0 to L

st(i,j,L) = ct(i,j,L)

for l = L downto 0

st(i,j,l) = ct(i”,j”,l+1)

),,(

)1,','()1,','((),,(

ljiarea

ljicljiarealjic

t

tt

)1,,(),,( ljiarealjiarea tt



22 4

1215 12

13

17

2117 17

17 17 17 17 17 4 4 22



Segmentation ResultSegmentation Result


EliminationEliminationModuleModule


Elimination AlgorithmElimination Algorithm

• Direct Copy Algorithm

• Horizontal Gradient Algorithm

• Nearest Boundary Algorithm

• Enhanced Nearest Boundary Algorithm


Direct Copy AlgorithmDirect Copy Algorithm• Directly copy the upper pixel to current pixel

iSyxCforyxCyxC ),()1,(),(

Where Si is segmented region of the image for i = 0,1,2,……


Direct Copy AlgorithmDirect Copy Algorithm


Horizontal Gradient AlgorithmHorizontal Gradient Algorithm

• Determine color line by line

• Smoothing the changes of color from one side to another

)()()()()(PQ

NP

PQ

NQ

d

dQC

d

dPCNC

Where C(M) is the color verctor of pixel M and dXY is the distance between X and Y


Horizontal Gradient AlgorithmHorizontal Gradient Algorithm


Nearest Boundary AlgorithmNearest Boundary Algorithm

min

min

min

min

)(

)(

)(

)(

)(

ddSC

ddRC

ddQC

ddPC

NC

SN

RN

QN

PN

Where dmin = min(dPN, dQN, dRN, dSN)

• Replace the color by the color of nearest boundary


Nearest Boundary AlgorithmNearest Boundary Algorithm


Enhanced Nearest Boundary Algorithm Enhanced Nearest Boundary Algorithm

• Improved from Nearest Boundary Algo.

),(),()(

),(),()(

),(),()(

),(),()(

)(

min

min

min

min

QPSSRSandddifSC

QPSSRSandddifRC

SRSQPSandddifQC

SRSQPSandddifPC

NC

SN

RN

QN

PN

Where dmin = min(dPN, dQN, dRN, dSN) S(A,B) is Euclidean distance between A and B


Enhanced Nearest Boundary AlgorithmEnhanced Nearest Boundary Algorithm


Others Others ModulesModules


Color Change ModuleColor Change Module

Segmentdominate color

Operation apply

Red Swap red and blue channel

Green Swap red and green channel

Blue Swap green and blue channel

Dominate color segment: % of Pixel of dominate color> 75%Dominate color pixel: DChannel >180 and otherChannel<50


Smooth ImageSmooth Image

• To reduce noise and distortion

• To make the image more realistic

• Gaussian Filter (Neighbor size=3, sigma=1)


Semi-Automatic Semi-Automatic PhotoHunt GenerationPhotoHunt Generation


Semi-Automatic PhotoHunt GenerationSemi-Automatic PhotoHunt Generation


Game EngineGame Engine


Game EngineGame Engine

PHP S cript

U s e rs

Au tom at i cPh oto

Ge n e rati onEn gin e

P h o to S c o re

Gam eplayi n g

a ppl i cationwri tte n in

Flash

G a m e E n g i n e fo r W eb A p p lica tio n

im a g e s

o rig i n a l p h o to

w i th g en e ra ted p h o to

g e n e r at io n r e q u e s t

r e su l t in g p h o t o

p la y ing v iew

gen e

rat e

d p h

oto

s av e

d ph

o to s

p la yin g sc ore

p la y in g s co re

p la y in g sco re

p la y in g req u e st

u p lo a d ed im a g e s


Game Engine – Front PageGame Engine – Front Page


ConclusionConclusion


LimitationLimitation

• Only elimination and limited color change effect supported

• No artificial intelligence control to the Applied effect

• Modified area found noise and distortion

• Not all user input can be segmented


Looking Forward…Looking Forward…

• Improve the segmentation algorithm

• Implement the Duplication and Transformation module

• Support numbers of object recognitions

• More Features on the Web-based game

• Ultimately, to achieve the complete automatic PhotoHunt generation


Q & AQ & A


The EndThe End

Thanks for your kind attention.


A B

C D

A1 Example of Pyramid SegmentationA1 Example of Pyramid Segmentation

21 25 2 0

22 15 1 1

13 11 2 3

13 12 12 2

c0(0)= 21 c0(1)= 25 c0(2)= 2 c0(3)= 0 c0(4)= 22 c0(5)= 15 c0(6)= 1 c0(7)= 1 c0(8)= 13 c0(9)= 11 c0(10)= 4 c0(11)= 5 c0(12)= 13 c0(13)= 12 c0(14)= 12 c0(15)= 2

c0(A)= 21.2 c0(B)= 2.2 c0(C)= 14.8 c0(D)=3.2

f0(0)= A f0(1)= A f0(2)= B f0(3)= B f0(4)= A f0(5)= C f0(6)= B f0(7)= B f0(8)= C f0(9)= C f0(10)= B f0(11)= D f0(12)= C f0(13)= C f0(14)= C f0(15)= D

a0(A)= 3 a0(B)=5 a0(C)= 6 a0(D)= 2

c0(A)= 22.6 c0(B)=1.2 c0(C)= 12.6 c0(D)= 2.5


A3 Pixel-based Segmentation -ThreshA3 Pixel-based Segmentation -Thresholding olding

0

10000

20000

30000

40000

50000

60000

70000

80000

1 21 41 61 81 101 121 141 161 181 201 221 241

Fig 3.2b Histogram of data generated by ThresSeg.cpp

Intensity (0-28)

Frequency

Threshold

otherwise

Tyxcyxc

1

),(0),('

10 2 DT

2/)( 1,1, ibioi mmT


A4 Edge Detection Segmentation-A4 Edge Detection Segmentation-Canny Edge DetectionCanny Edge Detection

• Phase1 - Gaussian convolution S(x,y)=smoothed image

• Phase 2 - Edge strength and direction

|M|=|H|+|V|

• Phase 3 - Non-maximum suppression- Non-maximum suppression trace along the edge in the edge direction and suppress any pixel value that is

not considered to be an edge to zero.

• Phase 4 - Hysteresis- continuing the tracking until the Threshold falls behind the lower second threshold

2/)),1()1,1(),()1,((),( yxSyxSyxSyxSyxH 2/))1,()1,1(),(),1((),( yxSyxSyxSyxSyxV

otherwiseyxHyxV

yxandVyxH

yxandVyxH

yx

)),(/),((tan

0),(0),(2

0),(0),(0

),(

1

),(*),(),( yxcyxGyxS


A5 Gaussian Filter CharacteristicsA5 Gaussian Filter Characteristics1. Separable (when it is in 2 dimension0)

(x,y) = (x) (y)

2. Normalized

3. Symmetric

4. Equal contribution

Each pixel should have equal contribution to the father pixel in the upper level.

1)(^

m

mw

)()(^^

mwmw

lyu 0602 automatic photohunt generation automatic photohunt generation shum hei lung to wan chi...

Documents

photohunt game slide

lyu slide

photohunt game effects

realtime image generation

discoverable slide

color segment

generated image

color pixel