03 0122 1000 3 storytelling
TRANSCRIPT
1
Automatic Storytelling in Automatic Storytelling in ComicsComics
Sheng-Wei (Kuan-Ta) ChenSheng-Wei (Kuan-Ta) ChenInst. of Information ScienceInst. of Information Science
Academia SinicaAcademia Sinica
2
Intrinsic Needs of Human Being!Intrinsic Needs of Human Being!
• Communication
• Storytelling– Today’s world is information-saturated– Computers are becoming a larger and larger part of
people’s lives (Intel 2005)– More and more people are documenting their lives
with digital images, video, music, and recordings (Imation 2003)
– As storage space gets cheaper, and people record more and more and more, they need some way to automate the categorization of all of these digital memories
3
• Recall, share and preserve (gaming) experience– Virtual community such as forums, blogs– Video, screenshots, comics or other art formats– Narrative, summary, story
Search “Lineage Story” or “Lineage Comics”…– Around 26,600 files in YouTube and 724,000 items in Google– http://www.lineage2.com/community/fanart.html– http://himewikia.blogspot.com/– …
4
• Example:
5
• Example:
6
Examples
7
More examples
8
Narrative• In short,
People Like Stories. Or, put more accurately,
People Remember Stories
• So we provide a summarization tool to aid storytelling of end-users
• But why Comics…
9
Comic• A storytelling medium
• Vocabulary of comics– Panel layout
• Border shape• Panel size• Bleed, splash page
– Speech balloon and sound effects– Motion lines
• Visual language– Draw attention to important events
Reference: http://scottmccloud.com/2-print/index.html (McCloud, Scott)
10
Comic Generation ToolsComic Generation Tools
• Automation support– The computer is charged with a greater role
in the creation of the comic
• Authoring support– Tools offer the most agency in creating panels
and layouts of any size and shape
Reference: http://dm.lcc.gatech.edu/~jalderman/comics/
11
Comic ChatComic Chat (1996)
Comic Chat visually represents conversations as sequences of
comic panels.
Automation support:• Changing posture and facial expression
of users’ avatars in chat room• Placing word balloons in natural
reading order• Zooming “camera” to frame the people
talking • Parsing text into panels procedurally.
Panels are all in the same size
Reference: http://kurlander.net/DJ/Projects/ComicChat/resources.html
12
• Microsoft Chat 2.5
Chat room:
Massage
Authoring support:Emotion Wheelallows users to choose the facial
expression and body language
Comic mode
13
• Microsoft Chat Character Editor
Character IconCharacter Contents• New Pose• New Pose
14
• Example:
15
Video MangaVideo Manga (1999-2003)
It provides a summary of video clips that allows users to quickly browse a longvideo in comic book format.
• Manga is named after a Japanese word for "comic book"
• A video is represented with different-sized keyframes that packed in a comic book format
• Video Manga allows users to quickly browse a long video
• The visual summaries support printing function
Reference:http://www.fxpal.com/?p=manga
16
Automation support:• A video is segmented based on the
color features of each frame• The segments are clustered
according to their similarities• We have introduced an importance
score to rank the segments• A segment is considered to be
important if it is long and rare• The keyframes are extracted from
highly ranked segments• The keyframes are sized according to
their scores• The frame-packing algorithm puts the
keyframes in a comic book format
Authoring support:• Handling captions to its frames
manually typed in by users
17
• Video Manga : Summary of staff meeting
18
Generating Comics From Generating Comics From 3D Interactive Computer 3D Interactive Computer Graphics Graphics (2006)(2006)
Automation support:• Recognizing scenes• Converting to visual depiction• Comic layout• Image processing
Reference:http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1626186
19
• Recognizing scenes– Time and space– Separating scenes when
• The interaction level is low• The location changes
• Converting to visual depiction– Transforming each scene
into a sequence of images depicting the main event
– Using idioms to depict interactions• Change of scene• Shooting• conversation
• Automation support
20
• Comic layout– B (Big):
peak interaction images and change-of-scene
– S (small): action-to-action pair or FP viewpoint shots
– F (fixed): images with speech balloon
– N (neutral): all others
• Image processing– Mean shift– Laplacian– K-means
21
• Example:
22
Comic Life Comic Life (2005)
Application for creating comics:
loading images into comic panels
Authoring support:• Simple and Easy-to-use interface
– Drag in your pictures, captions, sound effects (‘ka-blam!') and speech balloons and it’s done!
– Layout templates– Filters like hand-drawn, painted
and night vision
• More freedom to create comic but still some restrictions
Reference:http://plasq.com/comiclife-win
23
• Comic Life 1.3
24
• Example:
25
Manga StudioManga StudioApplication for creating comics: drawing, laying out comic and styling comics
Authoring support:• High versatility and flexibility to
create comics – A large collection of special effects – Sketch, ink, and color artworks
without the need for any other graphics applications
• Interface similar to Photoshop
Reference:http://my.smithmicro.com/mac/manga/index.html
26
OurOur ProposalProposal1. Automation support
• Frame selection• Layout computing
2. Authoring support• Comic layout• Graphic Effects• Internet support
3. User interface
27
Comic GeneComic Gene (2009)
It automatically summarize players’ interactions (logs and screenshots)
in virtual world into comics.
Automation support:• Recording log and shots (Addon)• Frame selection• Comic layout computing• Rendering comics
Authoring support:• Editing log files and screenshots• Comic layout
– Option setting– Changing size and position of the shots
Reference:http://mmnet.iis.sinica.edu.tw/publication_detail.html?key=chan09_comics
28…How to improve!
• Prototype
29
1. Automation Support1. Automation Support
• Automatically summarize logs and frames into comics– Log: events– Frame: video, photos, screenshots and so on
• Definition of important events– Frequency, weight
– Significance score
30
ChallengesChallenges• Technical side
– Choose significant frames using machine learning– Image understanding– Special effect placement (font size, color, location)
• Chat balloons• Onomatopoeic sounds
• Design side– Comic-like style
• Page layout– Essential factors of good comic books
• Shooting• Zoom in/out• Level of detail • Timing control
31
• Frame selection– How many images does a comic strip need?– Is the criterion of significance score enough?
• Layout computing– Algorithm– Detection
32
2. Authoring Support2. Authoring Support
• Page properties– How many images in a comic/page/row/… ?
• Important event/frames
• Comic layout– Panel layout
• Border shape, bleed• Panel size
– Speech balloon– Sound effects
33
• Graphic effects– Templates– Cartoon-like design
• Cartoon-like filter• Motion line
• Internet support– Uploading comics to individual album or blogUploading comics to individual album or blog
34
3. User Interface Design3. User Interface Design
• User-friendly Features– Navigation bar– Easy-to-use
• Click, wheel• Drag-n-drop
35
UI Example: VideolyzerUI Example: Videolyzer
Navigation barNavigation bar
36
UI Example: Comic LifeUI Example: Comic Life
Drag-n-dropDrag-n-drop
37
Current PrototypeCurrent Prototype
38
Sheng-Wei (Kuan-Ta) ChenSheng-Wei (Kuan-Ta) Chen
[email protected]@iis.sinica.edu.twhttp://www.iis.sinica.edu.tw/~swchttp://www.iis.sinica.edu.tw/~swc
Contact infoContact info
39
Thank youThank you