aea1 04 transparency excerpt

In this lesson, we will be focusing on different waysto create transparency. One of the keys to creatingan interesting composite of multiple images is to

make portions of those images transparent, so thatyou can see other images behind or through them.This is one of the main techniques that set motiongraphic design and visual effects compositing apartfrom video editing.

Masks, Mattes, and StencilsIn previous lessons, we introduced two basic forms of man-aging transparency: altering the Opacity property of a layer,and taking advantage of a layer’s built-in alpha channel thatdefines which parts of it are supposed to be transparent. Inthis lesson, you will go beyond these by adding your owntransparent areas to an image using masks, track mattes, and stencils.

Masking is a way to cut out sections of a specific layer. Atits default, a mask shape says “I want to see only the areainside this shape; make the area outside transparent.” Youcan draw your own shapes and paths directly on the layer, orcopy paths from the Adobe companion programs Photoshopand Illustrator and paste them onto an After Effects layer tocreate a mask shape. You may have multiple masks perlayer, and combine them in a variety of ways such asadding together their shapes or using only the area wherethey overlap. You can also control the opacity of a mask(making its cutout semitransparent), define its feather (howsoft its edges are), and invert it so areas inside of its shape –rather than outside – are transparent. As an added bonus,some effects can use mask shapes to draw lines, and you

Lesson 482

Excerpt from After Effects Apprentice. Copyright © 2007 Trish and Chris Meyer.

No part of this publication may be reproduced or distributed in any form or by any means.

Focal Press, ISBN 978-0-240-80938-0

Creating TransparencyUsing masks, mattes, and stencils to cut out portions of a layer.

� In This Lesson84 Rectangular Mask tool84 Free Transform Points85 masking in the Layer panel85 Mask Feather86 animating a Mask Shape87 creating a vignette; Mask Expansion88 masking with the Pen tool88 creating and editing Bezier masks89 effecting a masked area90 controlling Mask Shape interpolation91 using effects with the mask path92 Mask Modes and multiple masks93 Mask Opacity94 creating and editing RotoBezier masks95 Audio Spectrum effect96 Alpha Track Matte97 nesting a track matte composite99 effects and track mattes

100 Luma Track Matte101 animating matte layers102 Stencil Alpha and Stencil Luma

� Getting StartedCopy the Lesson 04-Transparency folder from this book’s disconto your hard drive. Open the file Lesson_04.aep inside of itto work through the exercises in this lesson.

can have text follow a mask (which we’ll show in the next lesson). You can evencreate a mask shape that is just a line or path rather than an enclosed area; theseare particularly handy when used in conjunction with effects or text.

Track mattes, by contrast, involve the combination of two layers. One layer –the matte – is used just to define transparency; you don’t directly see the imageit contains. This matte is then used to decide what portions of the layer immedi-ately below it are visible. There are two types of mattes: alpha mattes, which usethe alpha channel of the matte layer to define the transparency of the secondlayer, and luminance (or luma) mattes, which use the luminance – grayscalevalues, or brightness – of the matte layer to define the transparency of thesecond layer.

Stencils take the concept of track mattes further: Rather than define the trans-parency of the next layer below, a stencil layer defines the transparency of allthe layers below, cutting a hole through the entire layer stack. Just like mattes,stencils can also be based around alpha channels or luminance.

It’s easy to confuse these methods of creating transparency, so just remember:Masks involve one layer, mattes involve two layers (the fill and the matte), andstencils are basically mattes that can affect multiple layers below them. In thislesson, you’ll learn how to put all three to work.

Masks take a layer (A), and allow you to cutout portions to make them transparent oropaque (B). Mattes use either the alphachannel or luminance of one layer (C) todefine the transparency of a second layer (D) to create a final composite image (E).Images courtesy Wildscaping.com,Artbeats/Virtual Insanity.

Lesson 483




� future visionKeyingKeying – the art of making certain colors in a layer transparent – is another way ofcreating transparency. Unlike the techniquesdemonstrated in this lesson, keying relies onan effect to create transparency. Keying iscovered in Lesson 10.

A B

C D E

� tipHelp with ShortcutsFor a full list of pen tool and mask editingshortcuts, see Help > Keyboard Shortcuts (or Help > After Effects Help, and searchfor “keys for using masks”).

MaskingThere are several ways to create masks, and several ways to use them. In this firstexercise, we will focus on using the Mask tool to create windows on footage, toreveal a title, and to create a vignette.

1 We assume you’ve already opened the project Lesson_04.aep. If you’d like topreview what you’ll be creating in this exercise, open its Comps_Finished folderand RAM Preview the comp 01-Masking_final. Close it when you’re done bypressing Cw on Mac (Lw on Windows).

2 In the Project panel, open the Comps folder and double-click the comp 01-Masking*starter to open it.

Select the foreground layer Cityscape.mov. Then select the Rectangular Masktool from the toolbar along the top of the application window; its shortcut is q.Click and drag in the Comp panel to surround a few of the major buildings onthe right, plus some of the freeway. As you draw this outline, the areas ofCityscape.mov beyond the outline will disappear, revealing the background.

After you release the mouse, withCityscape.mov still selected, press mto reveal the new Mask and its MaskShape property in the Timeline panel(see right).

In the Comp panel, you should seea yellow outline around the maskshape. If you don’t, click the ToggleView Masks button along the bottomof the Comp panel to turn it on.

3 Press v for the Selection tool. Then double-click on the yellow mask outline;this will enable its Free Transform Points, which means you can edit its shape.Note the eight small boxes (“handles”) around its shape: As you hover yourcursor over these, the cursor will change to icons that indicate you can click anddrag to resize or rotate the shape. Dragging anywhere else moves the entire shape.

Tweak the size and position of the mask to taste, then hit E to accept youredits and turn Free Transform off.

4 Next, you want to reveal a different section of Cityscape.mov – but you can’tsee the rest of the image as it’s already been masked out. No problem: You canalso create masks in the Layer panel where you can view the entire image.

Double-click Cityscape.mov to open its Layer panel. If it opens docked intothe same frame as the Comp panel, right-click on its tab, select Undock Panel,then drag it side by side with the Comp panel.

Lesson 484




� tipConstrained MasksPress S while dragging a mask shapeto create a perfect square or circle; addC on Mac (L on Windows) after youstart dragging to draw the mask outwardfrom a center point.

3 Double-click the mask outline to enableits Free Transform Points; the shortcut isCt (Lt). Here we are grabbing thetop point to change the height of the shape.

2 Select the layer to mask, then select the Mask tool (above). Click and drag in the Comp panel to define the mask shapeyou want (below). Foreground courtesyArtbeats/Timelapse Cityscapes; background12 Inch Design ProductionBlox Unit 02.

Along the bottom of the Layer panel is a Viewpopup; check that it is set to Masks. Uncheck theRender button (to the right of View): This will revealthe entire layer without the effect of masking.

Select the Rectangular Mask tool again. This time,hold down the S key while dragging out a newmask shape along the left side of the layer; this willconstrain it to draw a perfect square. (Actually, it willlook slightly wide, as this layer has non-square pixels –refer to the Tech Corner at the end of the previouslesson.) The result of your masking will instantlyappear in the Comp panel as you draw; when you’redone, Mask 2 will appear in the Timeline panel.

5 After you are done drawing a mask shape in theLayer panel, you will see the individual mask points or “vertices” that define it; by default, all vertices areselected. Press v to return to the Selection tool. While all points are selected, you can drag any onepoint to move the entire mask, or use the arrow keys to nudge it by one screen pixel. You can S+click ordrag a marquee around individual vertices to select ordeselect them, and edit the position of these directly.You can also double-click the mask to use the FreeTransform Points as you did in the Comp panel.

Tweak your second mask shape to taste, and closethe Layer panel when you’re done.

6 By default, masks have sharp, crisp edges. However,it is possible to soften these. With Cityscape.mov stillselected, type Sf to reveal the Mask Feather para-meter in addition to Mask Shape. Turn the Toggle ViewMasks button (at the bottom of the Comp panel) off tomore clearly see the outlines, and scrub Mask Featherto soften the masks. Set Feather for both masks to taste.

7 After Effects calculates effects for a layer aftermasks have been taken into account. Apply Effect >Perspective > Drop Shadow to Cityscape.mov; noticethat both masks get the same shadow. Slightly increasethe Distance and Softness of the shadow to taste, andsave your project.

7 Effects are appliedafter mask shapes. That means the singleDrop Shadow effectapplied to this layeraffects both masks.

Lesson 485




4 Double-click a layer to open its Layer panel. Set the View popup toMasks and turn off the Render switch. You can then draw and edit masksundistracted by other layers or the results of previous mask shapes.

6 Scrub the MaskFeather parameter tosoften the edges.

Animating a Mask Shape (Mask Path)Masks do not need to be fixed windows onto another layer– their shapes can also animate. This is particularly usefulfor revealing the content of layers.

8 Press H to make sure you are at the start of the composition. Bring the Project panel forward, and open the Sources folder. Select the footage item Bring on theNight.ai, and type C/ (L/) to add it to your comp.

9 This black title is hard to read against your video. Withit selected, add Effect > Generate > Fill. Click on its Colorswatch, and change it to white.

That’s better, but it still looks a bit flat on top of yourcomposite. Add Effect > Perspective > Drop Shadow, andadjust the Distance and Softness to taste.

10 As we mentioned in a previous lesson, sometimes it’seasier to start where you want to end up, and work back-ward. With Bring on the Night.ai selected, double-click onthe Rectangular Mask tool. This will add a mask to it that is the same size as the layer – a perfect ending point for areveal. Enable the Toggle View Masks switch to see itsoutline in the Comp panel.

11 Type m to reveal the Mask Shape parameter in theTimeline panel. Move the current time marker to 01:00,and click on the stopwatch next to Mask Shape to enablekeyframing and create the first keyframe.

Hit H to return to 00:00. Then double-click on themask outline in the Comp panel to bring up the FreeTransform Points; the shortcut is Ct (Lt). Drag thehandle on the right edge all the way to the left until thetitle is completely hidden. A new Mask Shape keyframe will be created automatically at 00:00.

To soften the leading edge of your wipe, increase theMask Feather for the title’s mask. You need to increase onlythe first value, which is in the X (horizontal) direction.

12 Press 0 on the numeric keypad to RAM Preview youranimation. Add a dramatic pause by moving Bring on theNight.ai’s layer bar to start later in the timeline, such as at01:00 – its keyframes will move with it. Save your project!

Lesson 486

9 Add a title, use the Fill effect to color it white, and add the Drop Shadow effect.




11 To wipe on the title, animate a mask to reveal it. Here we aredragging one of its Free Transform Points to edit the mask shape.

Creating a VignetteMasks can also be used to create subtle effects, such asadding a “vignette” where the edges of an image aredarkened, focusing the viewer more toward the center of the screen.

13 Press H to return to 00:00. Create a Layer > New > Solid; the shortcut is Cy (Ly). In the Solid Settings dialog that opens, click the Make CompSize button, and set the Color to be something complementary to your scenesuch as a dark purple or blue. Give it a name that makes sense such as “vignette”and click OK.

14 Click and hold on the Mask tool; a popup will appear. You want a roundedmask for this task, so select the Elliptical Mask Tool option, and release the mouse.(You can also toggle between the two by pressing q.) Then, with the vignettelayer selected, double-click this tool in the toolbar to create a full-size oval mask.

15 Now you have a big dark oval obscuring the center of your image – probablynot what you had in mind. But that’s easy to change.

With vignette still selected, type mm (two ms in quick succession) to revealall of the mask properties, and make the following edits:

• Click on the checkbox next to Inverted: Now the mask solid will be transparentinside and opaque outside.

• Scrub the Mask Feather until you get a nice, soft falloff around the edges.

• Scrub the Mask Expansion property: This offsets a mask to draw inside oroutside of the original Mask Shape. (To better see its effect, temporarily turnMask Feather down to zero.) Increase Mask Expansion to push the vignette offinto the corners.

• If the corners are too dark, reduce the Mask Opacity value (or use the regularTransform > Opacity property). Then feel free to balance the Mask Feather,Opacity, and Expansions values off of each other to get the look you want.

14 Select the Elliptical Mask Tool (inset),then double-click in the toolbar to create afull-size oval mask on your new solid (above).

Lesson 487

15 To finish off the vignette, invert themask (A), increase its feather (B), increase its expansion (C), and reduce its opacity (D). Our version is in Comps_Finished > 01-Masking_final.




A B C D

� tipNumeric Shapes Clicking the word Shape… in the switchescolumn of the Timeline panel opens theMask Shape dialog, where you can entervalues numerically.

Congratulations – you’ve created a tasteful little animation, plus now have ahandle on creating masks. Go ahead and tweak your animation to taste. If youlike, compare your results with ours – see comp 01-Masking_final in the Projectpanel’s Comps_Finished folder.

Masking with the Pen ToolNext up is practicing using the Pen tool to create more detailed mask shapes. It has two basic modes: One that creates common Bezier curves with handles(which we’ll do first), and a RotoBezier mode which automatically determinesthe curve of the path. As with the Mask tool, you can create shapes in either theComp or Layer panels, plus keyframe their shapes.

Drawing a Path1 In the Project panel for this lesson’s file, locate 02a-Bezier*starter in theComps folder and double-click it to open it.

2 Select the Pen tool (the shortcut is g). When you do, a RotoBezier option will appear on the right end of the Toolbar; turn it off for now.

3 Drawing with the Pen tool is similar to drawing paths in programs such asPhotoshop and Illustrator, although the shortcuts can differ slightly. To practice,start by just making a random shape (don’t worry; you can always delete anymess you create):

• To create straight line segments, click with the mouse to create a series of points.

• To create curved segments, click and drag to pull out Bezier handles for thepoint you are creating.

While drawing with the Pen tool, you can edit the points you’ve just placed:

• To reposition any point or handle before you’re finished drawing a mask, temporarily toggle to the Selection tool by pressing the C (L) key.

• To toggle between a sharp corner point and a smooth point, press OS

(AS) and click on a point. The cursor will automatically change to theConvert Vertex Point cursor, which looks like an upside-down V. To toggle thelast point you drew, just click on it without holding down these extra keys.

• To break the continuous handles of a smooth point, click and drag onehandle; repeat to switch back to a smooth point.

• Click any point other than one youjust drew to delete it (the cursor willautomatically change to the DeleteVertex tool). To delete the most recentpoint, Undo.

• To add a new point between pointsyou’ve already drawn, click on the linejoining them – the cursor will auto-matically change to the Add Vertextool while doing so.

3 The Convert Vertex Point tool is used totoggle between smooth and hard corners,and to break or unbreak the continuity ofthe Bezier handles. Image courtesyWildscaping.com.

2 Select the Pen tool, and turn off theRotoBezier option for now.

Lesson 488




� tipSelect All PointsTo select all the points on a mask, with theSelection tool active hold down O (A)and select any one of the mask’s points.

3 continued To delete a vertex, place the cursor over a point; it will automatically change tothe Delete Vertex tool (left). To add a vertex, place the cursor between points (right).

• To pick up drawing a mask shape whereyou left off, make sure only the last pointis selected, then continue creating pointswith the Pen tool.

• To close a mask shape, click back on the first point. Only closed masks createtransparency. If you want to create an openpath, simply change to the Selection tool(shortcut = v) when you’re done creat-ing the path.

Editing the PathWe prefer to switch back to the Selectiontool for editing a mask – with the Pen toolactive, it’s all too easy to delete a point by clicking on it! However, some of theshortcuts will then change slightly; verifythe different modifier keys by hoveringthe cursor over a point and seeing whichcursor appears. Because closing the maskcreates transparency, we suggest you editthe mask shape in the Layer window withthe Render switch off so you can see theentire image. Feel free to zoom in as well.

4 Press v to change to the Selectiontool, and practice the following:

• To add or delete points, press C (L)and click. The cursor will automaticallychange to a Delete Vertex tool when you are near a mask point, and an AddVertex tool when you are between points.

• To toggle a point between a smooth and sharp corner point, press CO

(LA) and click on the point.

• To break or rejoin Bezier handles, press C (L) and drag on a handle.

5 After you’ve had some fun, delete your experimental mask, and practicedrawing a mask around the foreground tulip. It will require a combination ofsmooth shapes and sudden direction changes where some of the petals overlapor meet the stem. It’s fairly challenging, but a good example of a task you willoften need to perform in the real world. Our version is in the Comps_Finishedfolder (02a-Bezier_final), pictured to the right.

Lesson 489




� Applying an Effect to a Masked AreaTo apply an effect to an area of a layer in Photoshop, you would create a selection

first. In After Effects, you need to duplicate the layer, create a Mask Shape on the

top layer, then apply the effect to the top or bottom layer depending on whether

you want the area inside or outside

the mask to be effected.

For instance, let’s say you

wanted the foreground flower in the

PinkTulips.psd layer to be in color,

and the rest of the image to be in

black and white. Open the comp

02b-Masked Effects*starter, and

follow these steps:

• Select the PinkTulips.psd layer,

and select Edit > Duplicate – the

shortcut is Cd (Ld).

• Select the bottom layer, and

select Layer > Masks > Remove

All Masks.

• With the bottom layer still

selected, apply Effect > Color

Correction > Tint to make it

black and white. Our version is in Comps_Finished > 02b-Masked Effects_final.

You also could have started by duplicating your original layer and masking just

the top one. The end result is the same: Two layers, the top one has the mask, and

effects are applied as needed to one or both layers.

aea1 04 transparency excerpt

Documents