Activity 11: Lip Sync

Intent:

Lip synclip-synclip-synch (short for lip synchronization) is a technical term for matching lip movements with sung or spoken vocals. The term can refer to any of a number of different techniques and processes, in the context of live performances and recordings.

lip sync side

side profile

front profile

The phoneme guides found in SmartMouth.

other examples

 

Matching dialogue to a character’s mouth in order to create the illusion of speech is known as lip-syncing. This effort often consumes more time than any other animation task because you need to make adjustments on nearly every frame. In fact, lip-syncing short animations (even only 1–2 minutes long) can involve hours of tedious labor.

This article explains how to make lip-syncing in Adobe Flash Professional CS5 as painless as possible by utilizing the SmartMouth extension to automatically analyze audio content and assign corresponding mouth shapes. That’s right: you can sit back and relax your hands, back, neck, and eyes while SmartMouth processes the audio in the time that it takes the audio to play back, and then matches each frame using a speech algorithm. You’ll also learn how use the free FrameSync extension to quickly make manual adjustments and tweaks to your character animation.

Introduction to lip-syncing in Flash

The basic unit of speech is known as a phoneme. The mouth shape and facial contortions that correspond to phonemes are known as visemes. Animators generally refer to phonemes and visemes interchangeably, even though they are technically different concepts. The standard set of about seven phonemes/visemes (mouth shapes)—not including a closed mouth—are sufficient to create the illusion of speech on an animated character. The SmartMouth Flash extension includes a sample set of those shapes.

This small set of phonemes corresponds to a wide array of spoken sounds. The full range of sounds that are covered by this small set of mouth shapes are listed in Table 1.
Abbreviated phoneme Full letter list Corresponding examples
none No vocal sound (or inaudible)
A A, I ah as in “cat,” as in “say,” i as in “kite”
O O, U, (W), (R) o as in “boat,” u as in “clue”
E E, (I) e as in “street,” eh as in “trek”
S C, D, G, K, N, R, S, TH, Y,Z s as in “stress,” t as in “tent”
L L l as in “lull”
M M, B, P m as in “might,” b as in “back,” p as in “pass”
F F, V f and v as in “favour”
Table 1. The full phoneme list.Note: Letters in parentheses denote that some sounds made by those letters (for example, W and R) produce a viseme—specifically puckering of the lips, similar to the visemes produced by U and O sounds.

By reusing a small set of mouth shapes, you avoid reinventing the wheel on every frame. Instead, you can simply leverage the repository of mouth shapes that you’ve already created.

 

Task:

  • Create a new Activity 10 sub-folder under the Unit 2 Sub-Folder of your Thawed Drive,
  • Go online and find a front profile and side profile of a cartoon character of your choice.  Save both images into your Activity 10 folder,

front and side profiles

  • Create a new Action-script 3.0 document and set the stage to 720 (Horizontal) X 480 (Vertical) at 24 fps,
  • Import both to the library and create a new layer for both profiles,
    • Layer profiles on different sizes of the stage and re-size so they fill almost half of the stage. Lock layers and begin illustrating one at a time (DO NOT INCLUDE THE MOUTH).
    • Convert each profile illustration to a graphic symbol title “side profile” and the other as “front profile,”

Animation Unit, Lip Sync Activity Part 1 from Digital Voices on Vimeo.

 

  • Now lock the profile layers and turn off the visibility of each layer.
  • Create a new layer and start to illustrate the lip-sync mouth instances for both the side and front profile of your character.  Convert each to a graphic symbol title “profile type and sound,” such as “side A” or “front O,”
  • Search for the common saying or famous quote by your character.  The saying must be school appropriate and must be at least a full sentence.  Mickey Mouse example “Arithmetic is being able to count up to twenty without taking off your shoes.”  In a new layer use the Text tool to input the saying/quote.  Position in on the bottom of the stage and lock the layer

Animation Unit Lip Sync Activity part 2 from Digital Voices on Vimeo.

  • Turn off the visibility of all but the front profile illustration.  Save the lip sync mouth image from above, create a new layer on your timeline titled “lip sync” and import the image to the stage.  Lock the layer and create a new illustrated mouth layer.
  • Illustrate each instance of the lip sync and convert to graphic symbol.  Delete from stage and create next instance of lip sync.  Hint! try zooming in really close when creating the instances of the lip sync.  Make sure the size of the mouth fits the head.

  • Repeat the previous step, but this time illustrate the side profile lip syncs.

Animation Unit Lip Sync Activity part 3 from Digital Voices on Vimeo.

  • Test your syncs by adding the lip sync symbols as new key frames,
  • You may need to colour in the skin tones in each symbol for overlapping,

  • Hint! you may need to use the onion skin feature for positioning.

 

  • Try to make your character say the quoted you found without adding the audio.  Play around with the spacing between keyframes.  The audio will be added in the next activity.

 

Saving, Exporting and Posting: