Planning the mouth shapes

Back to the dialogue shot! There are basically two ways of setting mouth shapes...one is very simple and actually doesn't demand planning another that is a little bit more complicated but more precise.

Do everything in Maya

This is the way that doesn't really demand much planning, you actually do it as you go. I don't know (yet) how it works with other software, but in Maya, you can import the sound file and there's a pretty cool feature there that allows you to hear the foneme that's happening on each frame. So you scrub through the timeline with the left mouse button and hear the line "sound by sound". This is great and you can easily go on listening to the sound and setting keyframes for the mouth as you go.

The problem with this method is...sometimes when you click at the timeline in Maya, the sound you get isn't the sound that will be on that frame when you playblast it: you have to playblast it all the time to make sure it's working. So it's a handy and simple method, but not too precise.

Plan it!

You can plan where each mouth shape will happen by using a sound software to see for how many frames a certain sound actually is played. There's a free software that can do that (yay!): http://audacity.sourceforge.net

Open your sound file in Audacity (or the sound program of your preference) go to View->Set Selection Format->film frames 24 fps, so you can see the time in frames. Now you see the spectrum of the sound and can scrub through it, pretty much like you'd do it in Maya. Make a table and write the sound on one side and the frame where it starts/ends and there's your plan! It's a very simple x-sheet actually.

Here's an image to explain the basics:

1) this tool allows you to select the region of the sound wave you want to focus on

2) zoom tool so you can see that section of the sound clearly

3) ok, this is the play button, of course. But if you have selected a part of the sound, it will only play that. This way you can isolate the sounds.

4) here you can check the frames where this sound is played

Intro to dialogue

This is my first dialogue shot! Started working on it in the end of class 4 and ended it in the first few weeks of class 5. Lots of work and it can be improved still :-)

It's a big step moving to a scene where there are facial expressions and lip synch! So this will be a simple introduction to animating a dialogue.

1. Body pass

To begin with, as usual...reference! I made the reference for the body movements first and animated that (going through blocking, first pass, etc). I played with eye movements and blinks at this point a little, just so the face doesn't look too stiff. Also had a bit of jaw rotations going on too, but just very simple mouth movements. This is how it looked when this body pass was over:

2. Animating the face

To start doing the facial animation, I did another reference video so I could see the expressions clearly and also the mouth shapes. This one was done sitting still in front of the camera. I did check the other reference every now and then too: although I did it mostly for body reference, there are some facial expressions there that make sense, besides, it's still helpful if I want to improve the body, etc...

Using the reference and the sound file I planned the mouth shapes (visemes). First it's necessary to write down the line that will be said but the way it sounds. Each sound is called a phoneme. So for my line, for example

"I...I haven't totally decided...he can ahm...he can live or... (inhale)...he can die"

I wrote the sound as:

"Ae...Ae heven toudaly dicided...hee kenah...hee ken leev oah...hee ken dae"

This transcription has to be done in a way YOU understand it, after all, you're the one animating it. Then it's good to mark the important visemes, the mouth shapes that yo MUST HAVE there or the viewers will notice something's wrong. The order of importance as I learned in AM would be:

1.both lips – p (as in play), b (bye), m (mom), w (wine)
2.lips and teeth – f (four), v (vote)
3.tongue tip and teeth – th (thin)
4.tongue tip and ridge between teeth – t (tick), d (date), s (sue), z (zoo), n (noodle), l (lemon)
5.tongue blade and palate – sh (rush), j (rouge), ch (rich), r (run), y (yaw)
6.tongue back and top of throat – k (back), g (lag), ng (sang)
7.vocal folds – h (hey)

You have to be aware that one sound may affect the mouth shape of another sound. That's called co-articulation. For example, when you say "cold" and "key" you say the sound "k" in both. But when your mouth is saying the "k" it's already preparing to make the next sound. In the first word (cold) the shape will be narrower and the jaw rotation a bit more open, anticipating the "oo" sound that follows. For the second word, the shape will be wider.

Here I have the important mouth shapes in red:

"Ae...Ae heven toudaly dicided...hee kenah...hee ken leev oah...hee ken dae"

In order to know when to set each mouth shape, it's very helpful to make plan on which frame each sound occurs. I'll talk about this in another post. So I set the mouth shapes following the planning and the facial expressions following the reference, did another polish on the body and the result is what you saw up there :-)

There's a lot more to dialogue...but this is a start! I'll talk about planning the mouth shapes in the next post.