How the Avatar sequel was brought to life behind-the-scenes

"It was a scene of Jake Sully waking up. In the middle of it, he laughed. And I got nervous. I looked over at him. A couple of seconds later, he laughed again. I looked over.

"He turns to me, and goes, 'Jon, that’s a good laugh, mate. That’s me up there on the screen.' And that’s the commitment we make to these actors."

Now, with the long-awaited Avatar sequel The Way of Water, the filmmakers are pushing things even further, using evolved technology and new tools to push the boundaries of what’s possible in VFX. To find out more, we caught up with Landau and VFX supervisor and second unit director Richard Baneham.

We just had to remember one thing – never, ever call what they use motion capture…

It’s not motion capture

"I’m going to stop you," Landau told RadioTimes.com, seconds into our first VFX question. “We call it performance-capture.

"Let me give you the distinction, just so you understand. Motion-capture, to me, is missing a key letter in front of it – an 'e', for 'emotion capture'. We want the emotion. We want the performance.

"If we weren’t capturing the face, which is the important thing for us to capture – and if we’re only capturing the body, maybe we would go with 'motion-capture'. But we want the facial performance."

Interestingly, while the exact facial performance recorded doesn’t always make it to screen, VFX supervisor Baneham says it’s no reflection on the actors – and in fact, it was usually to help capture their performance more accurately.

"The intention is not to change the data from the actors’ performances to make it more credible. It’s to bring it closer to the actors’ performance," he said.

"What we do is, we have references cameras onstage. We shoot with sometimes 12, sometimes 16 cameras, to try to witness everything that the actor does. Sam [Worthington] says it best: when we’re on our stage, there’s nowhere else to hide. When you’re shooting on a normal live-action movie, there’s one, maybe two, cameras – and you’re aware of their presence.

"In this, we’re asking our actors to be incredibly earnest, because the camera itself could end up anywhere. In some ways, I think it’s a more honest version of acting. Or, at least, we ask our actors to be honest all the time, and we get it all the time. It’s a hefty requirement."

It’s all about the face

Sometimes, that means drilling down into incredibly granular facial movements, as in the case of Cameron’s returning Titanic star Kate Winslet.

"We keep all of the movement that she does in her face, in the twitches, in the subtleties," Landau explained. "We don’t actually photograph anything. The eyes are recreated digitally. But there’s a great moment in the movie – great actors don’t have to say anything to communicate what they’re thinking. There’s a moment in the movie where Tonowari looks over to Ronal, and is looking for her acknowledgement of something.

"The way Kate played it – she turns to him slightly, and maybe cocks her head a little bit; she closes her eyelids with a certain weight to them. Because we could all close our eyelids and it means something different. And she gave this incredible performance. And that’s what we now have in the shot of her as Ronal.

"So our goal – and we do the performance-capture, we shoot reference cameras. We look at the reference cameras. 'What did the actors do?' Because we want it to be there."

It’s all changed since the original Avatar

Finding these details is now easier than ever as, according to Landau, the filmmakers reached a "new level" with the technology required to visualise Pandora and the Na’Vi in this sequel.

"On the first movie, we used a single, standard-def camera to capture their performances," he told us. "Now we’re using two high-definition cameras. So we’re getting a much higher fidelity performance.

"Weta Digital has a whole new set of tools that includes deep learning to make sure we maintain all those subtleties of performance that we were talking about. And they have the capability of rendering and creating these shots at a much more truly photo-real level."

Improvements in technology also helped overcome difficulties found on the first Avatar, where the filmmakers were limited in how many performance-captured actors could be in one scene at a time before crashing the systems.

"Now, this is information I will share cautiously," Baneham laughed. "We want to make sure our director doesn’t get overly enthusiastic about what’s possible. Because what we push up against the limits…

"But yes, we could do six people on the first movie. We did 18-ish successfully during production [on this one]. I think we pushed to about 22 or 23 in a single session. But, of course, that pushes right up against the limit, and that’s where you can crash."

However, according to Baneham the core technology used on the original Avatar movie remains surprisingly unchanged for this sequel, 13 years later.

"The truth is, the base technology, the ideas, the core group of technologies we used to make the first movie, we used on this. They’re just improved," he said.

"The DCCPs that we use – the Digital Content Creation Packages – are more robust, and can push more pixels and more colours. So it allows us to have a much more fully realised lighting suite of those. And there’s a bunch of other things that will allow us to move a little more quickly, and a little more fulfilled in our image capture process."

The Way of Water

Another big change for this sequel? A large amount of the action takes place in or underwater, which led Cameron to train his actors in free diving so that they could film performance capture in water tanks.

And behind-the-scenes, there were even more thorny problems to work out to bring the scenes to life, especially for scenes that move from under the sea to above it in one smooth motion.

"We obviously have a portion of this that’s underwater," Baneham told us. "The challenge of it was trying to replicate a capture system that we have in, obviously, a dry volume, on a stage, in water.

"But in water is one thing. What we had to do was scenes both above and below water, which creates an interface between the two sections.

"The solution for us was to create two volumes – two discrete volumes – one above water, one below water. We had to temporally and geographically align those, and then sync them, from a capture standpoint, to allow our actors to sit with a portion of the body in the above volume, and a portion of the body in the below volume. And then put it together in real time, so that we could review it in the camera about one-eighth of a second after it’s captured."

Water palaver, eh?

Looking to the future

Of course, James Cameron has two to four more Avatar movies in the pipeline – and both Landau and Baneham expect the technology to continue evolving.

"Look, do we have aspirations for the technology to keep moving?" Baneham said. "Of course. Jim is not a director to sit with 'good enough'. It just doesn’t happen. 'Good enough' are dodgy words on our set. We try never to utter them."

One possible golden future? Eliminating the need for tracking dots and special suits on the actors altogether, allowing them to move more freely and avoid a lot of time in preparation.

"I would say, going forward, a markerless would be aspirational," Baneham said. "It seems like that’s where the technology is headed in the not-too-distant future. Whether or not that is our next step, and whether or not there is a development that will allow that to come to pass – who knows? But today, that would be a nice step. It’ll get those actors out of makeup quickly."

"We’re marching ahead with the movies," Landau added. "Now, you know, no-one can predict tomorrow – as the past two or three years have certainly proven to us.

"But our plan is to make them. And why? Because the scripts have stories that are compelling, that continue to drive – thematically and emotionally – stories while introducing people to new places on Pandora, and new cultures on Pandora."

Interviews by Stephen Kelly

Read more:

• Avatar: The Way of Water review – An enchanting but meandering sequel
• Avatar 2 cast tease future sequels: "There’s mysteries there"
• Avatar producer: '3D got a bad name because people didn't do it right'

Avatar: The Way of Water is showing in UK cinemas from Friday 16th December 2022 and the original Avatar is available to view on Disney Plus – you can sign up to Disney Plus for £7.99 a month or £79.90 for a year now.

Check out more of our Film coverage or find something to watch now with our TV Guide and Streaming Guide.

The Christmas double issue of Radio Times magazine is on sale now – subscribe now. For more from the biggest stars in TV, listen to the Radio Times View From My Sofa podcast.