Reverse Art Directing

**Kim Sinclair, **profiled in the last issue, began art directing in another era, a pre-digital era. Yet this year he won both a BAFTA and an Oscar for his work on the cutting edge 3D VFX epic Avatar. Here, he talks to Peter Parnham about yet another exciting new role – virtual production design.

Working on Avatar was a big wakeup call, recalls Sinclair. There was so much new terminology to understand and no textbook. “Visual effects had always been like a closed shop. I had read about it in magazines and things, but I never really understood it,” he confesses.

But before he’ll be drawn into discussing his role as Supervising Art Director on Avatar, Sinclair makes sure to dispel any notions that Avatar sets were simply computer graphic creations. He does this with an impressive array of statistics: 50 people in the New Zealand art department, 150 in the construction crew, 44 sub-contractors, 647 contracts.

Nor was it as simple as building sets and shooting in them. Sinclair says that thanks to pre-visualization, performances were shot using sets that were not yet designed and built. Pre-visualization, pre-viz for short, takes traditional storyboarding to the next level by using computer graphics to simulate and plan shots in virtual space – a space like the world of a 3D computer game.

With the high cost of VFX or stunt sequences, this kind of planning allows the director to refine the shots and the pace before spending any money on set. The pre-viz also helps visually communicate intentions to the rest of the crew. Individual objects and characters in the scene may start as rough outlines or animations and be replaced by real footage as it is shot.

Avatar was not only VFX-heavy; the movie shoot was also stereoscopic (shot with two cameras for 3D). As it was one of the first of the new wave of movies shot for 3D, its production designers had to figure out what worked by experimentation. “We were very, very conscious of the way 3D looks in the design, and how it was shot,” says Sinclair. “Avatar was properly done, in that it was all about getting depth, and not having joke objects sticking out of the screen.

Through the fourth wall it was all about getting depth, and not getting kinds of crazy layers. You look at some 3D and it’s a question of how far you go before you get to the backdrop (the 2D background), which in Avatar means when you eventually get to the sky dome. Normally even that was 3D, so the clouds and everything were 3D”.

“The nice thing about Avatar was that every take was reviewed in 3D. Every single time Jim (James Cameron, the director) did a take and the actors didn’t stuff it up, we’d race to a little container with a proper screen projecting 3D. It was important to view it projected properly in 3D rather than just on the monitors. If it wasn’t okay, we’d retake the shot. For example, foreground often looks terrible in 3D. So, we’d do it again, but we’d move something out of the edge of the shot, or put something deeper in the shot. We might change the convergence, the IO [inter-ocular distance – stereoscopic camera settings], change the set dressing or change the camera direction a little bit. Then we’d move on – Jim’s not the sort of director who does three takes for luck”.

“The whole thing was conceived as a 3D film, the shots were choreographed to look good in 3D and it was art directed to look good in 3D. I’ve now seen Avatar in 2D and 3D and it works really well in 3D. But I’m not intrinsically mad about 3D.

I saw A Christmas Carol, and I thought the 3D in it sucked. Technically it worked well, but you constantly saw the camera do things to show off the 3D, which took you out of the story. At one point the camera goes through a sign with an O in it. If you do that technically with a physical camera, it would be an amazing shot and it would probably be in the movie. But to do that with a virtual camera, what’s the point of that? I found it irritating.”

Sinclair believes the expense of stereo rules it out for many productions. “There are films that it wouldn’t enhance anyway, and I think there’ll be a huge backlash as many films will be rushed through in 3D, done badly.”

Tintin – a new approach to art direction

Advances in production technology don’t stop with pre-viz, VFX and 3D. “At present I’m working on Tintin, which is 3D performance capture,” explains Sinclair. “It is completely animated. I feel I’m getting on top of the lingo too now – there’s a lot to learn though.”

Performance capture is a refinement of motion capture which captures details of actors’ facial expressions. Actors generally work in a clear space wearing simple dark cat suits with colored marker dots placed on their bodies and face. A minimum of necessary objects are similarly marked. The performance is captured and fed in to a computer to reflect the actors’ movements and expressions in real time.

At a later stage the body shapes of the characters can be altered, and skin, hair, and costumes are added to transform the figures into animated or photo-realistic characters. The whole process yields surprisingly natural-looking movements and facial expressions in digital characters. The Avatars in the "Avatar" movie stand as a testament to the realism of the technique in a production which was a mixture of live action and VFX.

But Sinclair says the style and approach of the animated Tintin is quite different. “We get a cut [edited sequence] before we actually start building the sets – there are no physical sets, it is completely digital.”

Because the performance is done with real actors in real time, directors can direct in the normal way; however, because the creation is a computer model, the ‘camera’ is a computer-calculated point of view rather than a physical restriction. This means it can fly anywhere in the space – hence the term virtual camera.

“Directors love the virtual cameras; in the right hands it’s good for them, it gives them amazing freedom,” notes Sinclair.

But the creative freedom that performance capture gives the director also creates challenges for the production team that has to create the rest of the screen world from what have become pre-viz files – the foundation onto which the final animation is built.

“If you are doing virtual production, the role of the designer at the moment is so stuffed up. We reverse engineer it. If people are sitting at a table I can change what the table looks like, but I can’t change the size, the height, the shape, or where it is in the room. And they’re all decisions that are art direction decisions, so ideally someone should work with the director early on in the process to pin these things down.

“Pre-viz has this great saying – 'it’s just a placeholder.' But if it’s in the wrong place, then all the actions are wrong. Every object is an asset, including the set, and characters for that matter. They’re replaced one by one. For example, with simple things like a glass, we need to find the glass we want to use. In this case, I couldn’t change much: it could be cut crystal, and it could be blue plastic. We have to build everything used in the performance capture, and then just swap it out. We’re constantly pushing and tweaking, trying to make things look as good as possible.

“It’s no secret that Tintin is set in post-war Europe, and normally you’d be trying to get a sense of that time and place. It’d be easy because you’d build a set, and you’d get props, and it would feel like Brussels. Try and do that in a virtual world where you’ve got nothing – it is pretty scary.

“We spend a lot of time asking 'What are we going to do here?' The animators can re-animate it, but they never want to because the performance capture and the facial capture are so good that they don’t want to change it.

“When we are finished, it’s modeled, it’s textured, it’s animated, it goes on and on. It’s a virtual pipeline, so we’re the first step, but essentially we provide the design.”

Traditional approach still attractive

Sinclair believes virtual production doesn’t mean the end of physical sets, special effects, and live action production. “It’s very, very expensive,” he explains. “It will get cheaper, but it will still be expensive. I can’t see replacement of those things. It’s not cheaper for us to physically build a full-size 747 and have it crash into a skyscraper. But it’s still cheaper for us to build a house than it is to model a digital house, texture it and put it into the movie.”

While the whole virtual process sounds quite complicated and energy-sapping the way Sinclair describes it, he says it doesn’t compare to traditional production. “When there’s a live action film you get that amazing energy. For twelve weeks there are fifty people, including the director, who do nothing but live, breathe, walk and sleep the movie. They’re thinking about it the whole time, you get the energy, because everyone’s got the same goal. You get the enthusiasm, everyone talking about it, and everyone concentrating on it.

In virtual production, you’ve got something that takes place over several years and the actors are only on set for four weeks. They don’t put on costumes, so they like it, because it’s like a theatre workshop. They get to work one-on-one with the director, instead of everybody dicking around with their hats, and their guns, and their car, and their background, and saying, ‘I’m sorry, we’ve got to go again – there was cloud against the sun’. When that’s done, somebody else does the other parts of the process. There’s just not that energy, and by the time you ask the director questions about direction, it might be their third job since that, and they can barely remember what they were thinking when they shot. “So that’s all, well, horrible,” grins Sinclair, “and it’s the future.”

In a good place

With Sinclair’s Oscar in the cupboard and his prowess with the new technology, you would think that he would find it easy to get work, and would be able to sit back and take only the most plum offers. He sets the record straight: “Getting onto jobs is just random; they’re all accidents, it is just the most bizarre thing. Someone in a pub who knows someone, who knows somebody. It’s always a struggle.”

“New Zealand is so small; when you are working on a job as a designer, or an art director, it then might be six months or a year until another job comes along.” Despite this, Sinclair is not tempted to move overseas for the sake of work. “Not at this stage in my life,” he says. “Maybe if I was 20, I probably would. Instead, I like trying to get people to come to New Zealand and make films here.

That’s been my challenge for a long time, certainly for the last ten years. It would make your hair curl if I told you about all the jobs that have nearly come here. And some of them are still waiting, and I’m still hoping.”

To see what went into some of the physical sets in Avatar see www.kimsinclair.com. Follow the Latest News link and click on Making of Samson.