CGI scans will allow actors to return to their youth... forever

 

In the past, Hollywood stars would rely on cosmetic surgeons, make-up artists and lighting to extend their marketable shelf life. Now, it's visual-effects artists who tighten and brighten faces and bodies, allowing actors to play characters decades younger and tens of kilograms lighter - even letting them perform from beyond the grave.

The trend is borne out of the routine "beauty work" that is secretly and painstakingly carried out in post-production of movies to erase pimples, wrinkles and muffin tops from Hollywood stars. When pushed to the limits, beauty work can shave years off actors and reduce the need for punishing body transformations: why spend hours in the gym when perfect abs can be added digitally?

To future-proof franchises, studios have started to make 3D scans  to  capture actors' likenesses at a point in time. "It's the modern equivalent of taking a life cast," says Trent Claus, a VFX supervisor at Lola VFX, the company that pioneered digital de-ageing with Brad Pitt in The Curious Case of Benjamin Button.

The  data  collected in these high-resolution scans can be used as a reference point for skin texture and facial expressions in years to come, when the actor is required to play a younger character.

By applying machine learning to footage of actors before and after de-ageing artwork, studios can automate the process - much as how phone filters can make  selfies  zing - to avoid frame-by-frame tweaks.

Increasingly, visual-effects artists are actually enhancing actors' performances. They might introduce the perfect tear to roll down a star's cheek, reanimate brows frozen by Botox, or simply merge footage from several takes to create the desired cut without resorting to a re-shoot.

When stunts are too dangerous for the star, photo-realistic computer-generated models can be inserted into scenes. These virtual doubles are created using sophisticated camera rigs, such as the University of Southern California's Light Stage system, that shoot actors from every angle. As they pose, striking dozens of facial expressions and gestures, a library of movement is created that can be used to drive an animated performance. "It's a way for actors to future-proof themselves," explains Eric Barda, an effects veteran who worked on Tron: Legacy - in which Jeff Bridges time-travelled 30 years - and Benjamin Button.

Nailing the essence of a character using  CGI  is extremely difficult, particularly in emotional close-ups . "We're not looking at a static sculpture. We're trying to capture all the perceptual cues that make us recognise that person and read their emotions as expressions change and they are communicating," explains Martin Hill, the VFX supervisor at Weta Digital, which was tasked with creating a computer-generated Paul Walker after the actor died in a car accident part-way through filming Furious 7.

"You need the nuance, the timings, the way the actors deliver lines, the way their eyes move - all of the subtleties that make their personality," adds Barda. Failing to capture these aspects leaves digital characters squarely in the uncanny valley. (It might look like Jennifer Lawrence as Katniss Everdeen, but does it behave and feel like her?)

Once it's possible to perfectly capture a single performance in CG, the next step is to try and replicate human actors' ability to bring something unexpected to the set. "A good actor will give you a nuanced take between each performance. You'll get happy mistakes and you need a director to draw that performance out. Sitting for weeks and weeks with an animator and fine tuning a micro-expression is a painful way to achieve the same thing," says Mike McGee, co-founder of visual-effects company Framestore.

The end goal - which will require a significant leap in processing power - is real-time responsiveness in synthetic humans. "We want to be at a level where a director could work with a CGI actor in the same way as a real actor, and see the final frame as he's working," says Weta's Hill.

This creates the opportunity for actors to sell their future image rights for films they appear in after they are dead, or have their likeness feature in interactive  virtual reality experiences or games. As a result, savvy stars are starting to demand ownership of their virtual selves. "It would make sense for actors to own their data and then make their money back by licensing their scan for use in a particular movie," explains Paul Debevec, a professor at the University of Southern California's Institute of Creative Technologies, who created the Light Stage system and is now working in  Google's VR team. Debevec says he's seen a small number of "very high-end actors" starting to do this.

Does this mean actors will be able to relax at home while their digital doppelgängers do the hard work? McGee is doubtful. "The celebrity of actors off-camera forms a big part of the hype around Hollywood films," says McGee.

 Having said that, it may in the future become possible to apply perfect CG overlays to live footage - whether that's on the red carpet or during press junkets, jokes McGee. "Then who knows what you're looking at?"