What is compositing?

Did New York city have to be demolished to get that shot? Can we really conjure mystical clouds from the end of a stick? Do we have ten foot tall blue people on our neighboring planets? Since when did we have to share the pavement with dinosaurs? Film has the power to make us believe that the answer to all the questions is 'yes'.

These visual spectacles are possible through a cinematic effects technique called compositing.

So what is compositing?

Compositing is combining visual elements from different sources into one single image, making it appear as if all of them are part of the same scene. The trick is to do this so seamlessly that the viewer will not be able to tell that the image is made up of several juxtaposed or overlaid elements.

Compositing is extensively used in filmmaking to generate imagery or footage that is physically not possible to shoot all at once. It has broken down all boundaries to what can be shown, surrendering to the imagination of the creative filmmakers and artists. Today, with digital compositing, there is nothing that cannot be created. Live action people can be placed in virtual worlds, animated characters can be put into real world, or both the characters and the world can be generated on the computer. It’s unbelievably awesome!

Here is an explicit clip about the composition work done for Star Wars : Threads of Destiny that was nearly 80% composited.

Evolution of compositing techniques

Evolution of compositing techniques

• Physical Compositing
• Multiple Exposures
• Background Projections
• Matte Printing
  
• Digital Compositing

Physical compositing

In physical compositing all the separate parts of the image are physically placed in front of the camera and are recorded in a single exposure. These elements have to be placed at specific distances from the camera so that they are all in sharp focus. The most common types of physical compositing are partial models and glass paintings.

Partial Models : These are typically set extensions like ceilings or upper stories of buildings etc. In order to cut costs on sets, filmmakers often build only part of the set, making up for the rest with models. A model built to match the actual set is placed closer to the camera so that it appears to be part of the set. The models have to large enough so that they can be placed at a far enough distance from the camera so that the model and the set far beyond are both in sharp focus.

Glass paintings : Large glass pane that is big enough to cover the camera frame is placed at a distance from the lens so that the pane is in focus. The entire set is painted onto the glass with gaps so that the background is visible through these gaps. While holding up the glass pane live action is shot through the glass and composited with the painted area. The painted glass can be an economic foreground or background, saving a lot of time and money that would have gone into on location shots or studio constructions.

Multiple exposures

Multiple Exposures

Exposing the same film frame again and again results in a negative that is a composite of all the individual exposures.

Superimposition of images can be done during production, by shooting the first sequence and then rewinding the film to reshoot over it again. They can also be shot separately and combined later in post-production by using the optical printer to impose the images on one another.

The purposes of such effects can express a passage of time and/or exhibit parallel events occurring at the same time.

Background Projection

Background Projections

A background plate, still or moving, is shot first and is thrown on a screen behind the live action. Both are then photographed together creating the illusion of simultaneous occurrence of both events.

Front Projection

In Front Projection the image or footage is projected from in front of the screen and the live actors. The screen is made of a highly reflective material so that the image is reflected only off the screen and not the actors.

Rear Projection

In Rear Projection the plate is projected onto the back of a translucent screen. The plate is loaded into a projector and the film is flopped. So when seen from in front of the screen the action is right. The projector needs to be really bright in order to match the foreground lighting. The advantage of rear projection over front is the elimination of any possible reflections off of the actors.

Here is an example of Rear projection of New York city streets giving the illusion of car movement :

Projection is sometimes better than other post techniques like matting etc because, since the background and the action are shot together, the mismatch of mattes creating ghostly borders can be avoided.

Matting

Matting

Matting is to composite several elements one after the other onto the same film. As opposed to double exposure, here each part of the frame is exposed only once. While capturing one part of the image the other parts of the film, that are not to be exposed, have to be blocked/ masked out.

So mattes/ masks are the supplementary plates that block out certain portions of the film. Mattes can be of two types :

Static mattes – The shape of the mask does not change from frame to frame

Traveling mattes – The shape of the mask changes with the action.

Here is an example of a static matte where the window pane is masked out and later filled in with the footage of a train passing along:

Creating mattes

How is Matting done ?

1. Creating mattes is the first step to matte-print compositing.

This can be done in several ways :

The earliest versions of mattes were made with cut-out cardboards that were held in front of the camera to block out certain parts of the frame from getting exposed. Then it would be replaced with the opposite matte that covers up the exposed part and exposes only the unexposed part of the film.

Hand drawn mattes are created with a technique called Rotoscoping , where mattes are manually created by drawing the outlines of elements from live action plates.

One of the most common ways of creating mattes for television or motion pictures is Chroma Keying. In this color keying technique particular color from the image can be removed and made transparent to a background placed behind it. The action is shot against a background with a single color or a limited range of colors, usually blue or green screens since they are farthest away from skin color. In compositing the portions of the video that match a preselected color are replaced by an alternative video. Therefore a chroma key subject should not wear the color of the background.

Green and Blue Screens

Blue screen – In the days when films was extensively used, blue screens were preferred because it is complementary to skin color and blue emulsion of the film has the finest grain.

Green screen - Lately green has become more popular because digital cameras retain more detail in the green channel. Also it does not require much lighting to be bright.

The biggest challenge of setting up blue or green screening is getting even lighting and avoiding shadow. In order to get a clear key it is better to create at least a two stop difference between the subject and the background.

Here is a history of green and blue screen effects and how they have infiltrated most films:

Yellow Screen

Yellow Screen - Also called sodium vapor technique, it solves some of the problems with blue/green screens. A white screen is lit with a powerful source of sodium vapor lights. Sodium light has a narrow spectrum source which falls between the sensitive layers of the color film. So complete range of colors can be used on the set.

Cameras with a beamsplitter prism is used to expose two separate films simultaneously. The first one is a regular color negative film which is less sensitive to sodium light. The other is a black and white film that is sensitive to sodium vapor light color. So the live action plate and the matte are created simultaneously.

Matte printing

2. Then using the mattes, all the separate elements are put together using an Optical Printer

The Optical Printer

The Optical Printing Composite Process:

• Traditional matting, for a long time, was compositing by printing elements of different film strips onto a final duplicate film stock using an optical printer. This technique, although requires great precision, gave exceptional results and possibilities
• Firstly the action is shot against a blue or green screen. A matching camera filter blocks out the background color allowing only the action to be captured in the film.
• A high contrast film is used to get a contrasting (mask) image. This image is reversed using another high contrast film.
• The 3 film sandwich is run through the optical printer-The fresh film at the bottom, followed by the traveling matte with the background covered, on the top is the negative of the live action. The resulting combined print is the live action with a plain, unexposed background.
• Then another 3 film sandwich-The exposed film, the opposite mask, the background to be composited.
• Finally we have the positive printed composite which can be converted to a dupe negative and combined with the rest of the film reel. The process gets more and more complicated as the parts to be composited increase.

Take a look at a complex optical printing shot done for Star wars by ILM :

Digital Compositing

Advancement in software and technology now allow us to composite digitally. Digital matting is better than traditional matting because in traditional optical printing the parts might go out of registration. Also since the final print is a third generation copy, and film loses its quality each time it is copied.

Digital Compositing

Digitally assembling multiple images into a final image. It also includes scaling, retouching and color correction of images. There is a wide range of software available in the market that enable us to composite.

One of the most common software readily available for compositing and other effects is Adobe After Effects.

Here is a vedio showing basic compositing using After Effects :

A video showing the 2D compositing on After Effects :

How can green screened footage be composited digitally?

Software now also allows for color correction, allowing digital images to have the authentic film look.

A fun video showing the myriad possibilities of composing digitally :

CG Work flow

Digital compositing workflows are of two types :

Node-based compositing – The entire composite is a tree like graph, laying out everything from the source media objects to the final output in the form of a map of nodes. It allows great flexibility for going in and modifying the process steps. But these are not effective in handling keyframing and time effects because the workflow does not stem from a time line.

Eg: Apple Shake, Blender, EyeOn fusion, Foundry Nuke

Layer-based compositing – Each media object is a separate layer in the timeline, each with its own time bounds, effects and keyframes. Well suited for quick 2D motion graphics; becomes awkward for complex composites with many layers. The composite order of elements can be viewed in a flowchart.

Eg: After Effect

Software and Companies

Software

Popular Software like Shake, Nuke, EyeOn Fusion are used extensively in the industry. Most of the blockbuster movies like Lord of the rings, Harry Potter, Cloverfield, King Kong, Fantastic four, Pirates of the Caribbean, Hulk, The Dark Night were done using these software. In most cases they are tweaked by the VFX companies to suit their needs.

Some of the top notch, state-of-the-art visual effects companies :

Industrial Light and magic (ILM) – Founded by George Lucas in 1975, it has be providing post-production visual effects services to the entertainment industry. http://www.ilm.com/

Sony pictures Imageworks - Los Angeles, California http://www.imageworks.com/

Rhythm and hues - Los Angeles, California http://www.rhythm.com/

Framestore – Working in digital film for over twenty years, it is the largest visual effects and computer animation studio is Europe. They worked on films like The Golden compass, Harry Potter. http://www.framestore.com/

Weta Digital – A world leading visual effects company based in Wellington, New Zealnad. http://www.wetafx.co.nz/

Soho VFX - Toronto and Los Angeles http://www.sohovfx.com/

Becoming a compositer

All this knowledge on compositing hopefully juiced you up on wanting to know more about compositing or inspired you to become a compositer.

Want to know what it takes to be a compositer?

Here is an article by John Lafauce Jr. who tells us how to become a feature film compositer.

http://library.creativecow.net/articles/lafauce_john/film_compositor.php