Pulling the Matte: Creating masks with keying

As a VFX artist, keying is a frequent task. But if you think that the computer automatically creates a feature film quality mask, you’d be wrong. In this article you will learn what to watch out for.

Text & Photos: by Marco Zanoni

I often find in my work as a freelance lecturer that there are some myths and many misconceptions about keying. Complete beginners are taken in by the idea that the computer automates the arduous task of mask extraction. This is well reflected in the students’ projects and can be seen when marking amusing dancing object edges, holey foregrounds and green screens still half present in the background. I would therefore like to dispel the biggest myth surrounding keying: Keying is not a completely automatic process. A lot of manual work is required to obtain a professional result. The different keys have the common goal of extracting a mask from the original material; with the help of keys an object can be subsequently removed from the background.

Luma key

The luma key is one of the simpler keys and works with the brightness value of an image, as the name suggests. This key is frequently used with pyrotechnic effects which are filmed in front of a black screen. Because of the high contrast of for example white smoke, fog and fire on a black background the key can create a mask based on the difference in brightness. Before use you should check if it is necessary to use a key or whether the layer mode screen (where pure black can be set to transparent) would be sufficient to remove the object from the background. A luminance mask, or a mask in the general sense is well known as nothing further than a grayscale image. Therefore the process of a luma key can also be carried out manually. An RGB image is converted into a one channel grayscale image and the contrast and brightness edited with the help of colour curves.

 

Chroma key

Any particular colour in an image can be used to create a mask with the help of the chroma key. In more premium variants you are able to specific colour ranges rather than individual colour values and key different colour areas.

Colour difference key

The strength of this key is the display and maintenance of half transparency. This means that this key is able to remove shadow from objects in the foreground. The functionality of this key is based on a comparison of the three colour channels: red, green and blue one below the other. With a blue screen for example, the key pays attention to the colour value which has the highest proportion of blue. Therefore the foreground has less blue in its colour value than the background, when using a blue screen. However in contrast to the chroma key the colour to be keyed cannot be freely selected but rather is restricted to one of the three colour channels. This is either blue or green in the majority of cases.

Now that the various tools have been discussed it is time to dispel a further misconception: with one of the keys and the one click method you can (not) create a good mask. Now I will set out the individual steps for creating a final mask.

Material analysis

Your first task as a VFX artist is to analyse the shot for possible problems to determine how you can create a good key. Once you have done this you can commence work. I will now show you the stand alone process of using a software package. Next spurious image elements are removed using a garbage matte. We must also decide whether creating a so-called key-in plate is sensible. Often keying is hindered by film grain or video noise. In this case it helps to remove the grains from the material in advance, to create a mask from this material and then to use this mask on the unedited original material. It is important to remember that a key alone will not deliver the required result. In our example we have the problem that the fine hair detail must remain. When viewing the resulting mask it is striking that it is hard and there is no semi-transparency at the edges. In exchange the mask is solid – there are no black holes in our foreground. For this reason our final mask will be    constructed from several partial masks. The current mask is the so-called core mask (core matte), which is combined with an edge mask (edge matte) created by an additional key. The edge mask is a flexible mask which reproduces the fine details of the edges of the object well. However there may still be holes inside that mask; to prevent this we combine our flexible edge mask with the opaque core mask.

Improving masks

Once the individual masks have been extracted normally some fine tuning is required. The starting point to improve a mask is always to create the best possible mask. With the help of contrast and gamma correction the edges of the mask can be optimised. It is also possible to edit using filters such as a blur filter for example.

Spill suppression

After the object has been extracted the edges of the object have a green shimmer. Here despilling, also called spill suppression is used to change the unwanted colour to a neutral grey or a colour which will later fit with the background. A good way to achieve this is selective colour correction using an outline mask (outline matte). To do this an edge detect filter is used on the created edge mask. The resulting contours will be finally used to define the zones for colour correction.

Conclusion

If you spend sufficient time on the details this will show in your compositions. The steps described here are the foundations of keying but additional steps may need to be incorporated, depending on your requirements.

Typical keying problems

• Interlaced video material
  •                Semi-transparency
  •                Fine detail
  •                Shadows
  •                Spill
  •                Reflection of the screen on the reflective foreground areas
  •                Motion blur and fuzziness
  •                Over / underlit screens
  •                Impure screen colour
  •                Spurious image elements
  •                Film grain