Toon Shading Part 2- Edge detection

In this blog we are talking about toon shading, its uses and how to create it. This is part 2 of my blog about toon shading, this time we are going into edge detection. For part 1 about cel shading click here.

What is edge dection

Edge detection used when we want to outline a specific object in the scene, whether it be for visual appeal or for game play features. The outline is usual a black line around the object, however sometimes it can be different colors depending on its purpose.

Uses of Edge detection

The first use of edge detection is for enhancing the visual appeal/style of the game. This is gives the game a more comic book or Anime/Manga feeling to it if designed right. In the picture above we are seeing edge detection being applied to borderlands from 2K games and Gearbox studios. Edge detection here allows the NPC and gun to stand out from the environment.

The second use of edge detection would be for gameplay uses, where it would indicate to the player of something important. This would literally highlight the object of interest, and inform/warn the player of this object. In this picture, from Left 4 Dead, a Valve game, we are having this highlight to inform the player of other allied players, so we don't shoot them. However L4D goes beyond that and uses the highlighting as in game feed back to tell the status of their allies. Whether they be yellow for being in danger, or red for being low in health, this is an excellent implementation .

How To Use Edge detection

Ask yourself, how do you detect where edges of objects end in real life? Do we say there is an edge when there is a solid change in colour? If we did that to the program it would detect all the changes in colour, and create edges there like in the picture below. Thats not what we want, despite it looking cool :D

To properly do this we need the normals of each object to help us out. For a quick review, normals are a vector that which is can tell the orientation of the object, and commonly used for lighting. Normals are of the the object are the right way to go, as the program can now properly tell where each object ends, and where a new one begins.

However to do edge detection, we also need to use an FBO, click here for my tutorials on FBOs. The reason being we need the scene's normals, however it is more efficient if we use a shader prior edge detection, which will output the normals of all the objects in the scene. We will apply this shader to the full screen quad, which the FBO is being applied to, after rending the scene to the FBO. This makes it a "post processing of the scene" where we apply the edge detection for all the objects.

Hope you learned a thing or 2 from this, this blog took a deal of effort to write. Note doing this to individual objects, like how we did it for the player in L4D, is a totaly different story.