I am becoming increasingly interested in combining hand-drawn artwork with generative art techniques. There are many ways to do this, but I want to focus on a simple one today: masking. The basic process is to create hand-drawn masks that control how much of a generative layer shows through.

This piece is pretty straightforward, which makes it easier to explain.

For this explanation I'm my using usual setup of using Quil, a Clojure wrapper for Processing. For the hand-drawn masks, any program you're comfortable with (Photoshop, Corel, Gimp, etc) will work just fine.

BASICS OF MASKS

Before I begin with the process, I'd like to quickly explain what a mask is and how it works. In Processing (and most images, in general), each pixel has an alpha value that determines how transparent the pixel is. An alpha value of 1.0 makes the pixel completely opaque, while an alpha value of 0.0 makes the pixel completely transparent.

Processing's mask() function (called mask-image in Quil) allows you to set the alpha values for an entire layer at once. To do this, it takes a second gray-scale image called the mask. The brightness of each pixel in the mask determines the alpha value for the layer it's applied to. Where the mask is white, the layer is completely opaque. Where the mask is black, the layer is completely transparent.

For example, suppose we have the following background and we want to add a masked layer on top of it:

Our layer (without any masking) looks like this:

And our mask looks like this:

When we mask the layer and then apply it to the background, the result looks like this:

This should give you a good idea of how we'll use masks and layers.

THE DRAWING

Although I'm comfortable with drawing on a wacom tablet, there's something about using a pencil on paper that keeps my lines relaxed and smooth.

I began the process with a roughly two-hour pencil drawing:

If this drawing were going to be the final work, I would polish it. But, since I know that the masking process greatly simplifies the drawing, there's not much point in it. Even the photograph (taken with a cell phone camera) is pretty bad. The upside is that a one or two hour drawing can be a decent starting point.

STARTING THE MASKS

The next step is to import a photo of your drawing into MyPaint (or your choice of application) and begin drawing the layers. At this point you need to have a rough idea of how you want your final image to look. How many layers will there be? What areas of the drawing will each layer cover? Try to answer these questions up front, but don't worry about being perfect from the start. You'll modify and polish the layers several times as you refine the image.

For this work, I tried to use three layers based on values: an (implicit) dark layer, a midtone layer, and a light layer. The midtone layer covers shadow areas on the skin and other parts of the eyes. The light layer covers the bright skin. The dark layer includes the hair, facial hair, parts of the eyes, and the background, but I didn't actually draw this by hand. Instead, anything not covered by the other two layers serves as the dark layer. This division of layers may work for many pieces, but you certainly shouldn't limit yourself to this.

First, I created the mask for the light portions of the face:

While working on the mask layers, I like to keep the original image in a lower layer and set the transparency on the mask to about 50%. This allows me to see the details in the original image through the mask while I'm working.

Next, I created the mask for the shadow portions of the face:

One thing to note about about these layers is that there are some areas with a sharp border and others with a smooth, soft transition. For example, the shadow on the cheek is an area where there is naturally a smooth transition. To replicate this, the shadow mask and light mask overlap and both use gray areas here.

It can be a little bit strange to work with "inverted" mask layers, like the midtone mask. In later works, for layers where it makes sense, I've switched to using black for the fully opaque portions and white for the fully transparent parts. Before using the mask, I invert the grayscale image. This makes the mask layers look much more natural while they're partially visible above the original image.

FILLING THE MASKS, ITERATING

At this point, I save each of the mask layers out as a separate png image. To begin with, your application should probably apply each of the masked layers with simple colors. The code to do this looks roughly like this:

(defn draw []   (no-loop)    (color-mode :hsb 360 100 100 1.0)    ; color the background   (background 160 20 30)    ; apply midtone and light masks   (let [^PGraphics the-mask (create-graphics (width) (height))         ^PGraphics the-overlay (create-graphics (width) (height))]      ; midtone layer's mask     (with-graphics the-mask       (color-mode :hsb 360 100 100 1.0)       (background 0 0 0)       (image (load-image "midtone-mask.png") 0 0))      ; midtone layer's fill     (with-graphics the-overlay       (color-mode :hsb 360 100 100 1.0)       (background 200 20 40)       (mask-image the-mask))      ; apply the midtone layer to the image     (image the-overlay 0 0)      ; light layer     (with-graphics the-mask       (background 0 0 0)       (image (load-image "light-mask.png") 0 0))      ; light layer's fill     (with-graphics the-overlay       (background 55 7 95)       (mask-image the-mask))      ; apply the light layer to the image     (image the-overlay 0 0))) 

For example, here's what my image looks like with flat layers (although the masks used here are the final ones):

This step will expose a lot of small flaws in your masks layers that need to be corrected. I suggest spending some time fixing the masks here, but don't get too caught up in it. The image normally looks pretty bad with flat, solid layers. Once you begin using more complicated patterns and textures, some of the mask flaws may not be noticeable, or you'll want to fix them in a different way.

As you begin to replace your solid color layers with something more detailed, go back and polish the mask layers. I suspect you get the best result when you try to improve everything at the same time instead of working the masks to completion first and then finishing the patterns.

WRAP UP

As I mentioned before, this is only one of the simplest ways to integrate hand-drawn artwork with algorithmic art. There are many, many others that I hope to explore and document. I hope that you learned something from this, and I'd love to see any works that are made using this technique! Until then, take care :)