How to Use RGB Masks in Your Texture Workflow

Hey, guys! If you don’t know me, my name is Paul H. Paulino and I am a Texture Painter working at Method Studios in Vancouver, Canada. In the past two years I had the opportunity to work on projects such as Independence Day: Resurgence, Miss Peregrine’s Home for Peculiar Children, Justice League, Guardians of the Galaxy Vol. 2, Power Rangers, Black Panther and Godzilla II.

In this short article, I’m going to share a simple but powerful texturing/lookdev technique that I learned and it is widely used in VFX production. I hope it can be useful for your workflow.

Last year I began to write articles and tutorials sharing a bit of my experience and how I got here. If you have missed my past content, check the link below.

The artwork I used on this tutorial is called The Gumball Machine and it was a personal project inspired on one of James Gurney's paintings. You can check the final result on the link below:

Are you ready? Let’s go:

What is and Why do I Need RGB Masks?

If you are a compositor or lighting artist you are probably familiar with the RGB workflow and it can be used to improve renders. But for some reason, a lot of people don't know that the same knowledge can be applied during the texturing and look development phase.

Before we start talking about RGB Masks per se, I want to explain what RGB means quickly. If you work with CG, you probably know that already, but Gerald Bakker wrote this article on his website with a great in-depth explanation of color coding. I am going to quote the part that is interesting for our article but I would recommend you read his full text later.

Multiple ways of color coding exist. The most prominent of these is RGB. So there we are: RGB. R means: red. G means: green. B means: blue. The RGB color coding assumes that every color has three components: red, green and blue. Believe it or not, all colors that humans can see can be composed from a combination of red, green and blue.
— Gerald Bakker

With this info in mind, let’s try to apply it into our texture workflow. An RGB Mask is simply black and white data inside each color component, in which will give you the opportunity to have three different masks on the same map.

RGB Masks are not mandatory for a texture workflow, but it can come in handy and save you a lot of time if you are working with a look dev artist in a professional production environment. Imagine if you have a crazy amount of UV tiles and you need to export three masks. If you are using the RGB method, you can save exporting time, and you will have to deal with fewer files.

Keep in mind that these techniques I’m going to showcase are focused on VFX assets which have UV's. So before you start make sure to check if your asset has proper UV's otherwise you might have problems.

RGB Masks Types

There are no specific rules regarding how you must categorize your RGB Masks, but I’m going to share with you my current workflow; and feel free to adopt and/or modify it according to your necessity. I like to split my RGB masks into two categories: Isolation Masks (ISO) and Damage Masks (DMG). These categories are the most common in production, but you can have even more variety depending on your project. You migh have an RGB channel just for decals, signs, etc.

The Isolation RGB Mask can be used to isolate materials into their mask or if the shader artist requires a specific isolation of the model. When I create an Isolation Mask, I always add the suffix ISO the channel.

Like the name itself suggests, I use the Damage RGB Mask to store any damage information I used before. Into this channel I often add dirt, dust, smudges, scratches, etc.

When I create an Damage Mask I always add the suffix DMG the channel.

Creating Your RGB Mask in 3 Easy Steps

Creating an RGB Mask is as easy as it looks, but there are few things to consider. I’m going to showcase my workflow using MARI, but you can achieve the same result using Substance Painter, Photoshop, etc.

Step 01 - Create a Channel

Create a new channel and make sure that your background color is set to pure black. That will ensure that you don’t have any other color information contaminating your channel.

Step 02 - Add Solid Color Layers

Now, add three color layers. It doesn’t matter how you create these layers, just make sure you have pure red, green and blue colors. If the values aren’t absolute you might lose information from your mask later on.

Step 03 - Add Masks Into Each Color Layer

Now that you have your colors, make sure to name every single layer accordingly and create a mask stack. You can either use masks that you used on different layers and then share it to your new RGB mask or create them from scratch there.

In MARI, you can check each channel color by clicking on Component and selecting R, G or B. Use that tool to check your masks and see if they are correct. Remember, this option only allows you to visualize each channel, it does not affect the map. If you want that instead, you can use the adjustment called: Copy Channel or Shuffle.

If everything looks correct, you are ready to export your brand new RGB Mask! On the next step, we are going to test the mask inside V-ray.

Using RGB Masks in V-ray

For this tutorial, I’m going to show a quick way to isolate and use your RGB Masks using V-ray for Maya. I used the ISO mask in this example, but the same principle applies to the DMG mask.

Step 01 - Import Your Files

Import the RGB Mask we just created using the file node. If you have UDIM’s, do not forget to add .<UDIM> to the end of your file name.


Step 02 - Isolate Each Color

To use the mask, we have to make it black and white by isolating each channel. Each software has its way to separate RGB information, and for this demo, I used a simple layered texture node.

I grabbed connected the R, G and B info and plugged into the Alfa on my layered node. Make the base color white, and your mask is ready to be used.

Step 03 - Connecting the mask

Now that you have your black and white information, you can add it anywhere as a mask. In this example, I added into a blend material inside the blend amount slot as you can see below.

Now you learned how to create and use an RGB Mask! :D

Final Thoughts

I hope you found this article useful. This technique can be used in so many ways, and I'm excited to see how you guys are going to use it! Please send me a message if you used it creatively or if you need any help with the process. 

I want to keep sharing things that I’ve learned in the past two years that I’ve been in the VFX industry. If you enjoyed reading this article, don't forget to share it with your friends. And if you are interested in more content like that in the future, make sure to follow me on social media!

Do you have any suggestion for my next articles? Write your comment below and let me know what you are interested to learn about the VFX industry.
I'll be happy to spread the knowledge!


Check Out my Other Articles

01 - main.jpg