Shader Concepts

The graphic engine for id Tech 3 has taken a step forward by putting much more direct control over the surface qualities of textures into the hands of designers and artists. In writing this manual, we have tried to define the concepts and tools that are used to modify textures in a way that, it is hoped, will be graspable by users who already have basic knowledge of computer graphics but are not necessarily computer programmers. It is not a tutorial, nor is it intended to be one.

The Manual for the Q3Radiant editor program contains a section called Creating New Assets that has the necessary information for setting up the files to create your own custom id Tech 3 shaders. You should follow a similar convention when creating your custom shaders. It is reproduced here for your convenience:

Creating New Assets

If you are familiar with the required tools, creating new assets for use in Quake III Arena is not particularly difficult. As a rule, you should create new directories for each map with names different from the names used by id. If you are making a map that will be called "H4x0r_D00M", every directory containing new assets for that map should be titled H4x0r_D00M. This is to try and avoid asset directories overwriting each other as the editor and the game load in assets.

It is recommended that you study the scripts in this document and in the individual game shader scripts. Pay careful attention to syntax and punctuation. This is where you are most likely to make mistakes.

What is a Shader?

Shaders are short text scripts that define the properties of a surface as it appears and functions in a game world (or compatible editing tool). By convention, the documents that contain these scripts usually have the same name as the texture set which contains the textures being modified (e.g. base, hell, castle, etc.). Several specific script documents have also been created to handle special cases, like liquids, sky and special effects. For Quake III Arena, shader scripts are located in "baseq3/scripts" or in the compressed PK3 file "baseq3/pak0.pk3".

A Quake III Arena shader file consists of a series of surface attribute and rendering instructions formatted within braces ("{" and "}"). Below you can see a simple example of syntax and format for a single process, including the Q3Map2 directives and "Surface Parameters", which follow the first bracket and a single bracketed "stage":

textures/liquids/lava-example                   //Shader name
{
    deformVertexes wave sin 0 3 0 0.1           //General directive
    q3map_tessSize 64                           //Q3Map2 directive
    surfaceparm lava                            //Surface parameter directive
    qer_editorimage textures/common/lava.tga    //Editor directive
    {
        map textures/common/lava.tga            //Stage directive
    }
}

Shaders need to be referenced by the map editor, compiler (Q3Map2) and game engine - each of which uses a different part of the shader. In the above example shader, the different sections are listed on the right.

Shader Name & File Conventions

The first line is the shader name. It is used by the map editor, compiler, game engine, and models to reference the script and art files associated with the shader. Shader names can be up to 63 characters long. The names are often a mirror of a pathname to the texture image without the file extension (.tga or .jpg) or base dir (/quake3/baseq3 in our case), but they do not need to be.

Shaders that are only going to be referenced by the game code, not modeling tools, often are just a single word, like "projectionShadow" or "viewBlood".

Shaders that are used on character or other polygon mesh models need to mirror the art files, which allows the modelers to build with normal textures, then have the special effects show up when the model is loaded into the game.

Shaders that are placed on surfaces in the map editor commonly mirror an image file, but the "qer_editorimage" shader parameter can force the editor to use an arbitrary image for display.

Shader path names have a case sensitivity issue - on Windows, they aren't case sensitive, but on Unix they are. Try to always use lowercase for filenames, and always use forward slashes "/" for directory separators.

Following the shader name, is an opening bracket "{" designating the beginning of the shader contents, and a closing bracket "}" at the end of the shader. Notice that the contents of the above example shader are indented for better legibility.

Key Concepts

Ideally, a designer or artist who is manipulating textures with shader files has a basic understanding of wave forms and knows about mixing colored light (high school physics sort of stuff). If not, there are some concepts you need to have a grasp on to make shaders work for you.

Surface Effects vs. Content Effects vs. Deformation Effects

Shaders not only modify the visible aspect of textures on a geometry brush, curve, patch or mesh model, but they can also have an effect on both the content, "shape," and apparent movement of those things. A surface effect does nothing to modify the shape or content of the brush. Surface effects include glows, transparencies and rgb (red, green, blue) value changes. Content shaders affect the way the brush operates in the game world. Examples include water, fog, nonsolid, and structural. Deformation effects change the actual shape of the affected brush or curve, and may make it appear to move.

Power Has a Price

The shader script gives the designer, artist and programmer a great deal of easily accessible power over the appearance of and potential special effects that may be applied to surfaces in the game world. But it is power that comes with a price tag attached, and the cost is measured in performance speed. Each shader phase that affects the appearance of a texture causes the Quake III engine to make another processing pass and redraw the world. Think of it as if you were adding all the shader-affected triangles to the total r_speed count for each stage in the shader script. A shader-manipulated texture that is seen through another shader-manipulated texture (e.g. a light in fog) has the effect of adding the total number of passes together for the affected triangles. A light that required two passes seen through a fog that requires one pass will be treated as having to redraw that part of the world three times.

RGB Color

RGB means "Red, Green, Blue". Mixing red, green and blue light in differing intensities creates the colors in computers and television monitors. This is called additive color (as opposed to the mixing of pigments in paint or colored ink in the printing process, which is subtractive color). In Quake III Arena and most higher-end computer art programs (and the color selector in Windows), the intensities of the individual Red, Green and Blue components are expressed as number values. When mixed together on a screen, number values of equal intensity in each component color create a completely neutral (gray) color. The lower the number value (towards 0), the darker the shade. The higher the value, the lighter the shade or the more saturated the color until it reaches a maximum value of 255 (in the art programs). All colors possible on the computer can be expressed as a formula of three numbers. The value for complete black is 0 0 0. The value for complete white is 255 255 255. However, the Quake III Arena graphics engine requires that the color range be "normalized" into a range between 0.0 and 1.0.

Normalization: a Scale of 0 to 1

The mathematics in Quake III Arena use a scale of 0.0 to 1.0 instead of 0 to 255. Most computer art programs that can express RGB values as numbers use the 0 to 255 scale. To convert numbers, divide each of the art program's values for the component colors by 255. The resulting three values are your Quake III Arena formula for that color component. The same holds true for texture coordinates.

Texture Sizes

TGA texture files are measured in pixels (picture elements). Textures are measured in powers of 2, with 16 x16 pixels being the smallest (typically) texture in use. Most will be larger. Textures need not be square, so long as both dimensions are powers of 2. Examples include: 32x256, 16x32, 128x16.

Color Math

In Quake III Arena, colors are changed by mathematical equations worked on the textures by way of the scripts or "programlets" in the shader file. An equation that adds to, or multiplies the number values in a texture causes it to become darker. Equations that subtract from or modulate number values in a texture cause it to become lighter. Either equation can change the hue and saturation of a color.

Measurements

The measurements used in the shaders are in either game units, color units, or texture units.

Game unit
A game unit is used by deformations to specify sizes relative to the world. Game units are the same scale we have had since way back in the Wolfenstein days - 8 units equals one foot. The default texture scale used by the Radiant map editor results in two texels for each game unit, but that can be freely changed.
Color units
Colors scale the values generated by the texture units to produce lighting effects. A value of 0.0 will be completely black, and a value of 1.0 will leave the texture unchanged. Colors are sometimes specified with a single value to be used across all red, green, and blue channels, or sometimes as separate values for each channel.
Texture units
This is the normalized (see above) dimensions of the original texture image (or a previously modified texture at a given stage in the shader pipeline). A full texture, regardless of its original size in texels, has a normalized measurement of 1.0 x 1.0. For normal repeating textures, it is possible to have value greater than 1.0 or less than 0.0, resulting in repeating of the texture. The coordinates are usually assigned by the level editor or modeling tools, but you still need to be aware of this for scrolling or turbulent movement of the texture at runtime.

Waveform Functions

Many of the shader functions use waveforms to modulate measurements over time. Where appropriate, additional information is provided with wave modulated directive functions to describe the effect of a particular waveform on that process. Currently there are five waveforms in use in Q3A shaders:

Sin
Sin stands for sine wave, a regular smoothly flowing wave function ranging from -1 to 1.
Triangle
Triangle is a wave with a sharp ascent and a sharp decay, ranging from 0 to 1. It will make choppy looking wave forms.
Square
A square wave simply switches from -1 to 1 with no in-between.
Sawtooth
In the sawtooth wave, the ascent is like a triangle wave from 0 to 1, but the decay cuts off sharply back to 0.
Inversesawtooth
This is the reverse of the sawtooth... instant ascent to the peak value (1), then a triangle wave descent to the valley value (0). The phase on this goes from 1.0 to 0.0 instead of 0.0 to 1.0. This wave is particularly useful for additive cross-fades.
Noise
Available for some functions only (rgbGen wave, but not alphaGen wave), adds a randomly changing value. The frequency value is (probably) hard-coded so changing it (probably) doesn't affect the speed of the flickering.

Waveforms all have the following properties:

base
Where the wave form begins. Amplitude is measured from this base value.
amplitude
This is the height of the wave created, measured from the base. You will probably need to test and tweak this value to get it correct for each new shader stage. The greater the amplitude, the higher the wave peaks and the deeper the valleys.
phase
This is a normalized value between 0.0 and 1.0. Changing phase to a non-zero value affects the point on the wave at which the wave form initially begins to be plotted. Example: In Sin or Triangle wave, a phase of 0.25 means it begins one fourth (25%) of the way along the curve, or more simply put, it begins at the peak of the wave. A phaseof 0.5 would begin at the point the wave re-crosses the base line. A phase of 0.75 would be at the lowest point of the valley. If only one wave form is being used in a shader, a phase shift will probably not be noticed and phase should have a value of zero (0). However, including two or more stages of the same process in a single shader, but with the phases shifted can be used to create interesting visual effects. Example: using rgbGen in two stages with different colors and a 0.5 difference in phase would cause the manipulated texture to modulate between two distinct colors. Phase changes can also be used when you have two uses of the same effect near each other, and you don't want them to be synchronized. You would write a separate shader for each, changing only the phase value.
freq
Frequency. This value is expressed as repetitions or cycles of the wave per second. A value of 1 would cycle once per second. A value of 10 would cycle 10 times per second. A value of 0.1 would cycle once every 10 seconds.

Special Case Syntax

There are a couple of special syntax types to be aware of that may be useful to shader artists.

Documenting With Comments

It is often useful to document parts of shaders, especially for testing purposes or when working with a development team of other designers. Comments added to shaders are ignored completely, making it useful for adding notes, or for temporarily removing directives for testing.

Shaders use the same double forward slashes "//" convention common with many programming languages for commenting. All text after the "//" on the same line are ignored from the shader. Block comments such as "/* , */" from other programming languages do not work. See various shaders for examples.

:q3map Suffix

The :q3map suffix can be added to the end of the shader name of "compile-time" shaders. This means that shaders with names marked with :q3map will be used by the compiler only and will be ignored by the game engine. This is especially useful if you want to add Q3Map2-relevant shader directives to basic lightmapped textures. This allows Quake 3 to generate the stages implicitly while Q3Map2 processes the Q3Map2-relevant sections of the shader. Consider the following shaders:

textures/q3map2_sm/floor
{
	surfaceparm metalsteps
	{
		map $lightmap
		rgbGen identity
	}
	{
		map textures/q3map2_sm/floor.tga
		blendFunc GL_DST_COLOR GL_ZERO
	}
}

Normally, we wouldn't need a shader for this texture to be displayed properly in game since the lightmap and texture stages are generated implicitly by Quake 3. The only reason why this shader exists is because we want Q3Map2 to process this surface as having metalsteps when the player walks on it. As a result, we end up writing out the entire shader.

To save ourselves from needless work, we can mark this shader as being compiler-only and have Quake 3 generate the lightmap and texture stages for us as it usually does by default:

textures/q3map2_sm/floor:q3map
{
	surfaceparm metalsteps
}

Directive Types

Only the shader name is case sensitive (use all lowercase). Shader directives are not, but by convention, you should use mixedCase (ie. q3map_alphaMod). Some of the shader directives are order dependant, so it's good practice to keep all directive of a similar type grouped together in the following order, from top-down:

General Directives

A shader may contain general directives, which affects the shader's appearance globally (affects the entire shader), and changes the physical attributes of the surface that uses the shader. These attributes can affect the player. These parameters are executed at runtime, that is, in order to see the changes one does not need to re-compile the map.

Q3Map Directives

The general directives may be followed by Q3Map directives. These directives are used by the compiler to change the physical nature of the shader's surface and are also global.

Q3Map Surface Parameter Directives

These are actually part of the Q3Map directives that are flags telling the compiler about the physical nature of the shader's surface. These are also global.

Radiant Editor Directives

Radiant editor directives change the appearance of the shader in the map editor only. It has no effect on the shader in game.

Stage Specific Shader Directives

The shader may contain one or more "stages". Each stage consists of an opening and closing bracket along with the contents of the stage (indented). The stage will contain a texture map specification which specifies the source image, and various other stages specific shader directives that modifies the appearance of the source image. Stage specific directives are processed by the renderer. They are appearance changes only and have no effect on game play or game mechanics. Changes to any of these attributes will take effect as soon as the game goes to another level or vid_restarts (type command "/vid_restart" in the game console).