Dev/Shader Assembly

From RibTools

Before reading

This article is relative to the internal implementation of shaders in RibTools. The intended audience for this article is a technical one. This is about writing a shader system, not about writing shaders.

What's explained below might be interesting for programmers, but it is not necessary for those that simply want to render a scene or write shaders in the RenderMan shading language.

Overview

In RibTools, classical high level RenderMan shading language (RSL) shaders are implemented by means of a shader compiler entirely written in C++ (no Lex/Yacc of sort).

The compiler produces in output an intermediary assembly source (file extension .rrasm) that is very close to the internal structure of the shader system's Virtual Machine.

When rendering a scene, RibRender will firstly look for a compiled version of the shader. If the compiled version is not found, the RSL source file (.sl extension) is loaded and compiled automatically.

The assembly version created by the shader compiler assumes the extension .autogen.rrasm. The .autogen naming convention here is used to avoid ambiguities between hand written .rrasm files that need to be preserved and compiler-generated files that can be overwritten at will by the compiler issued at rendering time.

RSL is an high level language whose syntax is similar to 'C' which is better explained elsewhere.

What follows is a rough description of the RRASM format. Notice that RRASM is mostly meant as an intermediary internal format and is therefore subject to continuous changes as the renderer and the shader compiler evolve.

The RRASM Assembly

Assembly is a friendlier representation of machine code. The machine code represented by RRASM is virtual. High level shader code is discretized by the compiler which produces simpler operations that can be executed by the shader's virtual machine.

By "virtual machine" (VM) here we intend a "Process virtual machine", a VM dedicated at reproducing a process' flow, not at simulating a whole computer architecture.

RRASM files are text files and are parsed one line at the time. Each line of text can be of any of the following types:

• An assembler directive
• A shader-type keyword
• A label
• Data definition
• Code instruction

Here is an overview of an RRASM definition in context (following code is representative of an actual shader. As it is, it doesn't actually compile):

.data    // <- an assembler directive
   // data definition follows
   P          global    varying    point
   N          global    varying    normal
   Km         parameter uniform    float
   myConstVal temporary constant   float    0.1

.code   // <- an assembler directive
   // code definition follows

displacement // <- shader type keyword

__defparam_Km:  // <- a (special) label
   mov.ss         Km     myConstVal 
   ret

__main:         // <- a (special) label
   normalize $v1  N

   [...]

loop1:          // <- a (normal) label
   mul.vvs $v0 P totalFrequency

   [...]

   cmplt $su1 $su2 loop1
   calculatenormal N P

   ret

Assembler directive

Assembler directives purpose is to guide the assembler during its operation. They are represented by strings that start with a dot (".") character.

Currently supported directives are:

.data

Text below is for data definitions

.code

Text below is for code definitions

Each RRASM file can only contain one of the above directives each. They must appear in the .data and .code order.

The .code directive must be followed by the shader-type keyword before the code definition can begin.

Shader-type keyword

A shader-type keyword reflects shader types as defined by the RenderMan Interface specifications:

light

A light shader. Defines color and positioning of a light source.

surface

A surface shader. Defines color and opacity of samples composing a surface. (This is the most common type of shader.)

volume

A volume shader. (Currently unsupported)

displacement

A displacement shader. Defines displacement to the surface being shaded.

transformation

A transformation shader. (Currently unsupported)

imager

A post-processing shader that works at the pixel level for operations such as color-correction. (Currently unsupported)

Label

A label is a place-mark. Standard labels are for code flow to refer to when jumping. Special labels are for interfacing directly with the renderer.

All labels are alphanumeric characters that end with a colon character (":"). Special labels are preceded by two underscore characters ("__").

There are currently two kinds of special labels:

__main

This defines where the shader program starts. There can be only one __main label in an RRASM file.

__defparam_*:

This defines the beginning of a chunk of code to be executed to initialize a shader parameter (the asterisk is a placeholder for an actual parameter name). Code following this label will be executed every time, before the shader main code is executed, only if such parameter was not provided in the scene definition file where the shader is being invoked.

Data definition

A data definition line declares symbols (variables or constants) that will be used through the shader code. Each definition requires 4 distinct fields for non-constant symbols, and 5 for constant symbols.

Example:

Symbol Name

The name of the symbol being defined

Storage Scope

This must be one of the following:

• parameter
  • A parameter that can be passed from the shader caller
• temporary
  • An uninitialized temporary value used within the shader
• global
  • A global symbol matching that of the renderer's state machine (e.g. Oi, N, etc.)

Storage Detail

This must be one of the following:

• constant
  • A constant value. It must have a temporary Storage Scope and must specify a value
• uniform
  • The symbol is unique across all the samples being shaded (e.g. a color for a surface with an uniform tint)
• varying
  • The symbol is changing across all the samples being shaded (e.g. a color that changes across a surface)
• vertex
  • The symbol is changing at the geometrical vertices as represented in the primitive definition in the scene file (.rib extension).

Storage Type

This must be one of the following:

float

A scalar floating point value

point

A point in 3D space

hpoint

An 4D homogeneous coordinate point (e.g. A point in projective space)

color

A color. (Unlike the RenderMan specifications, RibTools currently only supports RGB colors)

string

A string of text

vector

A 3D vector representing direction and magnitude

normal

A 3D unit vector representing direction

matrix

A 4x4 matrix

Note that for practical purposes, point, vector, normal and color are currently all stored and referenced by the RRASM code in the same manner.

Constant Value

This field is necessary for symbols with constant Storage Detail.

float constant values are a single float numerical values:

   myFloat temporary constant float 1.0

Other non scalar are a list of individual float values separated by one or more white-spaces:

   myPoint  temporary constant vector 1.0 2.0 3.0

Strings must be wrapped between quotation marks:

   myString temporary constant string "Hello !"

Code Instruction

RRASM instructions are composed of an operator and possible operands separated by white spaces.

Operator definition

Often (but, currently, not always) operators explicitly specify the type of operands that they act on.

For example, in:

   sub.vvv $v0 $v1 $v2

"sub.vvv" is the complete operator.

The "sub" portion tells the type of operation: a subtraction.

The three vs following the dot, tell that there are three operands all of vector type.

Here is a list of the type specifiers:

Notice that matrix and 4D vector types are not yet supported. Vector is 3D vector and is used to deal with point, vector, normal and color alike.

The type specifiers that an operator can take vary depending on the case. Some are not plausible and therefore not implemented.

For example:

   sub.svv $s0 $v1 $v2 // Bad: scalar <- vector

Would subtract two vectors and store the result into a scalar. This is not possible and therefore not implemented. Something like:

   sub.vvs $v0 $v1 $s2 // OK: vector <- scalar

..is instead plausible. In this case the same scalar value is subtracted from all the components of the vector $v1, like in the following pseudo-code:

   $v0.x = $v1.x - $s2
   $v0.y = $v1.y - $s2
   $v0.z = $v1.z - $s2

Operand definition

Operands can be symbols (defined in the .data section) or readily available virtual registers.

A virtual register assumes the following form:

$<type specifier>[<uniform storage specifier>]<unique number>

A type specifier is the same used in operator definitions (see above).

Uniform storage specifier is optional and it's used to specify that a register is uniform. Without that, a register is assumed to be varying.

For example:

$s1 is a scalar varying register

$su1 is a scalar uniform register

List of operators

mov.[ss|vs|vv] a b

Copies the value of b into a

a = b

mov.vs3 a b c d

Copies the scalar values b, c and d into the vector a

a.x = b

a.y = c

a.z = d

abs.[ss|vs|vv] a b

Stores the absolute value of b into a

sign.ss a b

if b < 0 then a = -1 else

if b > 0 then a = 1 else

a = 0

add.[sss|vvs|vsv|vvv] a b c

a = b + c

sub.[sss|vvs|vsv|vvv] a b c

a = b - c

mul.[sss|vvs|vsv|vvv] a b c

a = b * c

div.[sss|vvs|vsv|vvv] a b c

a = b / c

pow.sss a b c

a = pow( b, c )

dot.svv a b c

a = dot_product( b, c )

length.sv a b

a = length( b )

min.[sss|vvv] a b c

a = min( b, c )

max.[sss|vvv] a b c

a = max( b, c )

ld.s a b

Loads the immediate scalar value b into the scalar a

ld.v a b c d

Loads the immediate scalar values b, c and d into the vector a

cmplt.ssl a b c

if a < b then goto c

a and b must be scalar and uniform

c is a label

setle.bss a b c

a = (b <= c)

Set a to true if the condition is satisfied or false otherwise

setge.bss a b c

a = (b >= c)

Set a to true if the condition is satisfied or false otherwise

setlt.bss a b c

a = (b < c)

Set a to true if the condition is satisfied or false otherwise

setgt.bss a b c

a = (b > c)

Set a to true if the condition is satisfied or false otherwise

seteq.[bss|bvv|bhh|bmm|bxx|bbb] a b c

a = (b == c)

Set a to true if the condition is satisfied or false otherwise

setne.[bss|bvv|bhh|bmm|bxx|bbb] a b c

a = (b != c)

Set a to true if the condition is satisfied or false otherwise

noise.[ss|sv2|sv|vs|vv2|vv|sv] a b

Calculates the Perlin noise of b and stores it into a

ycomp.sv a b

Copies the y component of the vector a into the scalar b

zcomp.sv a b

Copies the z component of the vector a into the scalar b

setxcomp.vs a b

Set the x component of the vector a to the value b

setycomp.vs a b

Set the y component of the vector a to the value b

setzcomp.vs a b

Set the z component of the vector a to the value b

pxformname.vxv a b c

Transforms the point c into the space b and stores it in a

vxformname.vxv a b c

Transforms the vector c into the space b and stores it in a

nxformname.vxv a b c

Transforms the normal c into the space b and stores it in a

cxformname.vxv a b c

Transforms the color c into the space b and stores it in a

normalize(.vv) a b

Normalize the vector b and stores the result in a

faceforward(.vvv) a b c

a = b * sign( -dot_product(c, Ng) )

Calculates the normal facing the camera.

Ng is a global symbol that represents the current primitives geometric normal

ambient(.s) a

Calculates the total amount of ambient light and stores it in a

calculatenormal(.vv) a b

Re-calculates the normal at point b and stores it in a

This is used in displacement shaders.

solarbegin

solarbegin.vs a b

Marks the begin of a solar block

illuminance.v a

illuminance.vvs a b c

Marks the begin of an illuminance block

funcopend

Marks the end of a solar or illuminance, or if/else block

iftrue.b a

Jump to the matching orelse or funcopend operand if a is false

orelse

Marks a jump point below an iftrue.b operator

texture.[sx|vx] col texname

col = texture( texname )

Fetches a texture sample from texname at the current s,t coordinates

texture.[sxss|vxss] col texname s t

col = texture( texname, s, t )

Fetches a texture sample from texname at the specified s,t coordinates

ret

Returns to the caller