The dielectric shader I created earlier in the quarter afforded me an opportunity to re-learn the relevant computer graphics maths, but was not particularly concerned with physical correctness. My blurry reflection and refraction was just due to a large number of samples in a widening cone around the reflection/transmission direction, with all samples weighed equally. As I have continued my explorations with shading, I have discovered that this is not particularly physically-based, nor is it efficient.


For my independent project, I plan to build on these concepts and create a physically-plausible material along the lines of the mia_material_x (mental ray) material. There are several stages to implementing a shader of this type. First, I plan to implement a number of diffuse and specular models to better understand the underlying maths and practice implementation of a microfacet-based BRDF. Next, I will add reflection functionality, but by importance sampling the specular BRDF instead of the standard cone around the reflection vector.


RenderMan Pro Server 16 introduced a number of new features for physically-plausible shading, particularly Multiple Importance Sampling (MIS). MIS-aware shaders follow a somewhat strict structure utilizing RSL 2.0 structs and shader objects. Once I have a basic understanding of the maths for the various components, I will move on to creating the class-based shader and structs required to work with RPS 16's MIS features.


In the several times I have attended SIGGRAPH, the technical papers were always miles over my head. A secondary goal of this project is to spend enough time learning and recalling the relevant computer graphics math so that I can gain a general understanding of a shading-related technical paper from a quick reading.


There are quite a few additional features I plan to implement over the coming weeks and months, including AOVs, BTDF refraction, transluscency, emission, other BRDF models, and smooth integration into Slim (currently not practical since RPS 16 is not integrated into RenderMan for Maya).

THE BRDF AND MICROFACET THEORY

Before I describe the individual models implemented, I want to go over some basic concepts at work. In shading, we assume that the local illumination at a surface point depends on only two vectors - the view direction and the light direction (though there may be multiple lights as we will see later). The Bidirectional Reflectance Distribution Function (BRDF) represents the relationship between the light and view vectors and describes the light response at the shading point based on those two variables. Given a view direction v, the BRDF defines the relative contributions of each incoming (light) direction l. The BRDF is 'bidirectional' because it can also evaluate all outgoing directions v given an incoming (light) direction l. This reciprocal nature is a defining feature of all physically-based BRDFs.


BRDFs are also energy conserving - that is to say the total amount of energy reflected as light is less than or equal to the energy of incident light. This is not the case in some other less involved specular models.


The other key concept at work with physically-based BRDFs is microfacet theory. Suppose that a micropolygon of a surface has a normal n. Microfacet theory suggests that, while n represents a kind of average normal of the surface at that point, it is actually made up of microscopic gouges. These microscopic gouges are themselves made up of tiny microfacets that are each optically flat.



The properties of this microstructure define how the surface will respond to light from a given view direction (or light direction). Given a view direction v and light direction l, using basic reflection math, it is clear that the vector halfway between (h) represents the normal of the surface that reflects v into l (or l into v). In other words, this halfway vector h defines which of the tiny microfacets we are concerned with, given where we are looking from and where the lights are. Based on our input parameters of l and v, we can compute h.





With h defined, we can describe the overall BRDF for this particular view and light direction. The general form of a microfacet-based BRDF, based on a modified Cook-Torrance model [1], is


The equation may seem daunting at first, but each component will be described in simple-to-understand terms without math.


F(l,h) is the Fresnel function. This is present in any shader with reflections, and simply defines how reflections are low when viewed straight on and intensified at glancing angles (for most non-metal surfaces). In most shaders, this is implemented based on the macro-surface normal n, but in the BRDF, we are concerned with the fresnel effect of the microfacets with normal h. The difference is subtle at low roughness but very pronounced as roughness increases.


D(h) is the Distribution function. The distribution of microfacets is defined by this function unique to each BRDF model. This function determines the overall 'smoothness' of the surface, and is most responsible for the size and shape of the specular highlight. This function also has some parameter representing "roughness" and basically tells us what concentration this important vector h has among all of the various facets pointed all over the place. The higher that concentration, the 'smoother' the surface. A result of infinity would mean that all of the microfacets had the same normal (perfect mirror reflection).


G(l,v,h) is the Geometry function. This function accounts for occlusions in the surface. While the distribution term D(h) defines what concentration of microfacets have a normal h, it doesn't define whether or not we can see all of those faces (we can't). The geometry term G has two parts, shadowing and masking. Shadowing means a microfacet is not visible to light direction l and is thus not illuminated and not contributing to the reflection response. Masking means a microfacet is not visible to the view direction v (even though it may be illuminated), and thus is not contributing to the reflection response. In the real world, some of these reflections would bounce around and eventually become visible, but the error is so small as to not be worth the compute time currently.


The denominator 4(n.l)(n.v) is a normalization factor whose description is too involved for this webpage (a complete explanation can be found in Walter's 2007 paper extending the Cook-Torrance model. [2],[3]).

DIFFUSE MODELS

The Lambert model has been around for a couple of hundred years now and correctly describes how an object is lit based on the surface normal and light position (view position does not matter. Lambert's BRDF is just a constant). By observation, even a diffuse surface does not follow the Lambertian model. A good example is a full moon. The moon is lit very evenly and appears from Earth to have a smooth(ish) surface. However, the light does not fall off around the edges in a way consistent with Lambert's model.




From the previous explanation, we know that surfaces are generally made up of microscopic facets. It would be appropriate to assume that Lambert's model is correct - for these small microfacets. The distribution and visibility of these microfacets defines what the macrosurface looks like under observation. In simple terms, the surface does not darken as much around the edges, because if the surface is rough some portion of the microfacets will still be reflecting light at the viewer. In my shader, I have implemented Oren-Nayar diffuse shading [4], a popular model for describing rough diffuse surfaces. In the image above, I am using a roughness of about 0.5 - a value of 1.0 appears as a nearly flat disc (like the moon) but I felt it was too extreme for the moon example image. A roughness value of 0 indicates the surface is completely flat and thus is identical to the Lambert case.

SPECULAR MODELS

Understanding various specular models was one of the main goals of this project. First, I implemented several specular models with punctual/point light sources. At that point, the difference between the specular models is just slight changes to the shape of the fake specular ping. However, I implemented these models in the following order so that I might better understand the underlying concepts before trying to grasp a full physically-based material. All of these models are explained in great detail else on the web, so I will stick with very short descriptions.


I first started with the most basic specular model, Phong [5]. The Phong model attempts to address that surfaces do not simply reflect in one direction, but tend to spread out as an indication of roughness. Phong reflection is one simple equation, where the angle between the reflection and view vectors is raised to some power. The value of the poewr determines the sharpness of the highlight. Below are a couple of examples. This shader is not energy conserving in any way, so as the highlight becomes more blurred, the specular multiplier must be lowered to keep values realistic.


Jim Blinn made a key change to the Phong model by introducing the concept of a half vector [6]. The half-vector is the angle halfway between the view and light vectors. Phong requires the reflection vector to be computed, which is a more expensive operation. Blinn observed that if the half-vector is computed and compared to the surface normal, this was roughly equivalent to comparing the view and reflection vectors. The resulting exponent value is different, but the overall look is very similar to Phong but much cheaper to compute.




The next models I attempted to implement were the Ward isotropic and anisotropic models. Ward sought to create the simplest formula possible to match real-world measurements and as such the Ward model is more complex than the previous Phong and Blinn models. The isotropic version was very straightforward to implement from Ward's 1992 paper [7], but the anisotropic version requires tangent and bitangent vectors orthogonal to the surface normal. This is not straight- forward to compute using built in RenderMan functions, as the shading point only has a dPdu and dPdv function. This only provides predictable numbers with parameterized surfaces, so I used a piece of code from Ali Seiffouri's website to compute the tangent and bitangent vectors. I look forward to spending more time developing my own solution to this problem when I have an opportunity.




Finally, I implemented the Cook-Torrance specular model, or more specifically the modified Cook-Torrance present in Walter et al's 2007 paper. Cook-Torrance is a proper microfacet BRDF specular model, and is of the form described in the BRDF overview section. Unlike the other specular models mentioned before, the Cook-Torrance specular correctly dims out as the roughness increases.




I have opted to leave out the maths for the above specular models, but more detail can be found in the references if interested.

SAMPLING BRDFs

At first, I was eager to use uniform sampling to quickly get some results from my BRDF implementation. To do this, I used a gather() loop simply to generate random ray directions, then I evaluated the BRDF with those random ray directions as l. Many of these samples go through the entire BRDF calculation only to be discarded because they affect a normal of the microfacet structure not visible to the view direction for one reason or another. An interesting side effect of this is that for very rough evaluations, I could get decent renders with reasonable sample counts. As I approached mirror reflection, however, the number of samples required skyrocketed. This is expected, since the area of the reflection is so much smaller for mirror reflections, and I'm still just shooting rays all over the place and most of them are being discarded. Obviously a better method is needed.


To extend these BRDF models from simple punctual light support, the BRDF must be sampled. The first approach is a simple uniform sampling of the entire hemisphere above the surface. To do this, I used a gather() loop to generate random ray directions, which I considered as my light direction l. Many of these samples go through the entire BRDF calculation only to be discarded because they affect a normal of the microfacet structure not visible to the view direction for one reason or another. An interesting side effect of this is that for very rough surfaces, I could achieve smooth renders with reasonable sample counts. As I approached mirror reflection, the number of samples required skyrocketed. This is expected, since the area of reflection is so much smaller for mirror reflections, but I am just shooting rays all over the hemisphere and almost all are being discarded by the BRDF. Obviously a better approach is needed.


To more accurately sample a BRDF, we must effectively sample the distribution term by generating random ray directions that are relevant to the BRDF, as opposed to just firing them at random over the whole hemisphere. This is considered importance sampling the BRDF. The sampling equations are more mathematically involved than I will go into here, but are described in the various papers in great detail. Essentially we generate random microfacet normals (equivalent to the half-vector in the description above) that satisfy the BRDF, and then do the reflection calculation on these normals and the view direction to generate the light directions l. These samples tell the shader 'where to look' so we are not spending precious time sampling areas of the hemisphere that will not contribute to the reflection at that point. Each of these samples is then weighted by a Probability Density Function (PDF), which accounts for visibility and the concentration of facets with that particular half-vector normal. Averaging these samples provides a resulting reflectance with less variance than uniform sampling, and is much, much faster.


Uniform sampling wastes samples in non-contributing areas
Sampling the distribution sends rays in directions important to the BRDF


MULTIPLE IMPORTANCE SAMPLING

One strategy is to sample the BRDF and generate vectors to sample the environment/objects/lights. Another approach is to sample the lights/ environment and provide that information to the surface and evaluate the BRDF. This is particularly useful when lights are very small or very bright. If we only sample the material, we may miss these lights entirely due to the randomness of the ray directions. Worse, we may miss them in one frame and hit them in the next, causing ugly flickering (a problem I am quite familiar with from other projects).


Uniform sampling may miss small but important areas

An environment map is importance sampled by determining which areas of the environment map are brightest and sending the associated ray directions back to the material to use as sample directions. Similarly with very small lights, sending the material information about the position of the light makes sure enough samples go in that direction.


Importance sampling improves accuracy and reduces variance (same number of samples)

In many cases this is a good strategy, but in the case of large light sources, a large number of samples is required, and it can often be more efficient to just sample the material. Combining these two strategies is the basis of Multiple Importance Sampling. In the scope of this project, I used RenderMan's built in physically-plausible lights, which are capable of generating samples to send to the material, as well as applying their light contribution to samples from the material. RenderMan's new RPS 16 integrators take care of the weighing between the light and material samples.

RSL 2.0 - SHADER OBJECTS

Prior to this project, I had never attempted anything involving combined shading or shader objects. RSL 2.0 introduced two new concepts - shader objects and C-style structs. Both provide benefits to the shading pipeline speed.


First, the new shader objects (or class-based shaders) can contain separate methods for displacement, surface, and opacity in the simplest implementation. This allows for the sharing of data between these methods and more efficient execution by breaking the shading process into separate methods. A key example is the opacity method. In RSL 1.0 shaders, the shader may execute all the way to the end, only to end up with an opacity of 0. Malcolm mentioned this as a scenario with shading of tree leaves, and I suspected then that the speed could be improved by solving opacity first, and then wrapping the rest of the surface shader in a conditional based on the opacity being greater than some threshold. In RSL 2.0, the opacity method executes before the surface method, and will not run the surface shader if the opacity is below the threshold. This way we can accomplish the same behavior without ugly changes to the surface shader code. Additionally, values from these internal methods are cached, offering further speed advantages.


A more advanced version of shader objects utilizes multiple functions to break up the lighting process, as well as an initialization function to precompute values and speed up the shading process. This framework, along with user-created functions, is essential to creating a MIS-aware material in RPS 16.



//A simple example of a class-based setup
class rslTwoShader( shader params ) {
	float classVar;
	stdrsl_ShadingContext m_shadingCtx;
	
	public void construct() {
	}
	public void begin() {
		m_shadingCtx->init(); //initialize data struct
	}
	public void displacement() {
		//Runs first, modifies P and the normal
	}
	public void opacity() { //optional
		//Runs next, gives opportunity to exit before shading
	}
	public void prelighting() { //optional
		//Precompute any non-light-specific values for lighting
		//Useful for interactive relighting efficiency
	}
	public void lighting() {
		//Light-position dependent step, run per light
	}
	public void postlighting() { //optional
		//Any post-processing required
	}
	public void userCreatedFunc() {
	}
}

RSL 2.0 also introduced C-style structs. Structs can be used to store functions and data for reuse across the shading pipeline, and mainly serve to organize code and facilitate reuse. In my case, I used several Pixar structs and a custom struct for my final shader. One good example is Pixar's ShadingContext struct, which stores a myriad of data about the current shading point and provides many utility functions for dealing with that data. The ShadingContext struct is intialized in the begin() method of a class-based shader, and can be used throughout the shading pipeline for easy access to ray depth, the faceforward normal, tangents, etc.

RPS16 PHYSICALLY-PLAUSIBLE-SHADING

RenderMan Pro Server 16's physically-plausible shading makes use of both structs and class-based shaders. These shaders are constructed like any other, but utilize new lighting integrators and new required functions for multiple importance sampling.


First, an overview of the shading process with respect to multiple importance sampling. I have already covered how in some cases it is better for the material to generate the sample directions, and in other cases it is better for the light to provide sample directions to the material. With both lights and shaders, two new functions must be defined in RPS 16 to work with the MIS system.


The generateSamples() method is defined in both the material and light, and is used to store the response of that portion of the final result. In the case of the light, generateSamples() stores the sample directions and the light radiance. In the case of the material, it stores the sample directions and the material response at that direction (the evaluation of the BRDF and the PDF, but not the lighting).




//used as a part of the full shader object
public void generateSamples(string distribution;
		output __radiancesample samples[]) {
	
	//distribution = "specular", "indirectspecular", etc.
	
	if(distribution == "specular") {
		/* append to the samples array - if it already has any,
		they are from the lights */
		
		color materialresponse = 0;
		float materialpdf = 0;
		for(i = start_size; i < size; i+=1) {
			//do my whole BRDF calculation, resulting in:
			matresp = refl * (F*G*D*)/4*(Vn.Nn)*(Ln.Nn);
			matpdf = (D*(H.Nn))/(4*(H.Vn)); //see paper for details
			//store this material response, pdf, light dir and distance
			samples[i]->setMaterialSamples(matresp, matpdf, Ln, 1e30);
		}
	}
}

Next, the evaluateSamples() method must be defined for both the material and the light. In the case of a light, evaluateSamples takes the samples generated by the material (already containing the material response and pdf), and adds the light radiance for that sample direction, thus creating a full sample with lighting and material response. In the case of the material, the sample direction already contains information about the light radiance, and material response and PDF are added to create a full sample.


These samples are stored internally by RPS 16 in a radiance cache and can be reused in the new diffuselighting() and specularlighting() shader methods, delivering speedups in some cases.



float evaluateSamples(string distribution;
	output _radiancesample samples[]) {
	if(distribution == "specular") {
		for(i = 0; i < num_samps; i+=1) {
			//direction provided by light generateSamples()
			vector Ln = samples[i]->direction;
			
			//evaluate BRDF as above
			samples[i]->setMaterialResponse(matresp,matpdf);
		}
	} else if (distribution == "diffuse") {
		//implement lambert or oren-nayar diffuse here
		//right now diffuse is only using light samples since it is
		//inefficient to sample the whole hemisphere of the material
		for(i = 0; i < num_samps; i+=1) {
			float cosTheta = (samples[i]->direction).Nn; //Lambert
			if(cosTheta > 0) {
				matresp = cosTheta * diffuseColor / PI;
			}
			samples[i]->setMaterialResponse(matresp, 0); //pdf = 0
		}
	}
}

There are also two new integrators, directlighting() and indirectspecular(), where these samples are put to use. These functions invoke the generateSamples() and evaluateSamples() functions of both the materials and lights, and internally handle the multiple importance weightings. The directlighting() function includes a "heuristic" parameter to adjust the balance between light and material samples based on different research.



public void lighting(output color Ci, Oi) {
	__radiositysample samples[];
	
	directlighting(this, getlights(), "mis" 1, "heuristic", "veachpower2",
			"materialsamples", samples, "diffuseresult", diff_col,
			"specularresult", spec_col);
	color indir_spec = indirectspecular(this, "materialsamples", samples);
	color indir_diff = indirectdiffuse(P, Nn, num_diffuse_samps);
	
	Ci += (Kd * (indir_diff + diff_col) * (1-fresnel)) + spec_color + indir_spec;
}

COMPARISIONS

Here are some comparisons of some traditional raytracing shaders versus this MIS BRDF-based approach. First the Lambert diffuse:


Click to enlarge

First, I should address that the environment map is an unclipped very high dynamic range image that is rotated differently in the MIS example than the gather loop example. This is due to my use of the plausibleEnvlight shader for the MIS example, and I did not have the time to find the matching angle by hand.


Now, the gather loop is a simple gather call over the entire hemisphere above the point, where the result is averaged and then multiplied by the diffuse color. This unclipped map has many many small light sources, and even with 512 samples I am completely missing some of those lights. This would be horrible in motion, but even as a still it is clear I would need at least 8 or 10 times as many samples to get a truly smooth image.


The two MIS examples are much smoother right away, and also display proper shadowing behavior from these small individual light sources. This is from the shadowSamples() method of physically-plausible lights that I am not covering in this writeup. Additionally, I am not calling indirectdiffuse so this can be considered just direct lighting from the environment. The reason for the increased smoothness is that in this implementation, the material itself does no diffuse sampling of the hemisphere and instead just asks the light for samples - in this case up to 128. The light provides useful samples via the generateSamples method, and a superior result is produced.


Click to enlarge

In this example, the MIS version is making use of both the light and material samples. While the result is still a bit grainy, it is far superior to anything I could accomplish with the gather shader. The gather shader in this example is my dielectric shader from earlier in the quarter, which collects samples in a cone about the reflection vector. It is clear that this approach is insufficient, as well as very noisy even with 256 samples. Like the diffuse, many many samples would be required to resolve this environment map smoothly.

AREAS FOR IMPROVEMENT

There are still many, many components I plan to add to this shader. A great deal of time was spent just learning the math concepts involved, then understanding RSL 2.0 shading and physically-plausible shading pipelines.


As time allows, I plan to continue developing the shader to include AOVs, subsurface scattering, translucency, refraction, emission, newer BRDF models, and smooth the integration with Maya.


Longer term goals include full integration of this shader into RenderMan Studio and use in actual projects, as well as potentially expanding the shader to support the use of physically-measured real world BRDF data.

FINAL THOUGHTS

Unfortunately, due to the short time available for this project, I did not get as far as I would have hoped with the shader implementation. However, I have gained a great deal of understanding of how shading works in general and how RenderMan handles complex shading models - a secondary goal of mine. I feel comfortable with most of the math I have come across in several technical papers and am confident moving forward that I can put this new knowledge to good use for future shading endeavours.

REFERENCES

[1] - A Reflectance Model for Computer Graphics
[2] - Microfacet Models for Refraction through Rough Surfaces
[3] - Microfacet Models for reflection and refraction - slides
[4] - Generalization of Lambert's Reflectance Model
[5] - The Phong Reflection model
[6] - Models of light reflection for computer synthesized pictures
[7] - Measuring and Modeling Anisotropic Reflection
Some slides from Cornell about MIS