May 30, 2011

Screen-space Lens-Flare in HomeFront?

I consider myself a practical graphics programmer. I believe mathematically correctness is less important than what looks right(or okay) to gamers.

I recently saw an interesting lens-flare technique that goes along with my belief in a game called HomeFront.
In this game, lens-flare effect is a mere full-screen overlay of a bubble patterned image, which is revealed only on the pixels where bright lights are.

Look at my awesome picture below:

So from the left top picture, let's say the yellow part is where bright light is. (and the chance is you probably have some type of HDR buffer already to do effects like bloom.)  Then it'll use the luminance on each pixel as blend factor for lens flare bubble texture, making the final scene reveal bubble pattern on those bright pixels

I found this lens flare technique looks good enough when the high luminance area is small enough. The only time it looked a bit weird was when a large light, such as campfire, covers a lot of pixel spaces, revealing too much bubbles altogether. It almost made me feel like I was doing bubble bath. Hah! But I won't complain. 

Given that HomeFront was made by our sister studio, Kaos, I can probably ask them if my speculation(?) is correct,  But if I do so, I won't be able to write this blog post without going through our legal team. So let's just leave it as my own speculation.

I liked this technique. That's all I wanted to say.

p.s.  I saw this technique on PC version.

May 21, 2011

How to Add Generic Convolution Filter to NVTT

A few month ago, I said I would write a post about how to add a generic convolution filter to NVidia Texture Tool once I get a clearance from our legal team.  And finally they got back to me.

The reason why I added this feature at work was because our artists wanted a sharpening filter on mipmaps.  This feature was present with the original NVTT 1, but removed from NVTT 2.  Given that sharpening filter is a simple 3x3 or 5x5 convolution filter, I've decided to add a generic convolution filter support which can take any arbitrary coefficients. With this approach, anyone can run almost every image processing algorithms based on on convolution.

NVTT Modification
So here's how.  It requires only a few lines of change on 6 files. So I'll just walk you through.

Step 1. Get revision 1277 from NVidia Texture Tools project page.
I haven't tested this on later revisions, but I think it should work unless there were major changes in that source code.

Step 2. Open up src/nvimage/Filter.h and add this constructor.

Kernel2(uint width, const float * data);

Step 3. Open up src/nvimage/Filter.cpp and add this function.
Kernel2::Kernel2(uint ws, const float* data) : m_windowSize(ws)
    m_data = new float[m_windowSize * m_windowSize];

    memcpy(m_data, data, sizeof(float) * m_windowSize * m_windowSize);

Step 4. Open up src/nvimage/FloatImage.h and add this function prototype.
NVIMAGE_API void doConvolution(uint size, const float* data);

Step 5. Open up src/nvimage/FloatImage.cpp and add this function implementation.
void FloatImage::doConvolution(uint size, const float* data)
        Kernel2 k(size, data);
        AutoPtr tmpImage = clone();

        for(uint y = 0; y < height(); y++)
            for(uint x = 0; x < width(); x++)
            for (uint c = 0; c < 4; ++c )
                pixel(x, y, c) = tmpImage->applyKernel(&k, x, y, c, WrapMode_Clamp);

Step 6. Open up src/nvtt/nvtt.h and add this function prototype under struct TexImage.
NVTT_API void doConvolution(unsigned int size, const float* data);

Step 7. Open up src/nvtt/TexImage.cpp and add this function implementation.
void TexImage::doConvolution(unsigned int size, const float* data)
    if (m->image == NULL) return;


    m->image->doConvolution(size, data);

How to Use
How to use this is very straight forward. Assuming you already have a TexImage object named image, you can do this.

const int kernelSize = 3;    // let's use 3 x 3 kernel

// Some random coefficients I found working great for sharpening.
const float sharpenKernel [] = 
    -1/16.0f, -2/16.0f,     -1/16.0f,
    -2/16.0f, 1 + 12/16.0f, -2/16.0f,
    -1/16.0f, -2/16.0f,     -1/16.0f,

image.doConvolution(kernelSize, sharpenKernel);


p.s. I've also emailed the patch file to Ignacio, the creator/maintainer of NVTT project.  Let's see if it ever makes into the codebase. :)

May 19, 2011

Theorycraft = Witchcraft? Maybe

Although I can't deny that posts from a lot graphics programming blogs help us to learn new cool stuff, I'm also often worried about the quality of posts, especially when people claim something not entirely true from a pure "theorycraft" instead of actual experience.  Things that make sense on theory don't necessary make sense in reality, that is.

If you are a decent graphics programmer, you should take only empirical results as truth.

May 16, 2011

Oren-Nayar Lighting in Light Prepass Renderer

This is a conversation I had with another graphics programmer the other day:

  • A: "Using Oren-Nayar lighting is extreme hard with our rendering engine because IT is Light Pre-Pass renderer."
  • Me: "WTF? It's very easy."
  • A: "No. This blog says it's very hard."
  • Me: "Uh... but look at this.  I've already implemented it in our engine 2 years ago, and it was very trivial."
  • A: "OMG." -looks puzzled-
Okay. So I explained to him how I did it. And I'm gonna write the same thing here for the people who might be interested.  (I think the original blog post wanted to say supporting various lighting models is not easy in a deferred context, which is actually a valid point.)

First, if you don't know what Oren-Nayar is, look at this amazing free book. It even shows a way to optimize it with a texture lookup.  My own simple explanation of Oren-Nayar is a diffuse lighting model that additionally takes account of Roughness.  

Second, for those people who don't know what Light Pre-Pass renderer is, read this.

K, now real stuff.  To do Oren-Nayar, you only need one additional information. Yes, roughness.  Then how can we do Oren-Nayar in a Light Pre-pass renaderer?  Save roughness value on the G-Buffer, duh~.  There are multiple ways to save roughness on G-Buffer and probably this is where the confusion came from.

It looks like most light-prepas approaches use R16G16 for G-Buffer to store XY components of normals.  So to store additional information (e.g, roughness), you will need another render target = expensive = not good.

Another approach is to use 8 bit per channel to store normal map, but you will see some bending artifacts = bad lighting = bad bad. But, thanks to Crytek guys, you can actually store normals in three 8-bit channels without quality problem. It's called best-fit normal. So once you use this normal storage method, now you have an extra 8 bit channel that you can use for roughness.  Hooray! Problem solved.

But my actual implementation was a bit more than this because I needed to store specular power, too.  So I thought about it.  And found out we don't really need 8 bits for specular power(do you really need any specular power value over 127?  Or do you really use any specular power value less than 11?)  So I'm using 7 bit for specular power and 1 bit for roughness on/off flag.  Then roughness is just on and off? No. It shouldn't.  If you think a bit more, you will realize that roughness is just an inverse function of specular power  Think this way. Rougher surface will scatter lights more evenly, so specular power should be less for those surfaces and vice versa. 

With all these observations, and some hackery hack functions, this is what I really did at the end.

G-Buffer Storage
  • RGB: Normal
  • A: Roughness/Specular Power fusion
Super Simplified Lighting Pre-pass Shader Code

float4 gval = tex2D(Gbuffer, uv);

// decode normal using crytek's method texture
float3 normal = decodeNormal(;  

float specpower = gval.a * 255.0f;
float roughness = 0;
if (specpower > 127.0f)
    specpower -= 128.0f;
    roughness = someHackeryCurveFunction(127.0f - specpower);

// Now use this parameters to calculate correct lighting for the pixel.

Ta da.. not that hard, eh?  This approach was faster enough to ship a game on Xbox 360 and PS3 with some Oren-Nayar optimization through an approximation.

May 9, 2011

May 3, 2011

Tales of Multiply Map and Gamma Correction

Using a detail map, or multiply map, on top of diffuse map is a somewhat popular way to eliminate the repeating pattern on tiled textures.

I've once implemented this upon an artist's request. The actual blend(?) math I used was:

DiffuseMap * MultiplyMap * 2

The reason I added * 2 at the end was to also give a detail map the ability to brighten the diffuse map.  (without this, it can only darken the base map.)  So the idea was if an artist paint a detail map with half grey, or 0.5, it wouldn't change the base diffuse map at all.

It had worked fine until we decided to change our rendering engine to "gamma-aware". While we were enabling sRGB texture read on certain textures, we accidentally did it on detail maps, too. Then, a month or two passed by, and a few assets were made with this new "gamma-powered" shaders.

Then a "tech-driven" artist found out painting a detail map with the value of 0.5 darkens the texture instead of keeping the base colour. OMG. This was because our new blending formula looked like this:

{ (DiffuseMap ^ 2.2) * (MultiplyMap ^ 2.2) * 2 } ^ (1/2.2)

Because of power of 2.2, it makes 0.5 smaller than 0.25 and multiplying this by 2 gives you a value less than half intensity, 0.5. So I had to fix it by disabling sRGB read on multiply maps. And this is how the blending looks like now:

{ (DiffuseMap ^ 2.2) * MultiplyMap * 2 } ^ (1/2.2)

Better, eh?

Then, I asked artists to change detail maps authored with previous shaders to comply with our new way. To me, converting them was very straightforward.
  1. Open a detail map in PhotoShop.
  2. From main menu, select Image > Adjustment > Exposure.
  3. Set gamma to 0.454545(same as 1 / 2.2).
  4. Press OK
But this time a "visually-driven" artist wanted to change base diffuse maps instead of detail maps because gamma-ing details maps makes them too dark, so it's hard to work with. He asked me what's the easiest way to do this. Unfortunately, I couldn't find a mathematically correct way that always produces the correct conversion, so he had to manually change them while visually verifying the result.

This was probably one of the few cases where mathematical correctness can make things easier than harder, but unfortunately I was not able to find the magic formula.  I just don't think it's possible here.

Or am I just stupid? Does anyone know the correct math here?