Here's the code:
float3 normal(float2 uv)
{
float x = IBW * 1.1;
float y = IBH * 1.1;
float2 uv0 = uv + float2(x, 0);
float2 uv1 = uv + float2(0, y);
float d = tex2D(depthSampler, uv ).r;
float d0 = tex2D(depthSampler, uv0).r;
float d1 = tex2D(depthSampler, uv1).r;
float3 v0 = float3(x, 0, d0 - d);
float3 v1 = float3(0, y, d1 - d);
float3 c = cross(v0, v1);
float3 normal = normalize(c);
return normal;
}
For each point on the screen, we look at the one above it, and the one to the side of it. We then construct vectors "going into the screen" by subtracting i.e. the vector AB is b - a. I have simplified this calculation since we already know the differences in x and y, we only need to know their z / depth differences. We then take the cross product of these vectors to get one that is perpendicular to both of them, and normalize it to get it into a sensible range. The result is quite pleasing, but there may be something wrong with my method.
Like I said, I was wrong. But I was almost right. The values of my normals are in the range -1 to 1, when I need them to be in the range 0 to 1 to get that true normal map effect. So add 1 and divide by 2: