What a height map actually does

Parallax, in cross-section

A height map changes no lighting term at all. It changes which texel you read. Cut the surface open and the reason is obvious: your eye is not looking straight down, so the point you can actually see is not the point directly beneath the pixel. Everything below is one slice through a surface, viewed from the side.

Cross-section through the surface —
Drag left and right to move the pixel being shaded
the polygon — where the geometry really is view ray naive sample (straight down) corrected sample (what you can see)
Pixel's own coordinate
—
Coordinate actually sampled
—
Offset applied
—
Texture lookups per pixel
—
The same surface, rendered one row of the height field, seen at the view angle above

Controls viewing & depth
42°
0.14
12

Push depth scale up and the one-step technique falls apart: it assumes the height under the pixel is the height everywhere along the ray, and that is only true for gentle bumps. Turn march steps down and steep parallax visibly terraces.

— what the shader does 

      

    

  


  

    

      The one thing it still cannot do. Every technique here only shifts a texture coordinate before the lighting runs. The polygon is exactly where it always was — the blue line in the cross-section never moves. So the surface gains convincing depth in its interior and keeps a perfectly flat outline, which is why a parallax-mapped brick wall looks superb face-on and gives itself away the moment you can see its edge against the sky. Fixing that costs real vertices: displacement mapping or tessellation.