Why the obvious metric is the wrong one
SSIM against MSE
Mean squared error asks "how far is each pixel from where it should be?". That is not the question a viewer asks. Damage this image in different ways, hold the MSE constant, and watch SSIM refuse to agree — because one kind of error destroys structure and the other merely moves it.
The match button solves for the uniform brightness shift with the same MSE as whatever you have set, then applies it instead. If MSE were a good perceptual metric the two would look equally bad. They do not.
SSIM is the product of three separate comparisons. Watching which one collapses tells you what kind of damage was done — something a single MSE number can never express.
| Distortion | Strength | MSE | PSNR | SSIM | Verdict |
|---|
Every row is tuned by binary search until its MSE matches the target, so MSE ranks all five as equally damaged. SSIM spreads them across most of its range. Click any row to load it into the sliders above and look at it.
Why MSE fails. It is a sum of independent per-pixel penalties, so it has no notion of a neighbourhood, an edge, or a texture. Shift every pixel up by 40 and MSE reports catastrophe — 1600 — even though the image is perfectly intact and merely brighter. Blur the image until MSE reads the same 1600 and the detail is genuinely, irrecoverably gone. One of these is a display calibration issue; the other has destroyed information. MSE cannot tell them apart.
What SSIM does instead. It slides an 11×11 Gaussian window over both images and, at every position, compares three things separately: mean brightness (luminance), the spread of values (contrast), and how the deviations correlate (structure). The brightness shift moves only the first term. Blur attacks the second and third. Because the three are multiplied, damage to structure cannot be compensated for by getting the brightness right — which is exactly the behaviour a perceptual metric needs.
Why this matters for super-resolution. A network trained to minimise MSE is being explicitly rewarded for hedging. When it cannot tell whether a region should be a sharp edge or a flat patch, the MSE-minimising answer is the average of both — a blur. That is why early super-resolution results looked soft, and why later work moved to perceptual and adversarial losses that penalise implausible structure rather than distant pixels.