One Number To Measure Them All!

2012.10.05

By now, most people interested in image quality have come across DxOMark, the camera measurement branch of DxO. DxO has been making highly-acclaimed software for years and - since they had to measure image characteristics anyway - made the jump to publish image quality ratings for RAW-capable cameras and lenses.

When cameras are measured, they are given a DxOMark score. One number which is published and repeated all over the web with great fanfare to show that one camera is better or worse than so many others. Just like the number of megapixels in a camera is often erroneously equated with image quality, so is the DxOMark score given to cameras.

The first thing to know is that no single number can ever describe the image quality of a camera. Image quality is simply too complex for that. DxO knows that and actually publishes considerably more data about each camera, only most people just compare the final score.

Take a look at the three digital cameras with a DxOMark score of 82 as of today:

These are:

• Pentax K-5: A sub-$1000 USD DSLR with 16 megapixels APC-sized sensor.
• Nikon D3S: A $5000 USD DSLR with a 12 megapixels full-frame sensor.
• Pentax 645D: A $9000 USD Medium-Format (0.7X crop) camera with a 40 MP sensor.

There is almost a factor of 10X in price difference among these 3 cameras, yet they get exactly the same score! Is there a point then to spend an extra $8000 for something that gets the same image quality? Not to mention more expensive lenses and extra bulk associated with a medium-format or full-frame DSLR.

Part of the answer to the previous questions lies in the other numbers published by DxO. You see, the DxOMark score is a weighted average, so cameras which perform dramatically differently can get the same overall score. Case in point, the three cameras above:

• The 645D achieves a color-depth of 24.6 bits per pixel which is what lifts its score the highest. This one extra bit (or almost) over the K-5 and D3s means that the 645D can distinguish twice the number of colors as the other two cameras. Yes, one bit makes that big a difference!
• The K-5 achieves a dynamic-range of 14.1 or 1.5 to 2 stops above the other two cameras. This equates to capturing a scene with 225% to 400% more contrast! Again, this huge difference lifts the K-5s score the most.
• The D3S, lagging behind in terms of bit-depth and dynamic-range, has its score lifted by a stellar performance at high-ISO. Its score of 3253, compared to 1162 and 1262, means that it produces much cleaner results at high sensitivities than the other two cameras.

A quick look at three digital cameras with a DxOMark score of 81 shows a similar balance:

Among these three models which have almost exactly the same color-depth, the full-frame models unsurprisingly leads the pack in terms of high-ISO while the two APS-C model catch up with greater dynamic range.

All this is not to say that DxOMark numbers are useless. No, every number DxOMark publishes has a precise meaning but one has to know what that it in order to make a comparison. DxOMark is in fact extremely rigorous and consistent with its measurements. So, with a good understand, individual numbers acquire meaning!

Finally, it is important to understand that there are so many aspects of image quality which are not measured by DxOMark and those have to be seen in images and results from each camera for now.

Now you should all know which of those cameras to buy for landscape, color-critical or low-light work :)

...and again, this assumes that only image quality matters, not ergonomics, features or available lenses. That's why digital camera reviews are still immensely popular.

• Digital Camera