Sunday, 9 November 2014

fudging figures (and fooling spec sheet gazers)

I know that people don't usually have a head for numbers, and specifications get bandied about with gay (numerically illiterate) abandon by both gear-heads and testers alike. The sad fact is that by fiddling the books, sales departments can pull the wool over the eyes of blind wangers who only read spec sheets (but don't understand them or look into the details). One that I happen to find amusing at the moment is the comparison of latest model digital cameras and the older versions. To explore this I'll discuss three cameras using the specs from DxO:

I happen to like DxO as a site for data, I've already discussed their P-Mpixel metric (and that I find it useful) but their technical camera data is very handy too, especially for seeing through the bullshit. Everyone on the interwebforums will tell you about how good the Olympus OM-D is, and if you read the various kool-aide sites they would have you believe that the OM-D was the first proper SLR micro43 and somehow the best micro43 camera. While it sure is a good camera the reality is that nothing much has really changed in terms of image quality since way back in July 2009 when Panasonic released the GH1 and set the bar. The problem for Panasonic is / was they made a camera which was really too good for the time, and perhaps (to quote a phrase) feeding truffles to the pigs. Who knows, perhaps they needed to. Just quickly, while I've tossed the GX7 in here for good measure I think its clear that what I say about the OM-D applies equally to it.

The first thing of importance when discussing the specs is that Panasonic actually delivered more on their ISO then they claimed. Quite stupid of them from a strategic marketing point of view, but they were facing an uphill battle at the time as people still didn't get the concept. Looking at the DxO ratings this is clear:

The actual measured ISO of the GH1 set to 100 is above the ISO100 level. Comparing this to the OM-D (which doesn't even have a 100ISO setting) we see that when the OM-D is set to 200 its measured ISO is actually about equal to the GH1 set to 100.

This difference in ratings vs measurements continues along the scale, with the measured ISO being better than the next stop up the scale of the OM-D. So much so that at ISO 400 the OM-D needs to be set to 1600 to be as sensitive to light at the sensor on the GH1. Indeed hovering over the GH1 point with the camera set to ISO400 gives a measured ISO = 591 while the OM-D needs to be set to 1600 to be closer to that (in fact its measured at 782 there which is about 1/3rd a stop more, but 800 is still under the GH1 at a measured 394)

Essentially this means you can get a higher shutter speed on the GH1 without going up an ISO step.

This of course has an effect on the specs for dynamic range and the Signal to Noise Ratio (SNR) which are dependent on ISO.

When you load the graph and hover the tooltip over the pints you see that there are no equal settings, but:
GH1 at a measured ISO of 137 = SNR of 36.8dB while the OMD @ 214 = 36.2dB

Its confusing that DxO put a 100ISO blob in this graph for the OM-D, but its significant that the lower axis is measured ISO not set ISO ... important. The interesting thing there is that the OM-D would appear to lead the GH1 but that seems to change further down the graph at around 200ISO and onwards.

So while the specs of the OM-D exceed the specs of the GH1, when you equalise back to what the setting actually means they are essentially on par.

The same goes for the the Dynamic Range

Where at the measured ISO point of somewhere around 1000 ISO the now elderly GH1 is performing better than the OM-D

Its certainly not the chalk and cheese that the bleating (you know, the sound sheep make is called bleating) one reads about how good the OM-D is and how much better all the new cameras are ...

As I discovered back in May when comparing my GH1 to my friends OM-D, there really isn't much in the whole upgrade to make it worth doing for me. While the out of camera JPG's from the OM-D seemd much better than the GH1

GH1 on the left OM-D on the right.

The OM-D image just leaps off the screen as being sharper and more contrasty.
However, the raw files processed up to appear almost identical (as indeed the specs would imply they should).

If you happen to like the OM-D physically to hold or (it seems more important now-a-days) how it looks and how it feels in your hand, then by all means the OM-D is a fine camera (I'm not degrading it). But if you just want a tool to do the job and cost benefit means anything to you then at under $200 on the second hand market, to me, the GH1 is still competitive and today represents bargain buying .


HF said...

In my opinion, the difference in defining ISO is not mattering that much for comparing sensors. To compare at equal values, just interpolate in between to compare at the same setting. Then you see S/N ratio is almost on par, whereas DR is in favour of OMD. The definition of the sensitivities is somewhat arbitrary, but DXO accounts for this by not plotting the data over the specified, but over the measured ISO. This means data is perfectly comparable if you just compare the separation of the measurement curves.

obakesan said...

Exactly my point HF, perfectly comparable and when you equalise the baseline difference they are equal to each other - not generations ahead are the marketing people would attempt to fool you into believing