M8 vs D200 DNG files

S

Stan98103

Guest
I did some testing with conversion of the M8 dng file from digital outback to tiff files versus conversion of raw files from my Nikon D200. Here are the results.

The M8 dng file from Digital Outback is 10.6 Mb. When I open this dng file in CS2 using Adobe Camera Raw with auto turn off and no other adjustments and save it in PSCS2 as an uncompressed tif file the size is 60 Mb.

A typical nef file from my D200 is around 16 Mb. If I convert a typical D200 nef file to a dng file using Adobe Digital Negative Converter (ver 3.4x89 with preference JPEG Preview none, Compressed lossless, Preserve Raw image, Don’t embed original), the resulting dng files are typically around 9 Mb. When I open either an original D200 nef file or the corresponding converted D200 dng file in ACR and save it in PSCS2 as an uncompressed tif file the resulting files are indistinguishable (overall appearance, file size, information on the histogram palate) and the file size is typically around 37 Mb.

I have repeated the above conversions with similar results on a dozen other M8 dng files (firmware ver 1.06) that I shot myself at a Leica demo day on Saturday at Kenmore Camera in Seattle. Those files also started life as 10 Mb dng files and ended up as 60 Mb uncompressed tiff files by the procedure outlined above.

Although the M8 dng files that I converted are different images than those from my D200 nef files, it seem like the M8 dng files somehow hold more information as reflected in the larger size of the resulting tif files (60Mb for M8 vs 37 Mb for D200). This is most probably because M8 dng files are 16-bit whereas Nikon nef files are only 14-bit.

Here is one more curious result from the M8 dng files which occurs for both the Digital Outback dng file and the M8 dng files that I shot myself. If an original dng file is processed with Adobe DNG Converter (ver 3.4x89 with preference JPEG Preview none, Compressed lossless, Preserve Raw image, Don’t embed original), the resulting dng file is only around 7 Mb! If that 7Mb compressed dng file is then opened in ACR and saved in PSCS2 as an uncompressed tiff file it is indistinguishable (overall appearance, file size, information on the histogram palate) from the tiff file produced by the original M8 dng file. Maybe that’s not surprising since the conversion is designated as “lossless”.

So how can Leica get 10Mb of 16-bit data into a 10Mb dng file? My guess is that they have developed some extremely powerful lossless compression algorithms for writing the original dng files in the camera. It also is interesting that these dng files can be further compressed by about 30% in a lossless manner with Adobe DNG Converter.

I’ll post my general impressions of the hour a spent with the M8 later this week.

Cheers,

Stan

BTW, I find it highly unlikely that the Leica dng files are 8-bit rather than 16-bit as purported in some threads. Would any car company launch a new car that it was staking its reputation on claiming that it had 8-cylinders when it really only had 4? Leica knows people will be looking under the hood.
 
Last edited by a moderator:
Stan98103 said:
BTW, I find it highly unlikely that the Leica dng files are 8-bit rather than 16-bit as purported in some threads. Would any car company launch a new car that it was staking its reputation on claiming that it had 8-cylinders when it really only had 4? Leica knows people will be looking under the hood.
Click to expand...

Here are some details from an M8 DNG file at this page:
http://www.outbackphoto.com/reviews/equipment/leica_m8/Leica_M8_review.html

Make: "Leica Camera AG"
Model: "M8 Digital Camera"
Software: "1.06"
DNGVersion: 1.0.0.0
UniqueCameraModel: "M8 Digital Camera"

NewSubFileType: Main Image
ImageWidth: 3920
ImageLength: 2638
BitsPerSample: 8
Compression: Uncompressed
PhotometricInterpretation: CFA
 
The Leica rep in Chicago this weekend specifically stated the M8 that will ship is a 16- bit camera that is 10MP and writes 10MB files to the card.
 
Michael Reichman claims it is not 16 bit. But is this in any way important if the files are as good as it gets - which they are.?
 
Stan98103 said:
When I open either an original D200 nef file or the corresponding converted D200 dng file in ACR and save it in PSCS2 as an uncompressed tif file the resulting files are indistinguishable (overall appearance, file size, information on the histogram palate) and the file size is typically around 37 Mb.

I have repeated the above conversions with similar results on a dozen other M8 dng files (firmware ver 1.06) that I shot myself at a Leica demo day on Saturday at Kenmore Camera in Seattle. Those files also started life as 10 Mb dng files and ended up as 60 Mb uncompressed tiff files by the procedure outlined above.

...

BTW, I find it highly unlikely that the Leica dng files are 8-bit rather than 16-bit as purported in some threads. Would any car company launch a new car that it was staking its reputation on claiming that it had 8-cylinders when it really only had 4? Leica knows people will be looking under the hood.
Click to expand...

Hi,

I don't think the size of an uncompressed tiff file is a good measure of file quality - it really can not say anything more than pixel count since any normal tiff files always use either 3x8 bits or 3x16 bits per pixel (for colour images) regardless of how many bits the RAW file had or the RAW-converter was working with.

To really get some information about what is in those RAW files we shuld look at the uninterpolated data contained in them. I think dcraw can help us with that:

The following uses dcraw to dump Reichman's M8 DNG to a linear B/W 16-bit per pixel b/w ppm file without any colour space conversions. Then I count the number of unique levels in the ppm file (the number should be expected to be large but no more than 65536) with ppmhist. ppmhist also outputs two extra lines that get counted so the M8 file contains 255 levels which is extremely low for a file with 16 bits per pixel.
Code: 
dcraw -v -4 -D -o 0 0610R8_0039.DNG

anders@legolas:~/tmp$ ppmhist --sort=rgb 0610R8_0039.pgm |wc -l
257

As a comparison a RAW file from my 30D gives shows a much more expected value (here the maximum number possible is 4096).
Code: 
dcraw -v -4 -D -o 0 img_1977.cr2

anders@legolas:~/tmp$ ppmhist --sort=rgb img_1977.pgm |wc -l
3728

Judging from this small sample I'd say that there is either something fishy with the M8 DNG file or with dcraw's ability to decode it beacuse it is certainly behaving as if it was 8-bit data in it. If someone has more M8 DNG files then we could perhaps get a better picture - maybe this scene really didn't contain more levels.

Cheers,

Anders
 
Last edited:
jaapv said:
Michael Reichman claims it is not 16 bit. But is this in any way important if the files are as good as it gets - which they are.?
Click to expand...

It is important because why does Leica claim the image is 16bit? Shady.
 
My suspicion, and that's all it is, is that the DNG files are 8-bit (as several people have inferred both here and on LUF, plus MR on LL) but that the demozaicing algorithm generates 16-bit precision interpolation. This will produce data at intermediate levels (between the 256 8-bit levels). Maybe Leica have hit on the (?) fact (my speculation) that this creates enough information (8-bit DNG plus that contained in the algorithm) to resist posterization etc under any reasonable degree of PP.
 
Last edited:
If it's only 8 bits, then it will be very important to get the WB correct in-camera.

However, I'm not convinced it's 8 bits. It's quite possible that the high bits are 8-bit gamma encoded, while the low bits are huffman coded in a separate block (possibly with a proprietary tag). This is the same scheme Canon uses in CRW, except the opposite. The advantage of doing it the opposite way is that any DNG decoder can make sense of it, albeit only as an 8 bit file.
 
jaapv said:
Michael Reichman claims it is not 16 bit. But is this in any way important if the files are as good as it gets - which they are.?
Click to expand...
Actually, (according to MR's posting on LUF) it was Thomas Knoll (the guy who wrote Photoshop and ACR) who said so ... if he says the M8 DNG files are 8 bit, I don't believe anyone else has the magic turning them into 16-bit. :angel:

Joe.
 
What you are forgetting is two things:

1) The 30D RAW files are lossless compressed RAW files. There already has been some filter-pass processing, and then this info is compressed.

2) Is there a possibility, maybe, just maybe, that the DNG files that have been circulating only have 8-bit information because they were shot with that setting? It's not like more than a handful of people have the M8 to confirm any of this.

It is known for ill-informed rumours to crawl around the Internet, ya kno?

Before cheering for the M8's failure and looking as ridiculous as those who claimed the photos of the M8 were fake, how about if we all hold on to our chones and wait? :D
 
Last edited:
gabrielma said:
Before cheering for the M8's failure and looking as ridiculous as those who claimed the photos of the M8 were fake, how about if we all hold on to our chones and wait?
Click to expand...

Who says it is a failure? The reviews of these 8-bit DNGs were good. I suspect Leica knew what they were doing.

Don't confuse those 8-bit values with linear 8-bit values. The DNG specification identifies the model for the raw converter to handle these 8-bit values, for the case where there is a LinearizationTable, which there is for the M8. This is a look-up table, but in the case of the M8 it simply has the following effect:

Resultant value = square of the 8-bit value divided by 4

In other words, the raw converter first obtains 14-bit values, but with only just over 250 individual values, with unequal separations, within that range:
0 > 0
1 > 0
2 > 1
....
10 > 25
....
40 > 400
....
70 > 1225
....
99 > 2450
....
 
AndersG said:
The following uses dcraw to dump Reichman's M8 DNG to a linear B/W 16-bit per pixel b/w ppm file without any colour space conversions. Then I count the number of unique levels in the ppm file (the number should be expected to be large but no more than 65536) with ppmhist. ppmhist also outputs two extra lines that get counted so the M8 file contains 255 levels which is extremely low for a file with 16 bits per pixel.
Click to expand...

That is exactly what I would expect from examination of the contents of the DNG file. (And with software version 1.06, it REALLY uses 8-bit values!)

The first thing the raw converter does for an M8 DNG is use a 256-entry look-up table (held in the DNG file itself) to convert to (up to) 16-bits. I say "up to" - in the case of the M8, in fact the result can be held in 14-bits (no more than 16383). It happens to be:

Resultant value = square of the 8-bit value divided by 4
 
Jan Brittenson said:
It's quite possible that the high bits are 8-bit gamma encoded, while the low bits are huffman coded in a separate block (possibly with a proprietary tag).
Click to expand...

Simply use a 16-bit value inside the camera, and take the square root. Then store the result in the DNG file!

If there was any separate proprietary compression, how would raw converters handle the DNG files without having details of the M8 built into them?

For information, even ACR 2.3 under Photoshop CS1 can open the M8 DNGs, and ACR 2.3 was released in September 2004.
 
Barry Pearson said:
I say "up to" - in the case of the M8, in fact the result can be held in 14-bits (no more than 16383). It happens to be:

Resultant value = square of the 8-bit value divided by 4
Click to expand...

Hi, Barry, am I understanding it correctly, when you said "up to", what the converter does is actually something like "interpolation"?

Thanks,
Joe.
 
jkendal said:
Hi, Barry, am I understanding it correctly, when you said "up to", what the converter does is actually something like "interpolation"?
Click to expand...

No. Just a conversion of numbers without any reference to the image itself. (I said "up to" because the specified DNG look-up table has 16-bit entries in it. But the M8 only uses values up to 16383, hence values that can be held in 14-bits).

If the DNG file says 72, then the result is 72*72/4 = 1296. Etc.

The Leica M8 itself uses lossy conversion to get from its internal values to the values stored in the DNG. I don't know whether it rounds the result after taking the square-root, or truncates it. Suppose (and I don't know) that it truncates. Then values 5184 up to 5328 would be stored in the DNG as 72. And all of these would be converted to 1296 (5184/4) as the initial step by the raw converter.

I don't have the knowledge to indicate whether this is an acceptable transformation for image quality. All I can do is tell you what the DNG specification and model determines. Others with knowledge of human perception of non-linear values, and/or the downstream consequences "non-smooth" histograms, etc, will have their say.

(What I will say is that, at the start of this year, I decided, after various tests, to convert my own camera's raw files from ACR to 8-bit Photoshop files, not 16-bit Photoshop files. I determined that, for my images, if I got the histogram right in ACR, I could work in 8-bit mode without relevant visual degradation to the final image. Judges don't, probably can't, detect the difference, in a print of A3 or somewhat larger. It is vitally important to keep asking "what difference does it make to the viewer"? I would fail with the M8, but not because of any problems with the raw files. My style of photography needs the detachment provided by a pentaprism, not the immediacy of a direct viewfinder).
 
A RAW (NEF) file from a D200 can be converted to a DNG file using Adobe DNG Converter. Using Adobe Camera Raw to open the original NEF file and saving it as an uncompressed TIF file gives the same result as obtained from the corresponding DNG file.

Cheers,

Stan
 
You must log in or register to reply here.
Back
Top Bottom