Mac Workflow Ideas

[sirvine]

I've gotten very used to using Aperture for managing my photos, but it doesn't play nicely with the Epson RAW format. Do any Mac users have suggestions for the best way to batch convert R-D1 RAW files to a format that Aperture will accept?

I know I can use CameraRAW or the Epson plug-in to do one-off conversions to TIFF, but this presents two issues:

1. How do I batch convert with CameraRAW or the Epson plug-in?

2. TIFF files are huge. Is there a more economical format that retains non-destructive capabilties?

 

[Dan Lazin]

Are you accessing Camera Raw through Photoshop? (I don't believe it works as an Aperture plug-in, but my Aperture experience is minimal.) I'll provide instructions for batch-processing in Photoshop below. First, though:

• Again, I don't know about Aperture, but Photoshop has the ability to save TIFF files compressed in one of three different ways. The options come up when you select the TIFF format in the save dialog; the next dialog (after you hit Save) offers a choice of no compression, LZW compression (same kind of compression used in the GIF file format), ZIP compression (not supported in all applications, although Aperture can probably handle it) and JPEG compression (not recommended, because it's destructive). Remember that any type of compression will take more time both to save and open.

• Also, note that Apple will likely announce/release an update to Aperture (either to 1.2 or 2.0) on September 25. You might want to see what you can do with the new version of Aperture before you commit to a dual-application workflow.

• Lastly, if by "managing ... photos" you mean that you're using Aperture both as a photo editor and a permanent photo library -- emphasis on the last part -- you might instead try Adobe's (free) DNG Converter, which converts camera-specific raw files into the universal digital negative format, which I'm pretty sure Aperture can handle. DNG should be much more usable than ERF a decade from now. The advantages here are that you preserve the latitude of the raw files for the future (adjusting white balance, etc) and that you don't have to touch Photoshop.

But anyway, to batch-process with Camera Raw through Photoshop:

1. Open an ERF in Photoshop. When the Camera Raw dialog appears, set it up the way you want -- In CS2, I default to auto-everything, including white balance. Photoshop CS2's Camera Raw is *much* better at auto corrections than CS1's was. Pick your colour space, pick your bit depth, etc. Then grab the little flyout menu just above the adjustment palette (the button is a little triangle inside a circle) and pick Save New Camera Raw Defaults. Then open the file.

2. Go to the Actions palette (Window>Actions if it isn't open). Click the flyout menu on the palette and select New Action. Give the action a name you'll remember. Then press the Record button (circle) at the bottom of the palette.

3. Go to File>Save As. Pick the appropriate directory to save your TIFFs, and pick the TIFF file type, naturally. Hit Save, and on the next dialog box, select the type of compression you want. (For most people, I would say None, but you already know what you want.) Hit OK.

4. Close the file, then hit the Stop button (square) on the Actions palette.

5. Now you've taught Photoshop how to do the batch process. Next time, you can skip straight to this step. Go to File>Automate>Batch. By default, it should select the most recent action you've created; if not, select the appropriate action in the second pull-down menu. The Source just below that should be Folder (again, the default); click the Choose button to select the folder you want to import ERFs from. Make sure you click on the Suppress File Open Options Dialogs checkbox; doing so will make Camera Raw use the default parameters you set in step 1. Hit the OK button and Photoshop will start crunching. Ta da!

Hope that helps.

 

[ClaremontPhoto]

I keep stuff really simple by uploading everything to iPhoto (it's free), anything good gets transferred to PhotoShop elements and inkjet printed. And if it's really really good then the original negative gets taken to the shop and Fuji'd by them.

 

[sirvine]

Dan,

Yes, by "plug-ins" I meant to refer to Photoshop CS2. Your comments on DNG are interesting, but my attempts to convert ERF to DNG results in unsupported image files in Aperture. I don't pretend to understand why, so maybe I've done it wrong.

As for Lightroom, I'd prefer not to introduce a time-disabled beta into the workflow if possible. I also don't like the app, but maybe just because I've gotten used to Aperture.

I guess I'll cross my fingers that ERF will be well supported in the next version of Aperture.

 

[Dan Lazin]

According to Apple's tech specs for Aperture (http://www.apple.com/aperture/specs.html):

"1. DNG files must be generated by the Adobe DNG Converter, with the "Convert to Linear Image" option turned off, and created from RAW formats that are otherwise already supported by Mac OS X 10.4.3 or later."

Who knows what that last bit means. Try again, following the above instructions. I can't think of a reason that OS X would have to support the file natively; my DNGs work fine in Photo Mechanic and other non-Adobe apps, so I don't know why Apple would have restricted them further.

If you're on an Intel Mac, first grab the new version of Camera Raw and DNG Converter that came out today. DNG is now a Universal binary, so it'll run at full speed.

Have you tried the batch-processing directions for CS2? I know it looks like a lot, but once you've set it up the first time, it's really simple. I suppose you could do the same thing with the Epson plug-in, but I haven't tested it. Since CS2, I prefer Camera Raw, especially for high-ISO black-and-white.

 

[Dan Lazin]

Oh, and if Apple's instructions for generating DNGs don't help, I wouldn't hold out for the R-D1 to be supported in Aperture 2.0. They just aren't common enough cameras for Apple's developers to bother without prompting. You might have luck if you send them an e-mail, though. There was a thread here a while ago where someone asked Adobe for R-D1s support, and Adobe acted.

 

[kepstein]

According to Apple's tech specs for Aperture (http://www.apple.com/aperture/specs.html):

"1. DNG files must be generated by the Adobe DNG Converter, with the "Convert to Linear Image" option turned off, and created from RAW formats that are otherwise already supported by Mac OS X 10.4.3 or later."

Who knows what that last bit means.

DNG files are just like putting the RAW file in a generic envelope. The RAW format must still be supported by your application.

 

[Dan Lazin]

I don't think that's true. DNG is supposed to be a standalone format, so that in 50 years, when no one remembers what an ERF is, Photoshop's successors will still know what to do with a DNG (chiefly because the hope is that DNG will become prevalent as people accept the standard). From Adobe's page on DNGs (linked in the third post above):

"The DNG format helps promote archival confidence, since digital-imaging software solutions will be able to open your raw files more easily in the future."

This wouldn't be true if the application still needed to know about ERFs. And what would the application do with a DNG from an M8, which doesn't have a proprietary raw format, only DNG?

Also:

"DNG removes a potential barrier to new camera adoption, since raw files from new models will be instantly supported by Photoshop and other applications."

This again supports DNG being standalone.

 

[sirvine]

DNG conversion didn't work, since Aperture won't support the underlying RAW format. I'm going to try the Raw Developer product to see if it improves on Camera RAW's batch import/conversion workflow. Since posting above, I've found that Camera RAW does an acceptable job of converting to TIFF.

I still feel very uncomfortable compressing the TIFFs, as my assumption is that all compression is destructive. I could very well be wrong about that.

 

[kepstein]

"The DNG format helps promote archival confidence, since digital-imaging software solutions will be able to open your raw files more easily in the future."

This wouldn't be true if the application still needed to know about ERFs. And what would the application do with a DNG from an M8, which doesn't have a proprietary raw format, only DNG?

DNG works because ACR is able to support almost every type of RAW format. DNG doesn't work with Aperture because Apple core RAW engine does not support Epson RAW format.

 

[Dan Lazin]

Neither LZW nor ZIP compression is destructive at all, so you're safe using those for TIFF compression. JPEG is destructive, so you should avoid JPEG-compressed TIFFs.

(The manner by which LZW and ZIP manage to compress things without being destructive is -- to be simplistic -- that they look for multiple pixels that can be described simultaneously. If three pixels in a row are the same colour, it is quicker (smaller) to say "3 x 132,210,023" than to say "132,210,023, 132,210,023, 132,210,023." That's not actually how it works, but it's the same idea, I believe.)

 

[Dan Lazin]

This bit about DNGs got me interested, so I did a little more research. Basically, Adobe thinks DNG ought to be camera-agnostic, the way I said earlier. And you'd think Adobe would get to be right, since they came up with the specification -- but indeed, Apple is using DNG in a different (broken) way.

From the blog of Photoshop's senior project manager (http://blogs.adobe.com/jnack/2005/10/dng_momentum.html -- in these quotations, he's commenting on his own post):

"At the time of writing this I believed that Aperture's stated DNG support would give them compatibility with the 100+ cameras that ACR supports. Since then users have reported that that's not the case. I'm not sure why that is, but members of the Photoshop team have been in touch with Apple to see whether they need more info."

And further down:

"As to why Apple chooses not to support DNGs from cameras they don't support directly, Joe Schorr, the Aperture product manager, has told me that they'd prefer to do per-camera profiling for their raw conversion, and without that for a particular model they don't feel comfortable supporting DNGs from that camera. The upshot is that DNG does make it possible for Aperture and other software to support other cameras, but Apple chooses to go a different way. If you disagree with that approach, I suggest you take it up with them."

This is kind of unfortunate; it means R-D1 users may not ever be able to use Aperture the way it's supposed to be used, and although I haven't had time to play with Aperture significantly yet, it's an intriguing product, and I wanted to give it a spin eventually. The whole point of Aperture, though, is end-to-end raw processing. Using TIFFs instead of raw files seems like it would neuter the whole experience.

I think there's a good chance this will be fixed in the Aperture update. Aside from the good practice of keeping standards standard, a fix would let Apple announce that they've added support for dozens and dozens of new cameras. It's a good sales line.

Oh, and I'm with this John Nack guy (same blog post): "The term is neither an acronym (RAW) nor a proper name (Raw), but rather a generic descriptor for a whole class of formats. Therefore Adobe just says 'raw.'"

 

[kepstein]

"At the time of writing this I believed that Aperture's stated DNG support would give them compatibility with the 100+ cameras that ACR supports.

Again, DNG would only be compatible with the cameras supported by ACR. What kind of universal support is that?

 

[danielnorton]

I'm new to the RD1 so this might not be the best way, but I use epson photo raw- you can set your preferences and then switch to the mode that shows the images as a contact sheet, select all and move them over then process the lot all at once.. works well for me.

 

[Dan Lazin]

Again, DNG would only be compatible with the cameras supported by ACR. What kind of universal support is that?

Functionally, ACR and the DNG Converter support every raw format created in modern history. It might be missing a few really obscure old cameras, but Adobe's goal has been to support pretty much everything that can spit out a raw file. But that's not what makes DNG universal. What makes DNG universal is that (supposedly, theoretically) once you've converted a raw file to DNG, it's indistinguishable from any other raw file as far as compatibility goes, just like how Aperture can open a jpeg from an R-D1 as well as it can open a jpeg from a Canon 1D.

The reason that the Adobe guy points out how many cameras ACR supports is that *that's the only way to create a DNG* (except for the four cameras that generate DNGs as their native raw files).

Think about it this way: a new version of ACR (3.5) and DNG Converter came out a couple days ago. It added support for the new Sony A100. So let's say you now rip a dozen DNG files from your A100's raw files. Now, just for fun, you revert to the old version of ACR, 3.4. 3.4 doesn't support A100 raw files -- but it will still read the DNGs you just created. DNG-capable software isn't supposed to care about what the camera is; if it can read one DNG, it can read them all. Except that Apple broke that functionality.

Sorry we've hijacked your thread with this, sirvine. I do think it's (tangentially) relevant: I want to get this cleared up so that people realize that (again, theoretically, and in practice as well so long as no one else joins Apple in this bizarre interpretation) DNGs are a much better format than ERF to store raw files in for the future (and, being raw, more useful than TIFF, too).

Sure, Photoshop still supports a bunch of formats that no one uses like Targa and PCX, but Photoshop is giant and kludgy; it's designed for painters and illustrators and graphic designers and special-effects guys as much as it is for photographers. The advent of applications like Aperture and Lightroom suggests that most photographers won't be using something like all-purpose Photoshop in the future. We'll be using something more tailored to photography in particular, and whenever they start coding that application (I don't think Lightroom and Aperture are quite there yet), I bet they'll start fresh. Throwing away all the legacy crud means throwing away support for hundreds of cameras worth of native raw files, because the developers can just say, "hey, we included DNG support."

End of this essay. Sorry again for the hijack.

 

[kepstein]

Think about it this way: a new version of ACR (3.5) and DNG Converter came out a couple days ago. It added support for the new Sony A100. So let's say you now rip a dozen DNG files from your A100's raw files. Now, just for fun, you revert to the old version of ACR, 3.4. 3.4 doesn't support A100 raw files -- but it will still read the DNGs you just created.

Did you try this, I don't believe that it will work.

"DNG IS NOT THE ANSWER
Let me first make one thing clear: DNG is not an open standard for defining and storing all needed RAW camera information.

DNG makes the RAW format problem worse, not better.

DNG is not an open standard in that it does not document all the essential information contained in current RAW format files like NEF and CR2 (which also don't document this information).
In many ways, DNG can be viewed as simply yet another RAW format with undocumented information - except that DNG has the added risk that information can be lost during conversion to/from DNG and other RAW formats.
From a software developer point of view, DNG is a step backwards. From a camera manufacture's perspective, DNG does not address the missing elements in EXIF.
From a photographers perspective, DNG is dangerous because people believe they are storing for the future with the format, when nothing could be further from the truth." (source www.openraw.org)

 

[Dan Lazin]

I've just tried the experiment I suggested, and indeed it did work. I converted an NEF from a Nikon D200 to DNG, then downgraded to Camera Raw 3.1. Photoshop could no longer open the NEF, but it opened the DNG perfectly. This is how DNG is supposed to work.

There's a good response to the criticisms in the quote you posted here: http://www.barry.pearson.name/articles/dng/commentary1.htm

I think it's fair to say that both of these writers -- Stuart Nixon at OpenRaw and Barry Pearson at the link I provided -- know more about the topic than either of us, so I don't feel qualified saying which is more right. Thomas Knoll, the creator of Photoshop (and Camera Raw, and I believe DNG) says Pearson has the better understanding.

Either way, I'll concede that DNG may not be perfect, but it's the best we have -- more usable than TIFF and more likely to survive than ERF. More on this later. (Should we start a different thread? I do want to hammer this out -- I think it's important for R-D1 users to figure out what the best way to prepare for the future is, because this camera will easily be forgotten -- but I feel bad about hijacking this thread.)

Right now though, it's time to make baklava.

 

[sirvine]

Don't worry about threadjacking--I'm enjoying this discussion because I'm ignorant on the topic. I'm having a hard time understanding the usefulness of DNG's ability to open in older versions of Camera RAW, but not necessarily in third party apps that support certain DNG files but not others!

Perhaps someone can explain to me why converting RAW to TIFF is a bad thing, if one uses a good RAW convertor. It seems to me that TIFF is truly a universal format for all intents and purposes.

 

[Dan Lazin]

There isn't really any "usefulness" in opening a DNG in an older version of Camera Raw -- that's just a way of proving that that's how the format is supposed to work, i.e. that the files ought to open in any DNG-aware application, regardless of whether the application has ever heard of an R-D1. Third-party apps are supposed to support DNG completely or not at all -- this move of Apple's is very strange.

The reason I would prefer to archive files in a raw format of some kind instead of TIFF is that raw preserves a lot more editing possibilities, whereas once the TIFF file is generated, you're stuck with it. (Sure, you can still edit the TIFF, but there's no way you can recover data that isn't there, like blown highlights, whereas the capabilities of raw processing improves all the time.) Camera Raw in CS1, as an example, wasn't very impressive. I generally used the Epson plug-in instead of CS1's ACR. But CS2's ACR is phenomenal at recovering moderately blown highlights, and its four-way auto correction does the best auto processing I've ever seen.

I can't wait to see what CS3's version of ACR can do. But you'll never get to play with it if all your files are TIFFs.

Oh, and then there's disk space: from a 9.8Mb ERF, I can create a 5.3Mb DNG (because DNGs include lossless compression, by default). That DNG still preserves 16-bit capability, which gives a whole lot more headroom before your editing becomes seriously destructive. A 16-bit TIFF generated from the same file, however, is 34.5Mb. (Curiously, adding LZW compression to that TIFF made it *bigger* -- it jumped to 42.2Mb.)

 

[kepstein]

There isn't really any "usefulness" in opening a DNG in an older version of Camera Raw -- that's just a way of proving that that's how the format is supposed to work, i.e. that the files ought to open in any DNG-aware application, regardless of whether the application has ever heard of an R-D1. Third-party apps are supposed to support DNG completely or not at all -- this move of Apple's is very strange.

So, now I'm confused. If I take a RAW file that is not supported in ACR, convert it to DNG, I could then open the file in ACR?

Thanks,
Ken