New dust removal software

[bensyverson]

I switched to VueScan a while back for my Coolscan, and started to want a little more control over the infrared dust removal. So I built a little tool to help out! I scan the negative as a slide, and have VueScan save it as "64 bit RGBI" TIFFs. Basically, a TIFF with the infrared in the alpha channel.

If you're interested, you can download an early beta of the app (Mac 10.6 or above). I'd love to hear your feedback!

Keep in mind this is BETA software. Use at your own risk. Always back up your data, and only run this on COPIES of your images. Choosing "Save" WILL write over the original, as designed.

The results have been pretty promising. Before:

After:

Before:

After:

Not only can you remove the dust, you can also remove the "pepper grain." Pepper grain is actually the result of tiny air bubbles trapped in the plastic base of the film. It shows up as dark specks on slide scans, or faint white specks on negative scans.

Being able to remove the pepper grain makes the scan smoother, less grainy and more like what you'd expect from making a wet print. Cleaned on the left, original on the right:

 

[bensyverson]

Here are some screenshots with commentary. Please keep in mind this is BETA software, use at your own risk, use this only on copies of your images!

Here's the image of the scan when I first open it:

First, I invert the image

You can view the infrared mask:

The "Overlay" mode lets you view the areas that will be replaced. Anything green gets replaced. In this mode, you should adjust the tolerance so that the desired amount of the image is selected. You can decide to remove just big chunks of dust, or you can get rid of everything down to the pepper grain.

The final "Composite" mode lets you view the result. Adjust "Radius" to determine the strength of the inpainting. A low Radius will only fill in pepper grain, whereas you'll need a large Radius to fully remove bigger pieces of dust. The higher the Radius, the longer it will take to process.

When you're done, choose Save. That's it!

 

[bensyverson]

And just for fun, the final result after some Curves. Gotta love that Hexar! 🙂

 

[brainwood]

Hi

Just tried to download the beta version from the link. I have 10.6 but it doesn't seem to work!!
I have Digital ICE on my scanner but would be keen to try this

Chris

 

[bensyverson]

Hi

Just tried to download the beta version from the link. I have 10.6 but it doesn't seem to work!!
I have Digital ICE on my scanner but would be keen to try this

Chris

Oh no! Does it not open at all?

 

[bensyverson]

One note—right now, when you open the app, it doesn't open a new window. To open an image, go to File > Open, or drag a file onto the app.

 

[brainwood]

Ok I understand now . i had expected a new window to open.

Initial attempts look very good . I'll have a play and let you know how I get on

Chris

 

[charjohncarter]

This is to thank you for this: (even) Beta look at this. So no Windows as yet? Keep us posted if we can use it with windows.

 

[f16sunshine]

Nice Ben, I may give a try over the long weekend. Have subscribed to this thread. Thanks!

 

[Brian Puccio]

Looks awesome, I'll certainly give it a spin this weekend.

 

[jan normandale]

Hi Ben, I've got a tonne of developing and scanning coming up. I'm going to download your beta and give it a test drive. Thanks !

 

[Particular]

Thanks! I'll will give it a try soon, and report back.
I have been using the VueScan scan to RGBI TIFF, and then reprocess to DNG, method for a while. It has been great, but the one weakness is the dust removal. I was finding that it was often slightly shifting the corrected pixels, so you would have a white spot with a dark one right next to it. I am really hoping the fine-tunable controls on this will be able to resolve that.

 

[Particular]

I downloaded IcedOut last week, tested it a bit, and I have been sitting on these comments all week (waiting for RFF to come back on line).

First of all I'd like to say IT WORKS!!!
I wan't to applaud Ben for tackling this problem and coming up with a tool that is perfectly useful as it is. There is a lot of potential for this to be truly great.

To start with the things I like:
- Simple clean interface. There is a good logical workflow. The invert buttons are really useful. The tabs Original/Mask/Overlay/Composite are in the logical sequence. Seeing the mask is great, it helps to demystify the process. The green masking in Overlay gives a good sense of what is being affected.
- I have tried it on some E6 film on various settings, and got results ranging from almost not difference to being totally clean. There is a good wide range of control.
- I even got it working with kodachrome. Using a light 'Tolerance' setting will clean the worst dust, mainly in the darker areas. You can't clean it completely, but it is better than nothing. I still intend to try it on some b+w emulsion.

Some comments and possible improvements…
Menu:
- Having a 'Save as' option makes it easy to make multiple versions (especially while testing).
- The 'New' option doesn't seem to serve a purpose.

Interface:
- By playing around I got a good sense of what the Tolerance slider does. I had a harder time figuring the Radius slider out. Both sliders could do with some indication of what is the higher and lower end of the scale. Having numerical input as well would help to get a better sense of the setting, and make it repeatable.
- I tried out the default setting on a scan and got a blurry scan with sharp dust and scratches. Perhaps there is a better default setting, but more important is to know which way to move the sliders to improve it.
- I don't mind using Apple-S, but perhaps adding a Save button would include the complete workflow in one interface window.

Wish List:
- Adding a Zoom (to closely inspect the Mask and Composite), and Rotation button would help. By default my coolscan scans portrait, and I usually rotate in Lightroom.
- Batch processing would make this really powerful. I rarely scan one image at a time, so if I can find the right setting for one image, I'd like to apply it to the rest of the roll/folder.
- When there is a big correction (scratch or hair) the correction is visible as a smooth line/smudge (see example image above). Perhaps it is possible to add a light grain effect to the correction to give it some texture which would help it blend in.
- Currently the output is a 24bit RGB TIFF. My normal workflow is to save a VueScan raw TIFF, which I then reprocess (ICE, inversion, crop) to a raw DNG. I then work with this in Lightroom as if it was a digital camera raw. For me to fit IcedOut in that workflow it will need to output a 48bit RGB TIFF (without adding profile or gamma), so I can take it back into VueScan to invert it and make it a DNG. If IcedOut saved the inversion and had DNG export I wouldn't need to go back to VueScan at all, which would be really great.

Super Wish List (just dreaming):
- If this was a Lightroom plugin, I could tweak the sliders while pixel-peeping. I would scan and save the RGBI version as DNG and do everything else in Lightroom. It would cut out a workflow step, and allow fine-tuning after the fact. I do not know if Lightroom supports RGBI (DNG does), and may require entirely different programming skills, but I am just throwing it out there.

Not all these ideas may be possible or make sense, but hopefully they'll help. I'll be watching the development of this app, and keep testing it. Good Luck.

 

[bensyverson]

CNNY, thank you so so much for testing it out!!! Almost everything you mention is on my to-do list before 1.0.

First up for me is definitely batch processing, but before that happens, I have to find a way to normalize the range of the sliders. Basically, Threshold is going to turn into "Amount," and it's going to control the percentage of the image you want to replace. I've had good results when the algorithm is replacing about 25% of the image. So once I nail that down, batch will be possible, as well as "reasonable defaults." Right now it varies image to image.

One thing I added during the RFF hiatus was an additional slider for evening out the mask. Dense areas of the scan block more IR, whereas thin areas let more through, so I was getting uneven results (no effect in the shadows). This fixes that completely.

I also upgraded the image input/output code, because Apple's default libraries were choking on 4x5 scans. One of the side effects is that the viewer now has a 100% pixel view alongside the "full image" view.

I have to look at whether Aperture and Lightroom will feed me images with intact alpha channels, but if they can, plugins are a natural.

The Radius slider basically controls how much infilling will happen. If you have nothing but very small specks of dust, a small Radius is faster to compute, but if you have big chunks, the Radius will have to be very high to completely paint over them. Usually there are 1-2 big pieces at the most, so I tend to paint those out manually. But you're right that some noise will help the replaced sections blend in more! I'll have to do that.

I'll have a new version as soon as I can fix everything in the new image reader. It's pretty cool—as it's reading in a 4x5, memory use hovers around 50MB! So you don't have to worry about choking your computer—or closing RFF. 🙂

 

[Particular]

Great, I am looking forward to the next version. I just sorted 60+ rolls for scanning, so I would love to keep testing.

 

[cabbiinc]

I don't have a Mac so I haven't had the pleasure of trying this out for you but I do have a problem that your software could be a solution for.

On my HP scanner the IR scan and RGB scan never line up. HP solves this with software, Ed Hamrick refuses to do anything about this in Vuescan and just claims that I should buy a Nikon scanner. But since I bought this scanner with 4x5 and larger sizes in mind a Nikon scanner won't do me much good (I have one). If you could line up the layers before the ICE takes place that would add so much functionality to an inferior scanner.

Just a thought
Dan

 

[Particular]

I don't have a Mac so I haven't had the pleasure of trying this out for you but I do have a problem that your software could be a solution for.

On my HP scanner the IR scan and RGB scan never line up. HP solves this with software, Ed Hamrick refuses to do anything about this in Vuescan and just claims that I should buy a Nikon scanner. But since I bought this scanner with 4x5 and larger sizes in mind a Nikon scanner won't do me much good (I have one). If you could line up the layers before the ICE takes place that would add so much functionality to an inferior scanner.

Just a thought
Dan

I am having the same problem with Nikon scans from vuescan. The corrections seem to always be a few pixels shifted over from where they should be, leaving a kind of 'shadow effect' when applying dust removal. The tryouts I have done with Ben's software did not have this problem, so I am not sure whether it is the scanner, or vuescan.

 

[hans voralberg]

Looks very promising 🙂 When is a new version expected Ben?

 

[cabbiinc]

I am having the same problem with Nikon scans from vuescan. The corrections seem to always be a few pixels shifted over from where they should be, leaving a kind of 'shadow effect' when applying dust removal. The tryouts I have done with Ben's software did not have this problem, so I am not sure whether it is the scanner, or vuescan.

Well I'm experiencing more in the neighborhood of mm than just plain pixels.

The two arrows on the right are the dust and the spot that the software tried to clean. The two on the left are the spot and the spot that the software tried to clean. Cleaning was set to the weakest setting.
On the bigger image that dark border across the top goes away if you don't do the IR scan with Vuescan.

HP's software usually lines up, sometimes not. When it lines up it won't eliminate the dust completely anyways, so it's kind of a moot point. It's been this way since I bought the scanner new. HP said that that's just the way it is and the scanner is operating per specs. Flatbeds are after all designed for scanning printed material and most film scanning is an afterthought.

Dan

 

[srtiwari]

Just downloaded it to an iMac running 10.6.8. Scanned a B+W image as a slide (Epson V700 and Vuescan) with enough dust on it for a trial. Unfortunately, 3 attempts to open the image made the program crash.
I assumed it would also work on B+W images. Is this not the case ?