Camera-Scan Challenge for Color-Neg, as Automatic as Possible

[Michalm]

And here are some interesting resolution and artifacts tests.
In the past i was lucky (actually not lucky) to use MINOLTA DiMAGE SCAN 5400II. It is a great scanner, but it had a global problem with non uniform backlight stripes, described here https://www.photo.net/discuss/threa...n-minolta-dimage-scan-5400-ii-or-what.479397/ (too bad uploaded photos are gone over time)
After many attempts to modify LEDs or diffuse that defect i gave up with it. I decide to take off the lens form that scanner and build a camera scan system.
Same time i had access to smaller MINOLTA DiMAGE SCAN Elite II model, so i was able to take some shots from it for compare.

For this test i build input ICC profiles for camera and both scanners based on Kodak IT 8.7 Scanner Calibration Target from http://www.targets.coloraid.de/
Profile Type: Single gamma + Matrix

Scan from camera debayered in Iridient Developer with Anti Aliasing setting: 2.
Custom contrast camera curve removed. No noise reduction, no sharpening, no any other adjustments applied to RAW file. 5 frames stacked in Median mode.
Poor quality consumer furniture LED panel used as backlight.

Scans from scanners are in linear gamma with disabled color management.

All images processed with my workflow described earlier:
Transform from Camera input ICC profile to ProPhotoRGB with L* gamma ICC profile -> Invert -> Apply RGB AutoLevels -> Recover back clipped data from RGB AutoLevels -> Contrast.
No custom White point picker.

This is rather complicated frame i use specially for tests. Is is very scratched and it have a lot of extreme saturated colors.

This is very interesting . I have not done any camera scans myself yet , but have been wondering about what would get me at least as good results as Coolscan V , and your samples here look encouraging. I have also been thinking lately about sourcing scanner lens as we can see from you samples scanner lens is the way to go. I just wonder how fiddly it is going to be to align the setup properly. Because of the lack of iris in the lens thin dept of field ANR glass is probably a must . I would probably start with BW film.

 

[charjohncarter]

Michalm, I have a flat bed scanner and I did this to check against my digital scanner (more than 10 years old). Very simple nothing like the confusing above stuff. Here it is:

Olympus 35RC Trix HC-110h by John Carter, on Flickr

Now flatbed:

Olympus 35RC Trix HC-110h by John Carter, on Flickr

and digital scan:

Olympus 35RC Trix HC-110h by John Carter, on Flickr

Are we in the weeds on this subject?

 

[shijan]

Scanners looks ok because it is all-in-one light-lens-sensor-debayer system made by large tech companies with large technical background. Their sensor calibrated to their light source. Light source temperature picked to match as close as possible to native sensor white balance. Fixed aperture lenses are flat, tiny, sharp and don't produce abberations.

Same time camera scan system may be very random and depends of multiple variations:
- Light source quality
- Lens quality
- Camera debayer settings
- Camera input profiles are averaged and usually designed to match multiple color temperatures. You also will get different results between input profiles from DCraw, ACR or some other apps packages and raw editors.
Custom made ICC profile based on IT8.7 target helps a lot here.
- It is not always easy to expose camera exactly without single channel clipping. Lack of raw histogram preview in many cameras.
- Negative invert process. Lazy photographers don't want learn the basics of invert process, but instead pay 100 bucks for apps that remove build-in contrast curve form raw file, do invert and autolevels in fixed Lightroom's ProPhotoRGB color space with sRGB gamma curve and do adjustments with confused non technical abstract sliders 🙂

Currently i am in the middle of a small DIY hardware project that should improve camera scanning usability.

 

[charjohncarter]

I have a good workflow for color negatives when I use my scanner software. I.E. I adjust my individual and total histogram to eliminate clipping.

I still have a problem and I might work on it some more with what you call negative invert when I do digital camera scans.

I do like this sentence: Negative invert process. Lazy photographers don't want learn the basics of invert process, but instead pay 100 bucks for apps that remove build-in contrast curve form raw file, do invert and autolevels in fixed Lightroom's ProPhotoRGB color space with sRGB gamma curve and do adjustments with confused non technical abstract sliders

Here is my digital camera coping DIY project that I did in 2008:

Untitled by John Carter, on Flickr

 

[ColSebastianMoran]

This is rather complicated frame i use specially for tests. Is is very scratched and it have a lot of extreme saturated colors.

Shijan, how does your process get rid of the scratches? I would think the scratches on film would persist in the process of multi-shot, stacking, and median.

I have used the median process to get rid of unwanted moving objects, but it doesn't remove static things like scratches. How did you remove the scratches?

I ask because dust/scratch is a frequent problem for camera-scanning.

 

[shijan]

Median stacking can't remove scratches. In camera scan you can partially remove scratches with diffused backlight.

Diffused light - softer film grain look, but also near invisible dust and scratches.
Collimated light (usually used in film scanners) - sharper film grain look, but all dust and scratches became extremely visible.


Or you can experiment with wet scan mount to 100% remove scratches.

 

[shijan]

Here are my optical crops example of same frame. This is larger possible magnification that i can do on my bellows. This optical crop is virtually equal to full frame scanned at near 16000 x 24000 = 384MP.

So yep, those are colored saturated film grain seeds. And i guess that colored grain look must be somehow preserved as starting point in smaller sized scans.

Problem 1:
With 26MP scans there is a conflict between film saturated grain particles and digital noise that is near the same size at that magnification. And as result - some saturated aliasing and moire artifacts overlapped with film grain. Probably i need to experiment with camera scans at higher optical resolution and stitching.
Problem 2:
Anti aliasing filters in raw processing apps are very different from basic Chroma noise reduction filters and remove digital aliasing artifacts as well as all saturated seeds of film grain. And surprisingly it is not depends a lot of scan size and film grain size. Saturation loss in film grain particles is visible at 26 MP example as well as at optically enlarged "virtual 384MP scan." Probably i need to search for better raw processing workflows.

 

[shijan]

And here is my scanning setup. Camera stand and base plate are all metal and very stable. No any shake at deepest pixel levels even with mechanical shutter.
It is still not finished setup, so currently i just use filmholder from old scanner and place it on top of crappy furniture led panel. This produce a lot of focusing variations across the frame yet.

Stand Parts:
SmallRig Multi-purpose Cheese Plate 1092
SmallRig Super lightweight 15mm RailBlock v3 942
SmallRig Baseplate with Dual 15mm Rod Clamp 1674
15mm Rods

Focusing system parts:
Novoflex dual rail MINOLTA Auto Bellows
RafCamera RMS female to M39x1 male thread adapter with M42x1 front female thread (manufactured based on my idea)
RafCamera 18mm clamp to RMS male thread adapter for Minolta 5400 DPI scanner lens
RafCamera Novoflex Minolta Bellows front plate with M39x1 (manufactured based on my idea)
RafCamera Novoflex Minolta Bellows rear plate with M39x1 (manufactured based on my idea)
Macro Extension Tubes M42 (used as a lens hood)
Lens Mount Adapter Ring M39 Lens to Fujifilm

 

[charjohncarter]

That looks sturdy. I had to get a macro lens for my unit because of film plane curvature of standard lenses. I also had to be careful of light piping onto the negative from ambient light but yours looks pretty light tight.

 

[Michalm]

And here is my scanning setup. Camera stand and base plate are all metal and very stable. No any shake at deepest pixel levels even with mechanical shutter.
It is still not finished setup, so currently i just use filmholder from old scanner and place it on top of crappy furniture led panel. This produce a lot of focusing variations across the frame yet.

Stand Parts:
SmallRig Multi-purpose Cheese Plate 1092
SmallRig Super lightweight 15mm RailBlock v3 942
SmallRig Baseplate with Dual 15mm Rod Clamp 1674
15mm Rods

Focusing system parts:
Novoflex dual rail MINOLTA Auto Bellows
RafCamera RMS female to M39x1 male thread adapter with M42x1 front female thread (manufactured based on my idea)
RafCamera 18mm clamp to RMS male thread adapter for Minolta 5400 DPI scanner lens
RafCamera Novoflex Minolta Bellows front plate with M39x1 (manufactured based on my idea)
RafCamera Novoflex Minolta Bellows rear plate with M39x1 (manufactured based on my idea)
Macro Extension Tubes M42 (used as a lens hood)
Lens Mount Adapter Ring M39 Lens to Fujifilm

Nice and compact but where is the film holder ?
I would be curious to see how would full frame camera shooting in the pixel shift mode and paired with your scaner lens attached fare .

 

[shijan]

Nice and compact but where is the film holder ?

As i told, currently i just put Film holder from old scanner on top of the backlight. It is a crappy setup and low quality light source for test purpoces only.

But i am in the middle of development of my own quality filmholder system that will perfectly fit to this 15mm rods system. This will be something very different from any current existing filmholders.
It will be manufactured in small batch, so if other people may use it as well. I'll create a separate thread when first test samples arrive from manufacture.

 

[ColSebastianMoran]

Median stacking can't remove scratches. In camera scan you can partially remove scratches with diffused backlight.

Diffused light - softer film grain look, but also near invisible dust and scratches.
Collimated light (usually used in film scanners) - sharper film grain look, but all dust and scratches became extremely visible.

But, how did you remove the scratches in the photos you posted?

 

[charjohncarter]

But, how did you remove the scratches in the photos you posted?

That is not an unintelligent question. That is also one process I would endeavo(u)r to ascertain.

 

[shijan]

But, how did you remove the scratches in the photos you posted?

I didn't remove any scratches manually 🙂 It is just different light sources types that amplify different amount of scratches.

You can do your own test in larger scale to understand how light works:

- Take some old scratched glass and put it at the small distance of white wall. (this glass wiil represent scratched film, and the wall will represent camera sensor)
- Take some a flashlight with non diffused single point LED and light trough the glass on the wall.
- You will see huge amount of hard projected scratches on the wall.
- Now put some diffusion sheet between flashlight and glass.
- Now you see that all projected scratches are blurred or completely gone.

The most interesting is that this system is distance dependent. Play with distance between light source, diffusor and scratched glass and you will see how changes the look of projected scratches.

 

[shijan]

By the way, if someone want to play with test sample, here is link to RAW file:

https://drive.google.com/file/d/1C5O33sOXNF_xOCRRwtZLRNPaIBKVmG9n/view?usp=sharing

And this is my subjective reference processed "look" for this frame. It also illustrate a difference between processing with different gammas.
Processing chain is the same as i described before:
Custom made Input ICC profile -> Transform to ProPhotoRGB L* TRC -> Invert -> RGB Auto Levels -> Grey Point -> Contrast.

Colors in this frame are complicated, and in addition i used low quality LED backlightlight source. So without custom-made Input ICC profile image colors will look very different from my reference sample and not pretty at all.

 

[jzagaja]

Negative inversion is easy - you only need true log output from camera like cine cameras. I have had Red camera samples and they are great. So try Eterna profile. Another approach is to apply log manually to linear 16 bit file but RAW converters are so unpredictable and to many unnecessary functions.

After you have log wide gamut image of negative you only need old Photoshop (6-8) function Auto Color. It works very well. Otherwise manual Levels. New Photoshop sucks.

L* conversion works because lightness channel is approx Gamma 3.0 that is approx LOG.

Fuji X-trans does not work well with film grain - you need oversampling.

 

[shijan]

jzagaja, seems you didn't read my posts in page 6 and 7 at all.

Log output from camera is not a real starting point. It is simply a transformation curve designed to transform data from sensor source linear gamma to Log. This helps to manipulate data from digital sensor in more analogue and human friendly way.

Film Negative itself is already in some sort of analogue Log, so stretching it even more and placing it inside another camera sensor Log is bad idea.

Eterna or any other picture profile is just a image look and it is for jpegs only. It shouldn't be used for film scans at all.

 

[Michalm]

My most recent film scanning process was to scan film in 16 bit gamma 1 as positive on Coolscan V scanner , then invert using Negfix 8 which would mostly get me the image that can be easly corrected with curves and levels etc. Am I correct to say that scaning with mirrorless camera and and then running it through MakeTiff Raw Converter would get me similar gamma 1 flat 16bit tiff than can be inverted using Negfix 8 in the same way? Also I do not suppose Darktable or similar raw converter is capable of generating flat tiff in the way that Make Tiff raw software can do?

 

[jzagaja]

Log output from camera is not a real starting point. It is simply a transformation curve designed to transform data from sensor source linear gamma to Log. This helps to manipulate data from digital sensor in more analogue and human friendly way.

It's very good starting point if LOG output is true and without slope limiting.
This is because negative has lower contrast and LOG curve makes contrast neutral and keeps film character (highlight/shadow compression). It was used for decades as most effective way to use just 10 bits per channel and prevent banding. OS/software architecture should switch to 10bit log HEVC storage and look up table for particular output device.

You only need good sensor without OLPF. Red cameras have clean colorimetric output so less work needed.

jzagaja, seems you didn't read my posts in page 6 and 7 at all.

You have shown interesting RAW converter and limitation of Photoshop AMP curve. If you found banding then you can apply floating point precision transformation through plugins. Photoshop and Adobe products are full of bugs.

Eterna or any other picture profile is just a image look and it is for jpegs only. It shouldn't be used for film scans at all.

Why? try it first. It can make life easier and much quicker conversion since you just need Levels. Colors from negatives are always specific and that's the beauty.

There is interesting plugin called AdjustShop where you can record Photoshop transformation as color look up table and use it e.g. Davinci for scanning cinematographic film.

 

[shijan]

My most recent film scanning process was to scan film in 16 bit gamma 1 as positive on Coolscan V scanner , then invert using Negfix 8 which would mostly get me the image that can be easly corrected with curves and levels etc. I'm I correct to say that scaning with mirrorless camera and and then running it through MakeTiff Raw Converter would get me similar gamma 1 flat 16bit tiff than can be inverted using Negfix 8 in the same way? Also I do not suppose Darktable or similar raw converter is capable of generating flat tiff in the way that Make Tiff raw software can do?

Formally linear scans from scanner are the same as debayered but uncorrected files from camera.

There is a problem with MakeTiff that i described earlier in post 243 https://rangefinderforum.com/forums/showpost.php?p=3026823&postcount=243 . MakeTiff don't respect White Balance setting selected in camera, and set WB to some insane value at the edge. As result image looks extremely green. And that insane green color cast corrected in RGB image produce huge amount noise. So i don't recommend MakeTiff, as well as i don't recomemnd ColorPerfect plug-in.

Irdient Developer can debayer to linear tiff, but it is commercial app.
PhotoLine can debayer to linear tiff (it use dcraw library), but currently quality is very basic there.
Probably some GUI for more advanced dcraw settings may do the trick https://en.wikipedia.org/wiki/Dcraw

To work with linear non color managed scans you need proper input ICC profile. You need to build your own custom-made input profile based on IT8.7 calibration target, or you can use original input ICC profile provided with scanner software.
But there is a huge problem with original ICC profiles from Nikon scanners. You can read in details in this article. https://smashandgrabphoto.wordpress...onsistent-colour-using-icc-profiles/#more-410
"It occurred to me that Nikon Scan installs a bunch of profiles for the scanner but they don’t show up in Photoshop or other imaging applications. I tracked down why this was. Nikon had altered the metadata in the header to make them not appear to be valid. Specifically, they changed the “type” from “scnr” (a scanner or input profile type) to “nkpf” (a non-standard value). If you take a hex editor (or even vi) you can just find the value “nkpf” in the first few lines of the file and change it to “scnr” and you will get access to use the profile in any application of your choosing."

There is a download link for fixed (hex-edited) Nikon Scan input ICC profiles available here https://www.dropbox.com/sh/eveu6pth4zqjgb4/AABfebLfpRR18UA09BzwNUTDa?dl=0