Camera-Scan Challenge for Color-Neg, as Automatic as Possible

[silkyfeet]

Oh, yes - scanning is almost an art - or a past time made to feel better by calling it "an art". In my case, it has taken a long long time to figure out what best method is to scan and get the best results, but there are more important things in life as you know.

The key to eventually getting a good scan is: training your eye to subtle colour hues and tone, KNOWING what you want to achieve in comparison, experimentation and - I cannot stress this more: perserverance, pervere, and continue to try.

Next time you scan, try to achieve the most colour NEUTRAL setup. High resolution is not necessary. And on import, do the same whether its Photoshop, Lightroom, Gimp or what ever.

cheers. I get so dejected when i have to scan i that stopped shooting colour if im not going to be able to head to a darkroom. i will keep at it and give your method a go. the weird thing is i understand colour correction and in the darkroom i am ok but on the computer... MY DAYS its hit and miss. I must say when i played with vuescan i did get rather good results but i was reluctant to shall out another 60-90 pounds when i spend money on silverfast. Silverfast always clipped my highlights or crushed blacks even when i used the zone system. just weird

 

[ColSebastianMoran]

Here's another approach for camera-scan of color-negatives:
- For 35mm mount a good macro add-on lens to your 12MPx phone. Exolens by Zeiss Macro and Moment Macro are excellent; most others are toys.
- Shoot the color-negative with Lightroom Mobile, creates a raw file (actually DNG)
- Sync that to Lightroom on your desktop
- Then convert with Negative Lab Pro or software of your choice.

Here's an example: iPhone 6s (12MPx), Exolens by Zeiss, Negative Lab Pro

 

[ColSebastianMoran]

And, some new software options emerging.

Grain2Pixel a Photoshop add-on "digital camera film scans in batch mode or in file-by-file mode".

NegMaster another Photoshop extension "aimed at photographers who want their photos to be as true to color as possible"

Some tools within and/or added to CaptureOne

And, CNMY which has been mentioned here earlier.

This is an invitation for anyone using these new tools to share comments and post an example. You are welcome to convert my cam-scan test file.

LINK TO RAW FILE:
http://2under.net/images/171004-TestBox-Fuji200-SonyNoFilter-DSC9558.ARW

 

[kiemchacsu]

thanks for the tips; i am new to this digi cam scanning;

i have been using nikon coolscan for years but now i think this a good approach for the future of film shooters.

 

[ColSebastianMoran]

Here is a survey article by Tony Huynh comparing conversion approaches including Grain2Pixel, Negative Lab Pro, Silverfast, and RawTherapee.

 

[Andre Negmaster]

Hello everyone! I am André Eikmeyer, Negmaster's developer. Richards Karash pointed me on this thread some time ago. You've got interesting things going on here.
I'm open for criticism and discussion. Feel free to ask me anything. Remember i'm just a human as well and got no elephant skin.
Looking forward to your feedback and i hope it makes my software better.

Regards from Germany
André

 

[ColSebastianMoran]

Hello everyone! I am André Eikmeyer, Negmaster's developer. Richards Karash pointed me on this thread some time ago. You've got interesting things going on here.
I'm open for criticism and discussion. Feel free to ask me anything. Remember i'm just a human as well and got no elephant skin.
Looking forward to your feedback and i hope it makes my software better.

Regards from Germany
André

Welcome Andre!

Can you reply with links to your software and perhaps an example image?

 

[Andre Negmaster]

Welcome Andre!



Can you reply with links to your software and perhaps an example image?

Hello there! I will prepare something and i will share my thoughts about our experiment here.

I did not post any links because of the advertising rules.

 

[Andre Negmaster]

Sorry for letting you wait. I still didn't have the time for preparing some useful input for you because i have a high update frequency for the program at the moment and i need the whole day for figuring things out. The higher the update frequency, the more support for the customers has to be done and so on...
But i can offer a good read about Negmaster and film color rendition in general on petapixel. There you can see comparisons to a competing product as well.

https://petapixel.com/2020/10/13/po...ive-attitudes-a-word-on-color-and-conversion/

 

[ColSebastianMoran]

An update: The FilmLab app for iPhone & Android now has a desktop version, and a new update. Processes RAW files from our camera-scans. The quality of the conversions is a bit better with this update. Very simple controls, choices for output (JPG or TIFF). Output appears limited to sRGB. Not as good as Lightroom and Negative Lab Pro, but a reasonable, very quick conversion.

Here's one of my test shots, the Tokyo street scene, from a good Sony RAW cam-scan, converted in FilmLab Desktop, minor adjustments in that program.

 

[Peter Jennings]

I just got back a few rolls of Portra and was dreading attempting to convert them since I let my Adobe Cloud account expire. I don't shoot that much color, and re-subscribing just to use Lightroom with NLP to scan these rolls was not an option. So, I tried the free trial of the FilmLab desktop app on my Mac and I couldn't be happier. It may not have all the bells and whistles of NLP, but it gets the job done - and done well. And, it's a standalone app! It offers monthly, yearly, and lifetime subscriptions. Very convenient.

 

[shijan]

Hello. I start to investigate film negative scanning questions about 10 years ago and currently develop some really fast but same time technically correct workflow.
So first let me share some thoughts with you and point to some mistakes. Please note that these tips are not for processing inside RAW editors and plug-ins, but for processing inside normal graphic editors.

In short workflow looks like this:
Debayer RAW file and convert it from Camera Input ICC profile to ProPhotoRGB with L* TRC custom-made ICC profile -> Invert -> Apply RGB AutoLevels -> Recover data clipped by RGB AutoLevels -> Adjust Contrast.

White balance camera to your light source and don't change WB anymore in RAW file.
You may notice that this is the first essential task that every real scanner do when you turn it on. It calibrates sensor WB to light source.
When you attempt to pick orange mask in RAW and change WB, this cause shift in input profile and as result colors became incorrect after Invert. Colors will shift a lot depending of Color Space and Gamma you use. I described this problem in details here https://github.com/darktable-org/darktable/issues/6770#issuecomment-726468734

Do Invert and per channel RGB AutoLevels in Wide Gamut Color Space and L* Tone Response Curve (TRC).
Use L* TRC ICC profiles form Elle Stone's Well-Behaved ICC Profiles package https://github.com/ellelstone/elles_icc_profiles (click to green button named "Code" and select "Download ZIP")
Note that Elle Stone use name "LargeRGB" instead of "ProPhotoRGB", and name "labl" instead of L*. To avoid possible compatibility problems i suggest use V2 ICC profile type, but not V4. So in that package you need to look for LargeRGB-elle-V2-labl.icc
You can also create custom-made working ICC profiles based on L* TRC with DisplayCAL Synthetic ICC app https://displaycal.net/, or with RawTherapee ICC Profile Creator http://rawpedia.rawtherapee.com/ICC_Profile_Creator

Most free as well as commercial tools usually do Invert in sRGB Tone Response Curve. This cause shifts in tonality and require more aggressive manual curve adjustments with too many "guesses" and "fixes". Working gamma affects highlights rolloff look and so film scans inverted in sRGB TRC usually start to look "too digital". Also in sRGB TRC Auto Levels may produce very different result if applied before or after Invert.
Don't forget to convert final image from L* TRC to sRGB TRC for final 8 bit JPEG export.
Hope soon or late developers will add L* TRC to their plugins and so final processed images will look even better. Not sure if this technically possible in Lightroom and similar RAW editors where color management options are hidden and too restricted.

Optionally you need to Calibrate your camera or scanner. To create custom Input ICC profile use IT8.7 Scanner Calibration Target like these http://www.targets.coloraid.de/ and use Rough Profiler app http://www.jpereira.net/descargar-rough-profiler-software-pefil-color-icc-free to generate "Lab cLUT" or "Single Gamma + Matrix" ICC profile type. This step is more-less optional for DSLR cameras because in most cases developers provide more-less ok input profiles for them.
Manual calibration also helps a lot to fix problems with low quality LED backlight sources.

 

[shijan]

And here is my step by step workflow:

Calibration:
- To create custom Input ICC profile use IT 8.7 Scanner Calibration Target http://www.targets.coloraid.de/
- Use Rough Profiler app http://www.jpereira.net/descargar-rough-profiler-software-pefil-color-icc-free to generate Lab cLUT based ICC profile.

Scanning:
- Scan film negatives in positive mode to 16 bit with linear gamma. Scanner color management should be turned off.
- Use manual exposure.
- Use scanner autofocus by manually selected point.
- Do NOT remove Orange mask during scanning with custom gain adjustment for each channel. This is same as WB adjusted in RAW and will cause problem with incorrect input color space described here https://github.com/darktable-org/darktable/issues/6770

RAW Processing.
I recommend to use Iridient Developer https://www.iridientdigital.com/ It can disable build-in camera factory correction curve and can export untouched TIFF with Linear gamma same as scanners do. It also can do internal color management processing and export TIFF to any ICC profile that you select. Debayer and sharpness options are also very nice.

- Open RAW file in Iridient Developer
- Remove build-in Camera Contrast curve.
- Use minimum possible Sharpness and Chroma Noise Reduction, because small changes in source image will produce large color shifts when AutoLevels applied.
- Debayer and convert from Camera Input ICC profile to ProPhotoRGB L* TRC working ICC profile.

Development (can be recorded to one click action):
I recommend to use PhotoLine image editor because It is probably the only existing graphic editor app that allow to manipulate 8 or 16 bit images contained inside 32 bit depth project. Also it's Curves and Levels tools are way better than any other app i ever used before. PhotoLine also can read RAW files and can disable build-in camera contrast curve, but it have not the best debayer options and the concept of manipulation with RAW in that app is rather unusual and confused.
It is always better to download latest beta version from the forum (which is formally always a release with new bugfixes) here https://www.pl32.com/forum3/viewforum.php?f=1 PhotoLine is developed by very small team, so original UI icons are not too pretty, so i can recommend to use my specially designed UI Icons theme available for free here: https://www.deviantart.com/shijan/art/825902404

- Open file in PhotoLine.
- Flip/Rotate (optional)
- Switch to "Document mode"
- Set Document to 32 bit depth
- If image was already transformed to ProPhotoRGB L* TRC ICC profile, Document should automatically use same ICC profile as well.
If image was in Linear gamma, you need manually assign Working ProPhotoRGB L* TRC ICC profile to the Document and Input Camera ICC profile with Linear gamma to the Image Layer. This will provide color management from Input Image Layer ICC to Working Document ICC. In Document Mode PhotoLine use non destructive (non baked) color management.
L* Tone Response Curve (TRC) is the core of this workflow. L* TRC will help to invert negative without unwanted tonal shifts.
32 bit depth mode will help to recover data clipped by RGB AutoLevels during editing in a quick and easy way.
- Crop black frame (in Document Mode original pixels are never deleted, so you can recover original frame later if needed), or just select area without black frame.
- Add "Curves" adjustment layer, and do "Invert".
- Add "Levels" adjustment layer and use "Auto Correction: Auto Mode for All channels" to crop huge amount of empty data from RGB channels (you can experiment here with clipping settings to subjectively find the best variation of colors produced by AutoLevels)

- Turn on "Mark Extreme values" to see clipped highlights and shadows.
- In same "Levels" tool adjust "Output Levels", to recover clipped dynamic range in highlights and shadows.
(Do NOT touch Gamma slider at this step, because this will damage original relations between colors. Do NOT attempt to use "Output Levels" like this in apps that process image only in 16 bit. Recovery like this only possible in apps that can operate in real 32 bit depth.)

- Add "Exposure" adjustment layer and adjust "Luminance".
(Do NOT use "Luminance" slider in "Curves" tool. By mistake it is named same as "Luminance" in "Exposure" tool, but actually adjusts image in very different way. Problem was described here https://www.pl32.com/forum3/viewtopic.php?p=48627#p48627)

- Add "White Point" adjustment layer to fine tune color Grey Point. Select in settings "Setup Grey Point" and "Fix White Point" and pick some grey point somewhere in your photo.

- Image will look slightly low contrast and slightly desaturated now, so you need to add another adjustment layer and adjust Contrast with s-curve or use 1D LUTs with Contrast presets as i usually do.
(Instead of separated Luminance and Contrast layers some people prefer to use single Curve layer to shape tonality and contrast together.)

- Export image converted to normal sRGB JPEG for preview, or to 16 bit ProPhotoRGB L* TRC TIFF for printing.

And to summarize, here is a quick video for Film Negative Invert in L* TRC in PhotoLine:
https://youtu.be/524_wmkhXSY

 

[shijan]

Also as all-in-one solution for those who prefer less technical workflows with RAW editors and abstract sliders i can recommend darktable with negadoctor module. It 100% free and don't require Adobe apps
https://www.darktable.org/usermanual/en/module-reference/processing-modules/negadoctor/
https://www.youtube.com/results?search_query=darktable+negadoctor

 

[shijan]

Here is side by side comparison based on camera RAW scan sample from other forum. Only Invert and AutoLevels where applied here. As you can see L* Tone Response Curve (TRC) subjectively produce the most realistic and symmetrical result that usually require very small amount of additional manual correction. It also produce the best separation between bright skin tones and extreme highlights.

And some examples of processing in 16 vs 32 bit:

This is how Output Levels work in 8 or 16 bit (clipped data lost and can't be recovered with "Output Levels"):

This is how Output Levels work in 32 bit (clipped data recovered with "Output Levels"):

 

[ColSebastianMoran]

@Shijan, thanks for the posting.

And, thank you for converting my test image. I assume you used the RAW file DSC9558 linked earlier in this thread. Can you mention how you used that file, where it entered your process? Did you just open it in Photoline and then go to 32bit and L*Gamma?

 

[shijan]

Yes, it is DSC9558 from start of this thread.
I just process those images in the same way as described in earlier post.

In Iridient Developer:
- Remove Camera Contrast curve.
- Debayer and convert from Camera Input ICC profile to L* TRC ProPhotoRGB working ICC profile.
- Do not use large amount of Chroma Noise Reduction, because it may change some colors. And even small changes in source colors may produce large color shifts later with AutoLevels.
- Export to 16 bit Tiff.
Probably instead of Iridient Developer you may use MakeTIFF, export to Linear gamma and then convert in PhotoLine to L* TRC. But i have no idea how MakeTIFF deals with color spaces. If it only transforms gamma and skips color spaces, you probably need to find proper ICC input profile for your camera or build your own input ICC profile based on IT.8.7 target scan.

Open in PhotoLine:
- Switch image to 32 bit mode
- Add Curves adjustment layer and do Invert (use Curves because there is no separate "Invert" tool in PhotoLine)
- Crop film frame (Or select area of the actual photo without black frame)
- Add Levels adjustment layer and do Auto Correction "Auto Mode for All channels" aka RGB AutoLevels (you can experiment here with clipping level settings to find the best variation of AutoLevels colors)
- Recover back clipped data with Output Levels.
- Add White Point adjustment layer, select in settings "Setup Grey Point" and "Fix White Point" and pick some grey point somewhere in your photo.
- Image will look slightly low contrast and slightly desaturated now, so you need to add another adjustment layer and adjust tonal curve and Contrast as you like
- Export for WEB to sRGB color space and sRGB gamma.

Also for best possible color in single click you need:
- Quality and uniform light source.
6500 high CRI (95+) LEDs should be OK. Halogen lamps combined with 80A filters may work, but produce huge amount of IR pollution. Digital sensors are sensitive to IR light and so final images may have brownish-magenta tint that may affect RGB AutoLevels and so may produce incorrect color. (this is known problem with with digital cameras but i didn't test this in real life with film scans)
- Custom-made input ICC profile in most cases will make things look better.
- Avoid light leaks, lens flares or reflections.
- Scratches and dust on the film may affect RGB AutoLevels and so produce incorrect color. So make sure you inspect and fix and all artifacts.
- Without proper White Balancing of RAW file to Light source colors will shift or clip.
- During camera shutter speed adjustment don't clip Red Channel in Orange mask. Clipping will destroy original relations between colors. Exposure to the right, but always keep some free empty space at the right side of histogram.
- Export TIFF images from RAW processors in ProPhotoRGB or BestRGB color spaces with L* gamma TRC. Those color spaces where designed to match average native film color space as close as possible.
Rec2020 and ACEScg color spaces with L* gamma TRC are also OK, but they use 6500K and 6000K white point instead of 5000K commonly used in most color spaces designed for image editing apps.
Do not use sRGB or AdobeRGB with L* gamma TRC as working color spaces, because they are way smaller than native film color space and will produce a lot of clipping and unwanted color shifts after invert.
Do not use ACES with L* gamma TRC color space. It is too huge compare to native film color space and may produce unwanted color shifts after invert.

 

[shijan]

I also want to share some very subjective personal thoughts about alternate apps that i use and don't use:

darktable negadoctor standalone app
https://www.darktable.org/
Free all-in-one tool. Allow native RAW workflow. Nice debayer quality and nice options for Fujifilm X-Trans sensors. For my personal taste it is too "consumer" and too abstract to use. It works OK with simple images, but don't allow to fine tune films with complicated colors and unusual tonality. darktable allow to use custom working color spaces but looks like operates with film negatives only in sRGB gamma. Controls may feels not too intuitive for the first look, but there are a lot of video tutorials on Youtube and Vimeo.

Negative Lab Pro plug-in for Adobe Lightroom
https://www.negativelabpro.com/
Negative Lab Pro package includes custom-made input profiles for most camera models. Those profiles are specially improved and adjusted for film scanning. Overall it produce near same result as darktable negadoctor, and from some point of view it feels slower and less comfortable to use. It have a lot of near useless consumer presets and "looks" variations.

ColorPerfect plug-in for Adobe Photoshop
https://www.colorperfect.com
Worst investment ever. Overpriced app that seems like designed to made impression of something "very complicated" and "very important". Worst ever made Interface. Very slow processing. Low quality results. Clipped dynamic range. Hard clipped contrast adjustment. No real color management. Can't read camera input color profile. Communication with developer is possible but goes to nothing. Instead of clean answers to simple straight questions he wrote back giant pseudo technical texts that explains nothing and suggest to process images in sRGB color space.

FilmLab standalone app
https://www.filmlabapp.com/desktop
I check demo version and it feels like some joke. Very limited controls. Colors are always off and it feels like it attempt to prettify image and add some sort of fake "filmic look" LUT on top.

Grain2Pixel plug-in for Adobe Photoshop
https://grain2pixel.com/
I attempt to test Grain2Pixel, but can't run it because it require newest version of Photoshop. Very strange limitation.

CNMY plug-in for Adobe Photoshop
https://eigakai.ro/ps-action/cn-scan-inversion
I didn't test this app yet, but hope to check it someday.

Negmaster plug-in for Adobe Photoshop
https://negmaster.com/
I didn't test this app yet, but hope to check it someday.

Analogue Toolbox for Capture One plug-in for Capture One
https://www.mwilmes.com/toolbox
I didn't test this app yet, but hope to check it someday.

SILKYPIX Negative film inversion tool standalone app
https://silkypix.isl.co.jp/en/how-to/function/the-negative-film-inversion-tool/
I didn't test this app yet, but inversion examples on their website looks very bad. Hope to check it someday.

VueScan standalone app
https://www.hamrick.com/
This is another example how to make simple things in too complicated and non human friendly way. It works, it scans, but i hate the logic of this app and hope i will never to use it. Also it may produce strange pixel patterns artifacts with some film scanners models. Have no idea was this problem fixed or not, but i always recommend to use original scanners drivers and original software if possible.

PhotoLine standalone app
https://www.pl32.com/
Sort of "Secret weapon". Without digging into details, from technical point of view compare to Photoshop it is like intergalactic spaceship compare to basic airplane from 1970 era.
PhotoLine was started in early 90's (near same time as Photoshop). It is made by very small team of developers in Germany and not too advertised as most other graphic editors. Developers focused mostly on technical side and don't care too much about interface look and pretty icons. So visually it is just not as pretty looking as Photoshop and i guess this is the main reason why many people reject to use it. Tools and logic in PhotoLine in most cases are superior to Photoshop, but some tools in some places feels slightly limited and require some updates and improvements.
I started to use it about year ago and same time start to help on forum with bugfixes and ideas as a volunteer. During this time developers add huge amount of changes, bugfixes and improvements. I also start PhotoLine UI Icons Customization Project to make it look more user friendly https://www.deviantart.com/shijan/art/825902404

Iridient Developer standalone app
https://www.iridientdigital.com/
Very solid, powerful and simple RAW editor. Great debayer options and sharpen quality. Probably the best one for Fujifilm X-Trans sensors. A lot of other unique options. Quick and easy to use. UI is very basic and not too pretty.

 

[shijan]

I also want to add some info about so-called inversion Log curves. I guess first time it came to film scanning masses in early Photoshop era as Film-Negative package by Timo Autiokari. That website was closed long time ago but original page "Scanning Negative Film using a Flatbed scanner" still archived here http://web.archive.org/web/20080502164100/http://www.aim-dtp.net/aim/techniques/index.htm
To be honest i still have no idea how exactly people generate those curves, and what log curve they use as a reference, but formally this method is technically incorrect and produce some problems:
- Photoshop curves in .amp format are very legacy and use low resolution data. Formally once applied, they produce low quality 8-bit starting point with further posterization and banding problems. It is very easy to see a gaps if take a look at histogram or at high resolution visualization.

- As you may know, Photoshop itself use 15 bit internal processing instead or real 16 bit, so this also may cause some additional tonality data loss during these extreme transformations.
- Inverted log curve violates normal ICC profile based color management inside apps.
- As you may know, Contrast and saturation are linked. So in normal digital workflows increased contrast always produce increased saturation in the image. But when you use inverted log curve relation between contrast and saturation became the opposite and you need manually increase saturation with other additional tools. Sometimes this flipped relation between contrast and saturation may produce interesting artistic effects.
Overall this method may take a place, but it just needs better quality curves and tools.

Here is also comparative visualization of different inversion methods:

 

[shijan]

Same L* TRC workflow is possible in Photoshop in 16 bit, but AutoLevels adjustment will be not so quick and easy there. Also there is no special tool for Grey Point in Photoshop, so you need to set it with Curves.

So here is how to deal with AutoLevels in Photoshop:

- Invert
- Add Levels
- Hold "Alt" key and press button "Auto" to open tool preferences.
- Select "Enhance Per Channel Contrast"
- Select Shadows/Highlights Clip somewhere from 0.01 to 0.1 (you can experiment here with clip level settings to find the best variation of colors)
- Select "Save as Defaults" so same settings will be applied next time when you simply click to button "Auto" without holding "Alt".
- Do NOT check "Snap Neutral Midtones" because it will change original relations between colors.

- Now you need to recover clipped data. In Photoshop this could be done only by adjusting Input Levels in each channel to same amount of numbers.
(Do NOT touch "Gamma" or "Output Levels" sliders at this step, because this will damage original relations between colors)

- Add Curves layer and pick grey point somewhere
(This is same as "Snap Neutral Midtones" but you appy it in proper place and you can manually control it.)

- Add another adjustment layer and adjust tonal curve and contrast as you like (Same as in PhotoLine workflow)

And to summarize, here is a quick video for Film Negative Invert in L* TRC in Photoshop: