craygc
Well-known
rscheffler;2502773But you still want to be as close to clipping highlights as possible in order to get as much light from the shadows. [/QUOTE said:
Avoiding blown highlights in digital night photogprahy
- Thread starter thereabouts
- Start date
- Latest activity Latest activity:
- Replies 46
- Views 9K
thereabouts
Established
Thanks rscheffler.
It's (or rather Sony has) essentially changed the ISO relationship with photography. So, although I am underexposing in the traditional sense. In practice, I would be utilising the extended dynamic range of shooting at 100 ISO and post-processing afterwards, to achieve the same (but customised for my purposes) result.
Apologies from me also, if I have misunderstood either your or Craig's posts.
As I understand from what Craig has been saying, with regards to the Sony sensors, I would still be shooting at a fast shutter speed at 100 ISO (as I did with my quick and dirty test above), but recovering the shadow details in post-processing. So shutter speed should not be an issue.
It's (or rather Sony has) essentially changed the ISO relationship with photography. So, although I am underexposing in the traditional sense. In practice, I would be utilising the extended dynamic range of shooting at 100 ISO and post-processing afterwards, to achieve the same (but customised for my purposes) result.
Apologies from me also, if I have misunderstood either your or Craig's posts.
rscheffler
Well-known
Sorry, if I didn't write it clearly enough, but that was also my point. You will get the most possible dynamic range out of a high contrast scene, if it is similar to, or greater than the dynamic range of the sensor when you shoot at base ISO and you adjust your exposure (shutter and aperture) to be as close to clipping as realistically possible. The value of an ETTR exposure is the more light you can put on the sensor before clipping highlights, the more information you will have to work with in the shadow values and the cleaner they will be, assuming the contrast range of the scene is greater than that of the sensor. This is particularly the case in night scenes where most of the scene will be very poorly illuminated. That said, there is debate about the effect of ETTR on color reproduction, IIRC.
Where there could be value in increasing in-camera ISO is in achieving usable images prior to post processing (such as for evaluation purposes), if the scene's brightness range remains within the dynamic range of the sensor at that ISO, rather than using the same shutter/aperture combination at a lower ISO.
My familiarity is more with the Leica M9 and M240. With the M9, it was quantifiably measured by Jim Kasson that setting ISO between 160 and 640 had virtually no difference between pushing in post. That's to say, if exposure at ISO 160 was two stops under, you would get the same results in post compared to shooting at ISO 640 with the same shutter/aperture combo and not clipping the highlights.
The practical benefit of shooting at base ISO all the time while protecting highlights is the ability to push in post exactly as much as is required when you don't have time to optimize shutter/aperture settings. This is very useful in scenes that are fluid where one view to the next may have different exposure ranges and there isn't time to fine-tune the exposure.
Where there could be value in increasing in-camera ISO is in achieving usable images prior to post processing (such as for evaluation purposes), if the scene's brightness range remains within the dynamic range of the sensor at that ISO, rather than using the same shutter/aperture combination at a lower ISO.
My familiarity is more with the Leica M9 and M240. With the M9, it was quantifiably measured by Jim Kasson that setting ISO between 160 and 640 had virtually no difference between pushing in post. That's to say, if exposure at ISO 160 was two stops under, you would get the same results in post compared to shooting at ISO 640 with the same shutter/aperture combo and not clipping the highlights.
The practical benefit of shooting at base ISO all the time while protecting highlights is the ability to push in post exactly as much as is required when you don't have time to optimize shutter/aperture settings. This is very useful in scenes that are fluid where one view to the next may have different exposure ranges and there isn't time to fine-tune the exposure.
rscheffler
Well-known
Thanks. BTW, my previous reply was directed at Craig's post following mine.
Yes and no.
Yes, you shoot at base ISO and a high shutter speed and push in post. This will give the same end result as setting a correspondingly higher ISO in the camera (with most Sony models, and from some other brands too).
That said, no sensor will magically make something out of nothing. If virtually no light reaches the sensor, you will not be able to create a wide dynamic range scene in post.
It also depends on what your technical expectations are.
The more light you can put on the sensor at base ISO by choice of shutter speed and aperture, without clipping the highlights, (essentially a 'correct' exposure) the more information you have to work with and the cleaner your images will be because you won't have to push them as much, if at all.
The problem you've been having with digital is that you're clipping the highlights you consider important to the scene. By shooting at a high ISO, you've essentially told the camera to push the exposure too far. And unfortunately when this happens in the camera, it's permanently locked into the RAW file too. Shooting at a lower ISO would have better protected the highlights because there was less in-camera push. Shooting at base ISO would be the best guarantee of this. However, if base ISO is underexposing highlights, then it's not fully optimizing the dynamic range potential of the sensor.
Craig's point (and mine) is that base ISO gives you the widest possible dynamic range as a starting point. Exposure controls how much of the scene's dynamic range is put on the sensor. If you can put a 'full' amount of exposure on a sensor, you will achieve the maximum technical results possible.
But sometimes that isn't possible, such as your situation when you're hand holding and need a certain minimum shutter speed and can't correspondingly open the lens sufficiently to maintain the correct base ISO exposure (or need deep enough depth of field). Here you'll end up underexposing the scene. With most current Sony sensors, that's OK because pushing it in post will result in the same final result as shooting it at a higher ISO in-camera. Doing it this way allows you to precisely push the image in post, rather than semi-guessing what the in-camera push (higher ISO) should be. If you overshoot with a higher ISO than necessary, the clipped highlight values will be permanently clipped in the RAW file.
I'm not sure my explanation is the clearest. I know in my mind how it works, sometimes it doesn't quite come out that way. 🙂 I would suggest doing some tests, on a tripod, just to figure it out. Tripod so you can directly compare between exposures. Find a scene you like, set the ISO to 100 and start at some ridiculous high shutter speed, like 1/4000, then work your way down the shutter speed range until you're definitely blowing highlights more than you think you should, like 2-3 stops. Tweak the images into your RAW converter and see which exposure you like best in terms of highlight appearance while giving you the best mid and shadow representation. This would then be the ideal exposure, giving you the fullest dynamic range and cleanest shadows. You may find results will be good enough for your needs from underexposed scenes. Let's say the -3 stop scene is more realistic from a hand holdable shutter speed perspective. it will be the same end result as setting the camera to ISO 800. Highlights will probably be similar to the ideal ISO 100 exposure, but shadow noise will be somewhat higher. You will have to decide at what point the tradeoff in shadow noise is no longer warranted by the higher shutter speed. This should give you an idea of how much you can push the files while still getting the technical results you want.
Hope this makes sense! 🙂
Yes and no.
Yes, you shoot at base ISO and a high shutter speed and push in post. This will give the same end result as setting a correspondingly higher ISO in the camera (with most Sony models, and from some other brands too).
That said, no sensor will magically make something out of nothing. If virtually no light reaches the sensor, you will not be able to create a wide dynamic range scene in post.
It also depends on what your technical expectations are.
The more light you can put on the sensor at base ISO by choice of shutter speed and aperture, without clipping the highlights, (essentially a 'correct' exposure) the more information you have to work with and the cleaner your images will be because you won't have to push them as much, if at all.
The problem you've been having with digital is that you're clipping the highlights you consider important to the scene. By shooting at a high ISO, you've essentially told the camera to push the exposure too far. And unfortunately when this happens in the camera, it's permanently locked into the RAW file too. Shooting at a lower ISO would have better protected the highlights because there was less in-camera push. Shooting at base ISO would be the best guarantee of this. However, if base ISO is underexposing highlights, then it's not fully optimizing the dynamic range potential of the sensor.
Craig's point (and mine) is that base ISO gives you the widest possible dynamic range as a starting point. Exposure controls how much of the scene's dynamic range is put on the sensor. If you can put a 'full' amount of exposure on a sensor, you will achieve the maximum technical results possible.
But sometimes that isn't possible, such as your situation when you're hand holding and need a certain minimum shutter speed and can't correspondingly open the lens sufficiently to maintain the correct base ISO exposure (or need deep enough depth of field). Here you'll end up underexposing the scene. With most current Sony sensors, that's OK because pushing it in post will result in the same final result as shooting it at a higher ISO in-camera. Doing it this way allows you to precisely push the image in post, rather than semi-guessing what the in-camera push (higher ISO) should be. If you overshoot with a higher ISO than necessary, the clipped highlight values will be permanently clipped in the RAW file.
I'm not sure my explanation is the clearest. I know in my mind how it works, sometimes it doesn't quite come out that way. 🙂 I would suggest doing some tests, on a tripod, just to figure it out. Tripod so you can directly compare between exposures. Find a scene you like, set the ISO to 100 and start at some ridiculous high shutter speed, like 1/4000, then work your way down the shutter speed range until you're definitely blowing highlights more than you think you should, like 2-3 stops. Tweak the images into your RAW converter and see which exposure you like best in terms of highlight appearance while giving you the best mid and shadow representation. This would then be the ideal exposure, giving you the fullest dynamic range and cleanest shadows. You may find results will be good enough for your needs from underexposed scenes. Let's say the -3 stop scene is more realistic from a hand holdable shutter speed perspective. it will be the same end result as setting the camera to ISO 800. Highlights will probably be similar to the ideal ISO 100 exposure, but shadow noise will be somewhat higher. You will have to decide at what point the tradeoff in shadow noise is no longer warranted by the higher shutter speed. This should give you an idea of how much you can push the files while still getting the technical results you want.
Hope this makes sense! 🙂
willie_901
Veteran
The answer to your question is simple for raw files.
Maximize the sensor exposure when the shutter is open such that the highlights required for the photograph you envision are recorded by the sensor.
In other words: maximize the analog signal-to-noise ratio modeled by the raw file.
This means one has to know something about the camera's data stream characteristics. For the OP's camera use the sensor's native ISO (100 or 200) and a shutter time appropriate for the handicap of not using a tripod (or to freeze subject motion) while maximizing exposure. This may not be the case for other brands or for different models within the same brand because in some cases the SNR has a significant dependence on ISO.
Then selectively increase the brightness of under-exposed regions in the image while selectively decreasing the highlight regions' brightness during post production.
It is often useful to automatically bracket exposures. This overcomes unavoidable inaccuracies in meter/histogram estimates. I use either 1/3 or 1/2 stop brackets. For the OP's work I would bracket the aperture. It might not be overkill to make 4 to 6 different exposures of the same static scene. One of these exposure will be close to the best possible exposure. It will have the highest possible information content.
There's really nothing else one can do.
-----------------------------------
The technical reasons behind this advice follows.
o The only way to preserve highlights is to use a shutter speed and aperture that does does not clip them.
Highlights can be clipped two ways.
The first is when the camera is set to the native ISO of the sensor. This varies from brand to brand and within different models of the same brand. The clipping in this case comes from exceeding the full-well capacity of the sensor sites which means the electrical charge for these sites exceeds the chip's storage capacity.
The second way to clip highlights has nothing to do with the sensor. In low light the shutter time and aperture typically underexpose the sensor (because ISO is above the sensor's native ISO). However analog signal amplification after the shutter closes (ISO) exceeds the analog-to-digital converters' maximum design DC voltage threshold. This clips the highlight regions even though the sensor is underexposed.
The meter is just an estimate and can be misleading. It would be a happy, but rare coincidence that the meter reading's accurately is not affected the bright light's color temperature. In-camera histograms (with a few notable exceptions) are also just estimates because the histogram is computed from a rendered JPEG. This means the histogram depends on in-camera JPEG rendering parameters. I am not claiming the meter reading or histogram display are worthless. I am claiming they are just a starting point in the process of optimizing exposure.
So, the loss of highlight detail is a caused of too much exposure or too much ISO amplification. Nothing else matters for highlights.
The shadow regions are another story. Selecting a shutter time and aperture that does not clip the sensor can have a profound affect on the shadow regions. The shadow regions have the lowest SNR. A shutter time and aperture exposure set to record all the highlight detail results in less exposure for the shadow regions. This is the reason a camera's data-stream read noise level is important. The lower the read noise the higher the shadow regions' SNR. This means more detail and more color accuracy in rendered images.
The data stream's analog dynamic range depends on the maximum signal levels the sensor can record and the read-noise level. This is the reason raw file DR is an important metric for digital camera performance. Increasing ISO always decreases the analog DR. When one does what the meter tells one to do, increasing ISO results in shorter shutter times and, or narrower apertures. Increasing ISO decreases the SNR... which in turn decreases the DR.
Heres's some data for a diverse range of cameras that computes the relative DR from statistical analysis of unrendered raw files. Just select the camera(s) of interest from the right-hand pane.
The term ISO-less has become popular in the past year or so. There are very few IOS-invariant cameras. ISO invariance means the read noise characteristics are statistically identical for all ISO settings. However many cameras can be used as though they are ISO invariant. For practical purposes, the read-noise differences are irrelevant for up to 3 to 5 EV of sensor underexposure. The OP's camera is one of these.
One should know how ISO affects the SNR for the camera in use. Using the optimum ISO setting delivers raw data with the best possible SNR and DR. SNR and DR have the greatest impact on the rendered images' aesthetics.
Maximize the sensor exposure when the shutter is open such that the highlights required for the photograph you envision are recorded by the sensor.
In other words: maximize the analog signal-to-noise ratio modeled by the raw file.
This means one has to know something about the camera's data stream characteristics. For the OP's camera use the sensor's native ISO (100 or 200) and a shutter time appropriate for the handicap of not using a tripod (or to freeze subject motion) while maximizing exposure. This may not be the case for other brands or for different models within the same brand because in some cases the SNR has a significant dependence on ISO.
Then selectively increase the brightness of under-exposed regions in the image while selectively decreasing the highlight regions' brightness during post production.
It is often useful to automatically bracket exposures. This overcomes unavoidable inaccuracies in meter/histogram estimates. I use either 1/3 or 1/2 stop brackets. For the OP's work I would bracket the aperture. It might not be overkill to make 4 to 6 different exposures of the same static scene. One of these exposure will be close to the best possible exposure. It will have the highest possible information content.
There's really nothing else one can do.
-----------------------------------
The technical reasons behind this advice follows.
o The only way to preserve highlights is to use a shutter speed and aperture that does does not clip them.
Highlights can be clipped two ways.
The first is when the camera is set to the native ISO of the sensor. This varies from brand to brand and within different models of the same brand. The clipping in this case comes from exceeding the full-well capacity of the sensor sites which means the electrical charge for these sites exceeds the chip's storage capacity.
The second way to clip highlights has nothing to do with the sensor. In low light the shutter time and aperture typically underexpose the sensor (because ISO is above the sensor's native ISO). However analog signal amplification after the shutter closes (ISO) exceeds the analog-to-digital converters' maximum design DC voltage threshold. This clips the highlight regions even though the sensor is underexposed.
The meter is just an estimate and can be misleading. It would be a happy, but rare coincidence that the meter reading's accurately is not affected the bright light's color temperature. In-camera histograms (with a few notable exceptions) are also just estimates because the histogram is computed from a rendered JPEG. This means the histogram depends on in-camera JPEG rendering parameters. I am not claiming the meter reading or histogram display are worthless. I am claiming they are just a starting point in the process of optimizing exposure.
So, the loss of highlight detail is a caused of too much exposure or too much ISO amplification. Nothing else matters for highlights.
The shadow regions are another story. Selecting a shutter time and aperture that does not clip the sensor can have a profound affect on the shadow regions. The shadow regions have the lowest SNR. A shutter time and aperture exposure set to record all the highlight detail results in less exposure for the shadow regions. This is the reason a camera's data-stream read noise level is important. The lower the read noise the higher the shadow regions' SNR. This means more detail and more color accuracy in rendered images.
The data stream's analog dynamic range depends on the maximum signal levels the sensor can record and the read-noise level. This is the reason raw file DR is an important metric for digital camera performance. Increasing ISO always decreases the analog DR. When one does what the meter tells one to do, increasing ISO results in shorter shutter times and, or narrower apertures. Increasing ISO decreases the SNR... which in turn decreases the DR.
Heres's some data for a diverse range of cameras that computes the relative DR from statistical analysis of unrendered raw files. Just select the camera(s) of interest from the right-hand pane.
The term ISO-less has become popular in the past year or so. There are very few IOS-invariant cameras. ISO invariance means the read noise characteristics are statistically identical for all ISO settings. However many cameras can be used as though they are ISO invariant. For practical purposes, the read-noise differences are irrelevant for up to 3 to 5 EV of sensor underexposure. The OP's camera is one of these.
One should know how ISO affects the SNR for the camera in use. Using the optimum ISO setting delivers raw data with the best possible SNR and DR. SNR and DR have the greatest impact on the rendered images' aesthetics.
Rob-F
Likes Leicas
This is new information for me. I didn't know about "ISO invariant." I wonder if any of my cameras have that feature? Nikon D300 and D700; Fuji x10, x20, X100; LeicaSonic D-Lux 6?
My Fuji X20 has a dynamic range selector that allows DR100%, 200%, or 400%. It allows DR400% at ISO 400 or higher; DR200% at ISO200 or higher; and DR100% at ISO 100 or higher. I wonder if that is related to ISO invariance?
Bob Michaels
nobody special
Ask yourself if those old rules about never having blown highlights or blocked up shadows really apply to night photography? Work at night photography long enough and hard enough and it is possible to get a night photo look like it was made in the daytime. But is that the effect we are trying to achieve? Or, are we just blindly following some rules that realistically do not apply?
Night scenes are seen by the human eye with burnt out highlights and blocked up shadows. Should our photographs bear some resemblance to what our eye perceives?
Night scenes are seen by the human eye with burnt out highlights and blocked up shadows. Should our photographs bear some resemblance to what our eye perceives?
Rob-F
Likes Leicas
That is a good point. In fact, when Hollywood crews cannot shoot a nighttime scene at night because of time constraints, they shoot "day for night," meaning they underexpose and use backlight and harsh shadows to get a night time effect in the daytime!
YYV_146
Well-known
Try DXOmark to check your camera's sensor performance, if the sensor behaves in a linear way, losing one stop of dynamic range per doubling of ISO, then it is in effect ISO invariant. For example, if your camera has 14 stops of DR at 100 and 11 stops at 800, you can brighten the ISO100 file by 3 stops and get an ISO 800 exposure file with the same 11 stops.
It doesn't have much to do with the DR optimizer, which only applies to Jpegs and not RAW files (at least in all the bodies I've used). The 200/400 limit is simply because the sensor needs to be at 200/400 to extend the DR range down to an equivalence of 100.
As a rule of thumb, most modern digital cameras, with the exception of Canon, are at some level ISO-invariant. At some point DR becomes too poor for an acceptable image, of course, but within those bounds in-camera ISO is basically just a signal about how much DR is being sacrificed for light.
Rob-F
Likes Leicas
Aha! I understand! This thread is being really helpful, thanks!
rscheffler
Well-known
Sure, no argument there. Night should look like night. When I read the OP I was a bit bewildered by the desire to retain tonality in specular light sources in such scenes.
But there is value in understanding how the equipment works; how to get a full exposure range. It provides a good base from which to work. It's what you do with it in post that matters more, IMO (in respect to the 'mood' of the image, separate from other critical picture elements).
JoeV
Thin Air, Bright Sun
I have the X10, which is similar to the X20. I understand they work in the dynamic EXR mode by interleaving two images one atop the other, using half the pixels for each frame, such that one frame is optimized for shadows and the other for highlights. So it's like an HDR setup, except the two frames are recorded simultaneously instead of one after the other. This makes it practical for handheld photography but the tradeoff is a 6 mp file instead of 12.
~Joe
Dwig
Well-known
Not quite. It's like HDR done from two different RAW conversions of the same original image, something that is just as practical handheld as the X10/20's trick. The two "exposures" are generated by processing the two set of photosites using different ISOs.
True multi-shot HDR uses two or more bracketed images; all of which are usually shot at the same ISO and the same aperture varying only the exposure time. What the X10/20's trick does it make use simple HDR images available to those who can't or won't do their own RAW conversions.
paulfish4570
Veteran
bracket, bracket, bracket where possible. i start at -1 stop in my x100, and -1/3 in my om-d e-m5, and work down from there ...
Ranchu
Veteran
http://home.comcast.net/~nikond70/Charts/PDR.htm
I thought this might be useful in determining which cameras have this iso invariant thing.
Rob-F
Likes Leicas
The Nikon Df looks impressive!
thereabouts
Established
Thanks for that. And yes, your post makes perfect sense and is very helpful.
Obviously, it is good to know the ideal to aim for. But it has been good to learn another 'legitimate' method of aiming for a better result.
thereabouts
Established
Absolutely. If you were to look at my back catalogue, you would see that I rarely follow the rules of photography as set down by… well whoever.
But, indeed, learning how best to use your tools, helps you break those rules with more precision. Learning what I have from this thread is going to prove useful.
Tonality? What tonality?
thereabouts
Established
But sometimes it's nice to be able to capture those details in the highlights. Something that I've found easier with film than digital, up until now.
Bob Michaels
nobody special
Personally, I see too many chasing technical perfection as indicated by histograms or other digital data and not paying enough attention to the visual impact of the actual photos themselves.
But maybe I just march to the beat of a different drum.
Similar threads
