Image Aesthetics and Sensor Color Filter Materials

[willie_901]

The International Image Sensor Society conducts workshops for researchers and engineers to share information.

This link leads to a PDF file titled "Technology of color filter materials for image sensor"

This short paper discusses how the materials used in the Bayer color-filter array affect image aesthetics.

Here is a summary.

• There is a trade-off between signal-to-noise ratio (shadow region detail) and color aesthetics.

As the filter efficiency increases, the SNR decreases. But more effective filtering increases color quality because the R, G and B sensor sites' raw values are not contaminated by light that is not pure R, G or B.

• As pixel density (megapixel count or resolution) increases, the signal level per pixel decreases.

This places more pressure on the SNR vs color fidelity trade off. Advances in filter material technology have reduced "color errors" while increasing signal levels (the light transmitted by each color filter to the photo diode).

• Filter material manufacturing methods plays an important role in color fidelity

Another factor in color aesthetics is the homogeneity of of the filter materials. When those millions of filters have different light transmission properties, raw rendering is compromised, The signals (voltage per pixel site) do not match the Bayer model. New manufacturing technologies reduce partial sizes in color filter materials which in turn lowers the deviation in per pixel light transmission. The result is increased color fidelity.

• Color filter thickness affects image color quality.

Technical improvements that improve photo-diode SNR (back side illumination) also increase "cross-talk" between pixel sites. Light with the wrong frequency leaks into adjacent sensor sites. This degrades color aesthetics. Thinner color filter materials minimize this problem. However, thinner color filters that are not susceptible to the limitations discussed above are more expensive to manufacture. Incidentally, most contemporary CMOS sensor beds are BSI devices.

 

[mrmeadows]

Willie_901:

Very interesting article. Thanks for the link.

I've not previously read about CMs, but the article prompts a question that I've not found an easy answer to but which you may be able to help me with. I assume that the overall goal for a color sensor is to have the combined spectral responses of the CM, lens array, IR filter and silicon diode to be as close as possible to the RGB color matching functions of, say CIE 1931 (cf https://en.wikipedia.org/wiki/CIE_1931_color_space). To do this, the spectral response of the filters cited in the article will presumbaly have their red response cut in the long wavelenths by the IR filter and the CM's blue response will raised relative to the red and green by the intrinsic response of the diodes. Is this true? Are there other important aspects of the sensor that will affect its overall spectral response? Why is there no apparent effort to produce the significant minor peak in the CM red response that exists in the CIE red function?

--- Mike

 

[mrmeadows]

Willie_901:

Previously, I wrote: "... and the CM's blue response will raised relative to the red and green by the intrinsic response of the diodes."

Oops. That can't be true. The intrinsic diode response will reduce, not increase, the blue response.

So what's the story?

--- Mike

 

[Dwig]

...
Here is a summary.

• There is a trade-off ...

and to summarize the summary, you're damned if you do and damned if you don't.

 

[willie_901]

Willie_901:

.... I assume that the overall goal for a color sensor is to have the combined spectral responses of the CM, lens array, IR filter and silicon diode to be as close as possible to the RGB color matching functions of, say CIE 1931 (cf https://en.wikipedia.org/wiki/CIE_1931_color_space).
--- Mike

I would say the goal of the color-filter array is to solve this problem:

"The problem of the interpolation of Bayer patterns (also referred to as demosaicing or demosaicking) consists in obtaining a full-color image by its Bayer pattern. The purpose of the algorithm is to interpolate each color plane at the points where the value of the corresponding color component is unknown."
link

That is: the empirical data should match the model for the data.

The Bayer color reconstruction model assumes the light pattern contains only pure R, G or B frequencies in a specific spatial pattern. This is not possible because perfect filters do not exist. Bayer interpolation algorithms are modified to compensate for the differences (errors) between the data and model. In some raw rendering platforms these customized models are called camera profiles. However modifying the model has limits. The M8 IR contamination problem is an example of the limits of interpolation modification. A demoasicing fix was not possible.

Here is an example of color error formulas.

As far as I know, raw data files do not have a color space. The color space is created after demosaicing. Of course contamination of RGB sites can make it impossible to map the data onto the color space.

To do this, the spectral response of the filters cited in the article will presumbaly have their red response cut in the long wavelenths by the IR filter and the CM's blue response will raised relative to the red and green by the intrinsic response of the diodes. Is this true? --- Mike

This issue involves the quantum efficiency frequency dependence in photodiode response.

"Many CMOS sensors have a yellow polyimide coating applied during fabrication that absorbs a significant portion of the blue spectrum before these photons can reach the photodiode region. Reducing or minimizing the use of polysilicon and polyimide (or polyamide) layers is a primary concern in optimizing quantum efficiency in these image sensors."

link

So the sensor assembly design and manufacturing process do attempt to compensate for the issues you raise.

This linked article addresses other technical issues that might answer your questions in more detail. The article has graph for a typical (?) CFA transmission spectral profile. Notice how the filter frequency transmission overlaps. Less overlap means less color error yet it also reduces the total signal level.

 

[willie_901]

[/LIST] and to summarize the summary, you're damned if you do and damned if you don't.

Exactly.

Which is how come color-filter materials and manufacturing advances have a significant, but unrecognized, impact on image aesthetics (signal-to-noise ratio and color fidelity).

 

[mrmeadows]

Willie_901:

Thanks for the discussion and the links.

I believe that I was unclear when I speculated in my first posting about the "overall goal" of a color sensor. My meaning of "overall goal" is that goal that applies to every color sensor, whether it has a Bayer CMA, is a three-sensor (RGB) system, is a Foveon-type or is a field-sequential-color type. For all of them it seems to me that the overall goal is to accurately sense all the colors that the human eye can sense. A perfect sensor would achieve the same color gamut as the eye, which is the LMS space provided by the long wave (L), medium wave (M) and short wave (B) sensitivities of the three type of cones in the human retina. That overall goal would then be attained, I contend, if the RGB spectral responses of a sensor were the same as the LMS responses of the cones, or if the sensor responses were equivalent to the LMS responses through a non-singular transformation, as are the CIE 1931 X, Y and Z tristimulus values through the x-bar, y-bar and z-bar color matching functions.

This overall goal is an ideal, of course, and no current sensors achieve it. In service to the overall goal, the engineering goals and challenges will be somewhat different for each sensor type. In all four cases I believe, as your discussion and links confirm, that the color filters afford the chief opportunity for tuning the overall response. The color filters seem to offer a wide range of tuning possibilities, whereas there is much less flexibility in tuning the spectral characteristics of the other components of a color sensor. The Bayer arrays have additional problems relative to the others, viz. pixel color cross-talk, demosaicing algorithms, unique SNR considerations, general fidelity to the Bayer reconstruction model, etc., and these appear to be where much of the current development is centered, as described in the links that you have provided.

Again ... my thanks for the discussion. Although these sorts of technical considerations have little or no bearing on my ability to find and to use my equipment to make satisfying photographs, I like to understand the technical side of photography as best I can, and your discussion has been very helpful to me.

--- Mike