alt Hacker NewsI love posts that peel back the abstraction layer of "images." It really highlights that modern photography is just signal processing with better marketing.
A fun tangent on the "green cast" mentioned in the post: the reason the Bayer pattern is RGGB (50% green) isn't just about color balance, but spatial resolution. The human eye is most sensitive to green light, so that channel effectively carries the majority of the luminance (brightness/detail) data. In many advanced demosaicing algorithms, the pipeline actually reconstructs the green channel first to get a high-resolution luminance map, and then interpolates the red/blue signals—which act more like "color difference" layers—on top of it. We can get away with this because the human visual system is much more forgiving of low-resolution color data than it is of low-resolution brightness data. It’s the same psycho-visual principle that justifies 4:2:0 chroma subsampling in video compression.
Also, for anyone interested in how deep the rabbit hole goes, looking at the source code for dcraw (or libraw) is a rite of passage. It’s impressive how many edge cases exist just to interpret the "raw" voltages from different sensor manufacturers.
Replies
The bit about the green over-representation in camera color filters is partially correct. Human color sensitivity varies a lot from individual to individual (and not just amongst individuals with color blindness), but general statistics indicate we are most sensitive to red light.
The main reason is that green does indeed overwhelmingly contribute to perceptual luminance (over 70% in sRGB once gamma corrected: https://www.w3.org/TR/WCAG20/#relativeluminancedef) and modern demosaicking algorithms will rely on both derived luminance and chroma information to get a good result (and increasingly spatial information, e.g. "is this region of the image a vertical edge").
Small neural networks I believe are the current state of the art (e.g. train to reverse a 16x16 color filter pattern for the given camera). What is currently in use by modern digital cameras is all trade secret stuff.
Another tangent. Bryce Bayer is the dad of a HN poster. https://news.ycombinator.com/item?id=12111995 https://news.ycombinator.com/item?id=36043826
I have a related anecdote.
When I worked at Amazon on the Kindle Special Offers team (ads on your eink Kindle while it was sleeping), the first implementation of auto-generated ads was by someone who didn't know that properly converting RGB to grayscale was a smidge more complicated than just averaging the RGB channels. So for ~6 months in 2015ish, you may have seen a bunch of ads that looked pretty rough. I think I just needed to add a flag to the FFmpeg call to get it to convert RGB to luminance before mapping it to the 4-bit grayscale needed.
> It really highlights that modern photography is just signal processing with better marketing.
Showing linear sensor data on a logarithmic output device to show how hard images are processed is an (often featured) sleight of hand, however.
Even old school chemical films were the same thing, just different domain.
There is no such thing as “unprocessed” data, at least that we can perceive.
> modern photography is just signal processing with better marketing
I pass on a gift I learned of from HN: Susan Sunday’s “On Photography”.
Also worth noting that manufacturers advertise photodiode count as a sensor resolution. So if you have 12 Mp sensor then your green resolution is 6 Mp and blue and red are 3 Mp
And this is just what happens for a single frame. It doesn't even touch computational photography[1].
[1] https://dpreview.com/articles/9828658229/computational-photo...
> The human eye is most sensitive to green light, so that channel effectively carries the majority of the luminance (brightness/detail) data
How does this affect luminance perception for deuteranopes? (Since their color blindness is caused by a deficiency of the cones that detect green wavelengths)
> The human eye is most sensitive to green light,
This argument is very confusing: if is most sensitive, less intensity/area should be necessary, not more.
Why that ratio in particular? I wonder if there’s a more complex ratio that could be better.
This is also why I absolute hate, hate, hate it when people ask me whether I "edited" a photo or whether a photo is "original", as if trying to explain away nice-looking images as if they are fake.
The JPEGs cameras produce are heavily processed, and they are emphatically NOT "original". Taking manual control of that process to produce an alternative JPEG with different curves, mappings, calibrations, is not a crime.
the bayer pattern is one of those things that makes me irrationally angry, in the true sense, based on my ignorance of the subject
what's so special about green? oh so just because our eyes are more sensitive to green we should dedicate double the area to green in camera sensors? i mean, probably yes. but still. (⩺_⩹)
hey, not accusing you of anything (bad assumptions don't lead to a conducive conversation) but did you use AI to write or assist with this comment?
this is totally out of my own self-interest, no problems with its content
> A fun tangent on the "green cast" mentioned in the post: the reason the Bayer pattern is RGGB (50% green) isn't just about color balance, but spatial resolution. The human eye is most sensitive to green light, so that channel effectively carries the majority of the luminance (brightness/detail) data.
From the classic file format "ppm" (portable pixel map) the ppm to pgm (portable grayscale map) man page:
https://linux.die.net/man/1/ppmtopgm
You'll note the relatively high value of green there, making up nearly 60% of the luminosity of the resulting grayscale image.I also love the quote in there...
(context for the original - https://www.youtube.com/watch?v=VNC54BKv3mc )