alt Hacker NewsWhat an unprocessed photo looks like
Comments
But does applying the same transfer function to each pixel (of a given colour anyway) count as "processing"?
What bothers me as an old-school photographer is this. When you really pushed it with film (e.g. overprocess 400ISO B&W film to 1600 ISO and even then maybe underexpose at the enlargement step) you got nasty grain. But that was uniform "noise" all over the picture. Nowadays, noise reduction is impressive, but at the cost of sometimes changing the picture. For example, the IP cameras I have, sometimes when I come home on the bike, part of the wheel is missing, having been deleted by the algorithm as it struggled with the "grainy" asphalt driveway underneath.
Smartphone and dedicated digital still cameras aren't as drastic, but when zoomed in, or in low light, faces have a "painted" kind of look. I'd prefer honest noise, or better yet an adjustable denoising algorithm from "none" (grainy but honest) to what is now the default.
You may know that intermittent rashes are always invisible in the presence of medical credentials.
Years ago I became suspicious of my Samsung Android device when I couldn't produce a reliable likeness of an allergy induced rash. No matter how I lit things, the photos were always "nicer" than what my eyes recorded live.
The incentives here are clear enough - people will prefer a phone whose camera gives them an impression of better skin, especially when the applied differences are extremely subtle and don't scream airbrush. If brand-x were the only one to allow "real skin" into the gallery viewer, people and photos would soon be decried as showing 'x-skin', which would be considered gross. Heaven help you if you ever managed to get close to a mirror or another human.
To this day I do not know whether it was my imagination or whether some inline processing effectively does or did perform micro airbrushing on things like this.
Whatever did or does happen, the incentive is evergreen - media capture must flatter the expectations of its authors, without getting caught in its sycophancy. All the while, capacity improves steadily.
I studied remote sensing in undergrad and it really helped me grok sensors and signal processing. My favourite mental model revelation to come from it was that what I see isn’t the “ground truth.” It’s a view of a subset of the data. My eyes, my cat’s eyes, my cameras all collect and render different subsets of the data, providing different views of the subject matter.
It gets even wilder when perceiving space and time as additional signal dimensions.
I imagine a sort of absolute reality that is the universe. And we’re all just sensor systems observing tiny bits of it in different and often overlapping ways.
I think everyone agrees that dynamic range compression and de-Bayering (for sensors which are colour-filtered) are necessary for digital photography, but at the other end of the spectrum is "use AI to recognise objects and hallucinate what they 'should' look like" --- and despite how everyone would probably say that isn't a real photo anymore, it seems manufacturers are pushing strongly in that direction, raising issues with things like admissibility of evidence.
Highly recommend this CaptainDisillusion video that covers this topic of how cameras process colors in a very entertaining way
An unprocessed photo does not “look”. It is RGGB pixel values that far exceed any display media in dynamic range. Fitting it into the tiny dynamic range of screens by thrusting throwing away data strategically (inventing perceptual the neutral grey point, etc.) is what actually makes sense of them, and what is the creative task.
This is a great article but I was surprised how anemic the tree was :)
Really good article though
This reminds me of a couple things:
== Tim's Vermeer ==
Specifically Tim's quote "There's also this modern idea that art and technology must never meet - you know, you go to school for technology or you go to school for art, but never for both... And in the Golden Age, they were one and the same person."
https://en.wikipedia.org/wiki/Tim%27s_Vermeer
https://www.imdb.com/title/tt3089388/quotes/?item=qt2312040
== John Lind's The Science of Photography ==
Best explanation I ever read on the science of photography https://johnlind.tripod.com/science/scienceframe.html
== Bob Atkins ==
Bob used to have some incredible articles on the science of photography that were linked from photo.net back when Philip Greenspun owned and operated it. A detailed explanation of digital sensor fundamentals (e.g. why bigger wells are inherently better) particularly sticks in my mind. They're still online (bookmarked now!)
https://www.bobatkins.com/photography/digital/size_matters.h...
I was lucky enough to take some introductory courses at the NTNU Colorlab in Gjøvik, Norway. What I learned there changed my view on vision.
Computer imaging is much wider than you think. It cares about the entire signal pipeline, from emission from a light source, to capture by a sensor, to re-emission from a display, to absorption in your eye, and how your brain perceives it. Just like our programming languages professor called us "Pythonized minds" for only knowing a tiny subset of programming, there is so much more to vision than the RGB we learn at school. Look up "Metamerism" for some entry-level fun. Color spaces are also fun and funky.
There are a lot of interesting papers in the field, and its definitely worth reading some.
A highlight of my time at university.
>if the linear data is displayed directly, it will appear much darker then it should be.
This seems more a limitation of monitors. If you had very large bit depth, couldn't you just display images in linear light without gamma correction.
Author should've mentioned how the first image "as my camera’s sensor sees it" was obtained.
Very interesting, pity the author chose such a poor example for the explanation (low, artificial and multicoloured light), making it really hard to understand what the "ground truth" and expected result should be.
>Our perception of brightness is non-linear.
Apart from brightness, it's everything. Loudness, temperature, etc.
Really enjoyed the article, thanks! A small nit - I think you have a small mistake in the value range at the start - 136000 should probably be 13600?
Maybe it’s just me but I took one look at the unprocessed photo (the first one) and immediately knew it was a skinny Christmas tree.
I’ve been staring at 16-bit HDR greyscale space for so long…
I work with camera sensors and I think this is a good way to train some of the new guys, with some added segments about the sensor itself and readout. It starts with raw data, something any engineer can understand, and the connection to the familiar output makes for good training.
Your Christmas tree has anorexia?
> As a result of this, if the linear data is displayed directly, it will appear much darker then it should be.
Then -> than? (In case the author is reading comments here.)
> There’s nothing that happens when you adjust the contrast or white balance in editing software that the camera hasn’t done under the hood. The edited image isn’t “faker” then the original: they are different renditions of the same data.
Almost, but not quite? The camera works with more data than what's present in the JPG your image editing software sees.
This is actually really useful. A lot of people demand an "unprocessed" photo but don't understand what they're actually asking for.
I have always wondered at the lowest level how a camera captures and processes photos. Much appreciated post.
I fill the original photo to Nano banana Pro, and it recovered well. It also explained how to recover it.
This is a great write up. It's also weirdly similar to a video I happened upon yesterday playing around with raw Hubble imagery: https://www.youtube.com/watch?v=1gBXSQCWdSI
He take a few minutes to get to the punch line. Feel free to skip ahead to around 5:30.
That's a cool walkthrough.
I spent a good part of my career, working in image processing.
That first image is pretty much exactly what a raw Bayer format looks like, without any color information. I find it gets even more interesting, if we add the RGB colors, and use non-square pixels.
Interesting to see this whole thing shown outside of Astrophotography, sometimes I forget it's the same stuff!
I am confused by the color filter step.
Is the output produced by the sensor RGB or a single value per pixel?
I love the look of the final product after the manual work (not the one for comparison). Just something very realistic and wholesome about it, not pumped to 10 via AI or Instagram filters.
For those who are curious, this is basically what we do when we color grade in video production but taken to its most extreme. Or rather, stripped down to the most fundamental level. Lots of ways to describe it.
Generally we shoot “flat” (there are so many caveats to this but I don’t feel like getting bogged down in all of it. If you plan on getting down and dirty with colors and really grading, you generally shoot flat). The image that we handover to DIT/editing can be borderline grayscale in its appearance. The colors are so muted, the dynamic range is so wide, that you basically have a highly muted image. The reason for this is you then have the freedom to “push” the color and look and almost any direction, versus if you have a very saturated, high contrast image, you are more “locked” into that look. This matters more and more when you are using a compressed codec and not something with an incredibly high bitrate or raw codecs, which is a whole other world and I am also doing a bit of a disservice to by oversimplifying.
Though this being HN it is incredibly likely I am telling few to no people anything new here lol
2 top HN posts in 1 day, maurycyz is on fire!
Honestly, I think the gamma normalization step don't really count as "processing", any more than the gzip decompression step doesn't count as "processing" for the purposes of "this is what an unprocessed html file looks like" demo. At the end of the day, it's the same information, but encoded differently. Similar arguments can be made for de-bayer filter step. If you ignore these two steps, the "processing" that happens looks far less dramatic.
Very interesting! Thank you for posting
The matrix step has 90s video game pixel art vibes.
Wow this is amazing. What a good and simple explanation!
OK now do Fuji Super CCD (where for reasons unknown the RAW is diagonal [0])
[0] - https://en.wikipedia.org/wiki/Super_CCD#/media/File:Fuji_CCD...
>No adblocker detected. Consider using an extension like uBlock Origin to save time and bandwidth. Click here to close.
So cute (I am running a DNS adblock only, on the work browser)
The article keeps using the acronym "ADC" without defining it.
I appreciate the authors honest with their astrophotography.
I've been studying machine learning during the xmas break, and as an exercise I started tinkering around with the raw Bayer data from my Nikon camera, throwing it at various architectures to see what I can squeeze out of the sensor.
Something that surprised me is that very little of the computation photography magic that has been developed for mobile phones has been applied to larger DSLRs. Perhaps it's because it's not as desperately needed, or because prior to the current AI madness nobody had sufficient GPU power lying around for such a purpose.
For example, it's a relatively straightforward exercise to feed in "dark" and "flat" frames as extra per-pixel embeddings, which lets the model learn about the specifics of each individual sensor and its associated amplifier. In principle, this could allow not only better denoising, but also stretch the dynamic range a tiny bit by leveraging the less sensitive photosites in highlights and the more senstive ones in the dark areas.
Similarly, few if any photo editing products do simultaneous debayering and denoising, most do the latter as a step in normal RGB space.
Not to mention multi-frame stacking that compensates for camera motion, etc...
The whole area is "untapped" for full-frame cameras, someone just needs to throw a few server grade GPUs at the problem for a while!
[dead]
[dead]
This is really interesting. I'll be back after reading it.
Very cool and detailed
I love posts that peel back the abstraction layer of "images." It really highlights that modern photography is just signal processing with better marketing.
A fun tangent on the "green cast" mentioned in the post: the reason the Bayer pattern is RGGB (50% green) isn't just about color balance, but spatial resolution. The human eye is most sensitive to green light, so that channel effectively carries the majority of the luminance (brightness/detail) data. In many advanced demosaicing algorithms, the pipeline actually reconstructs the green channel first to get a high-resolution luminance map, and then interpolates the red/blue signals—which act more like "color difference" layers—on top of it. We can get away with this because the human visual system is much more forgiving of low-resolution color data than it is of low-resolution brightness data. It’s the same psycho-visual principle that justifies 4:2:0 chroma subsampling in video compression.
Also, for anyone interested in how deep the rabbit hole goes, looking at the source code for dcraw (or libraw) is a rite of passage. It’s impressive how many edge cases exist just to interpret the "raw" voltages from different sensor manufacturers.