alt Hacker News> In practice they're rare enough that the per-page toggle handles them, but it's the honest limitation of the approach.
I don't understand how you handle raster images. You simply cannot invert them blindly. So it sounds like you just bite the bullet of never inverting raster images, and accepting that you false-positive some vector-based diagrams? I don't see how that can justify your conclusion "it wasn't necessary". It sounds necessary to me.
Replies
The raster image problem is real but there's a middle ground between "never invert" and a full NN classifier.
The author already computes BT.601 brightness per page. You can run the same calculation per-image bounding box instead of per-page, then add a bimodal pixel distribution check: if a raster image has most pixels near black or near white with few midtones, it's probably a line diagram or screenshot, not a photograph. That heuristic catches the main false-positive case (black-line diagrams on white backgrounds) with maybe 20 lines of image processing code.
It won't be perfect, and gwern's point stands that a proper trained classifier would be more accurate. But for a PDF viewer where you're already parsing content streams to get image coordinates, it's a lot cheaper than shipping a model and handles 80% of the problematic cases. The remaining edge cases (medical scans, thermal images) are rare enough that the per-page toggle is reasonable fallback.
Actually, raster images are never inverted, they're protected. The CSS filter: invert() hits the entire canvas (text and images together), then the overlay paints the original image pixels back on top, restoring them. The result is: inverted text, images with their original colors.
The choice to never invert raster images isn't a compromise, it's the design decision. The problem veil solves is exactly that: every dark mode reader today inverts everything, and the result on photos, histology, color charts, scans is unusable. Preserving all images is the conservative choice, and for my target (people reading scientific papers, medical reports, technical manuals) it's the right one.
It's absolutely true that there's a subset of raster images, like diagrams with white backgrounds and black lines, that would benefit from inversion. I could be wrong, but in my experience they're a minority, and the cost of accidentally inverting the wrong one (a medical photo, a color chart) is much higher than the benefit of inverting a black and white diagram, from my point of view. For now the per-page toggle covers those cases.