alt Hacker NewsThe things they don't prepare you for in school...
I was working at my first job and we had a ColdFusion app that was displaying some data from the database. I get a ticket one day saying our search page would crash when searching for a very specific document. The other 1 million+ documents all loaded fine to our knowledge, so why this one?
I was pretty junior back then and feeling mighty defeated as to why I couldn't figure it out. I debugged every single line and condition, trying to find some reason. After ruling out the code as a culprit, I took the data we were loading and placed it into Notepad++. Don't remember why exactly. I was wracking my brain trying to come up with explanation and lazily moving the text cursor left and right through the text, mostly out of boredom and despair.
That's when I noticed that I had pressed the right arrow key in my keyboard and the text cursor position hadn't changed! I pressed it again and nothing. Again, nothin. It took eight key presses to move the text cursor from one letter in a word to the adjacent letter. I was utterly bamboozled. Why was the text cursor getting stuck in the middle of this word?!
Shortly thereafter, I discovered "Show all hidden characters" setting in the menu. I toggled it and sure enough there were little black boxes with weird three letter strings in them. NUL, ESC, and others - right where my cursor was getting hung up.
That was the day I learned about ANSI control characters and the importance of data sanitization.
Replies
CudaText is better than Notepad++ in this regard. It shows ASCII control chars always. The option for "unprinted chars" only hides "arrows" over spaces/tabs.
There are also zero-space width characters, yeah utf is a rabbit hole
Similarly, I once copied a shell script out of Slack and saw a bunch of red errors from my IDE when I pasted it in. The errors were on every line that had a new line on it. The error was "â: Command not found", despite there being no such character on the line.
Pasted it into a hex editor, tracked down the bytes, and while I can't currently remember specifically what the encoding problem was, it was something to do with going between UTF-8 > ISO-8859 > UTF-8 again.
I've since aliased `pbpaste | xxd` (macOS, linux has similar CLI tools for working with the clipboard depending on your distro/DE), because weird shit like this comes up more often than I'd care to admit. Last rabbit hole was discovering that in macOS 15, Apple changed one of the the "space" characters in the default Screenshot file names, but only if your Mac is set to use 12-hour time from a normal ASCII 0x20 space, to a Unicode 'U+202F NNBSP' "non-breaking space" between the time and AM/PM, which was causing S3 uploads to fail.