logoalt Hacker News

gschizasyesterday at 8:54 PM3 repliesview on HN

Why not sanitize and denormalize (or whatever it's called) the text before feeding it to the spam filter? Or the LLM prompt?


Replies

porneltoday at 1:08 AM

Throwing it as-is into a simple bayesian filter works great too - nobody honest writes fun&zwsp;ding.

gerdesjyesterday at 11:32 PM

MS and the other big buggers are pretty incapable of running an email system that you would really want to use. They also have to support those that you might consider spammers too.

However, I'm sure that you are also familiar with the HN standard issue meme that it is impossible to run your own email system.

So, your email provider is crap and you can't do it yourself!

Bollocks! I use Exim (1) and rspamd (2) and run quite a few bijou email systems. It does require some effort.

You pays your money and you makes your choice ...

1. https://www.exim.org/exim-html-current/doc/html/spec_html/in... 2. https://docs.rspamd.com/

show 1 reply
thephyberyesterday at 9:44 PM

Do you have any evidence they didn't?

My suspicion is that the spam filter programmers didn't do a comprehensive evaluation of every code point on every plane of Unicode because... well that's a massive job. So your "sanitize and denormalize" tasks are actually massive mappings which were likely imperfectly created.