I have 5Gbps fiber at home. LAN is mostly 10Gbps.

I'm hosting few hobbyist public websites. One is the library of hymns, songs, ... - mostly PDF, MP3, MIDI.

OpnSense router -> Traefik -> Docker container.

That Docker container runs on NAS, which is connected via 1Gbps port.

Suddenly, last week I started to get over 1000 RPS. Was it scanner, parser, crawler, DDOS? I can't tell, as user agent was default Mac OS.

It was using ~1Gpbs bandwidth. From many IPs. If I would have it hosted at AWS / Azure, it would cost me > $40 / hour for the bandwidth. Unless I use some of their CDN.

My router CPU went high, but it handled it ok. NAS CPU went high, but it handled it ok. Traefik loaded the server heavily: it had to both receive and send 1Gbps. And write access logs for all requests. My Alloy & Loki logging system got loaded heavily.

NAS already have cache. I tried to enable cache on Traefik. It went OOM in 2 minutes: https://github.com/darkweak/souin/issues/844

When I searched, I found that Traefik cache with proper memory handling is available only in Enterprise version.

Solved the issue by putting the site behind Cloudflare proxy and enabling it's magic to protect from the bot.