You've misunderstood the bitter lesson if you think it applies here. It's not "frontier LLMs are always better", it's "algorithms that scale with compute/data will usually end up surpassing algorithms that are based on human understanding".

Fine-tuning an open-source LLM with reinforcement learning in order for it to be better at a specific task does not go against the bitter lesson at all.