A fine-tuned classifier purpose fit for a specific task can easily outperform a SOTA LLM on more modest hardware and often makes a lot more sense.
If your data is sufficiently noisy or your relationship sufficiently simple a linear regression will outperform a SOTA LLM.
If your data is sufficiently noisy or your relationship sufficiently simple a linear regression will outperform a SOTA LLM.