It’s a problem specific to autoregressive LLMs, the early tokens bias the output