Correct me if I wrong, I saw this when llama.cpp or ollama initializes, it is used as token separator of some sort.
GPT-2 used <|endoftext|>
GPT-2 used <|endoftext|>