logoalt Hacker News

antireztoday at 9:09 AM2 repliesview on HN

Because of masked attention in LLMs, if you put the options before the body (the email to analyze), the transformer already knows what it needs to look for, and can use more tokens to create state to address that specific task (BERT has no mask in the attention, so tokens attend also to next tokens). You could also do a few examples in the system prompt to improve calibration.

Another trick that works is to repeat the question two times: "I'm repeating the task and labels for clarity: ..."


Replies

ThePhysicisttoday at 10:24 AM

What a time to be alive, repeating questions to a model twice to increase accuracy.

show 3 replies