alt Hacker NewsIt might be an interesting LLM benchmark: how many can they list without breaking the rules (repetition or non-animals). Although I bet that big bucks would be then thrown at pointlessly optimizing for that benchmark, so...
Replies
OxfordOutlander • today at 5:16 AM
you might like https://github.com/aidanmclaughlin/AidanBench
Might be an interesting problem for understanding how various models perform recollection of prior tokens within the context window. I'm sure they could list animals until their window is full but what I'm not sure of is how much of the window they could fill without repeating.