Not 100% so for chain of thought models, they should recognize to spell the word letter by letter in some separated form and then count the tokens in that form. The Qwen distill seems to do exactly this really well:
> Step-by-step explanation:
> 1. Break down each word: "not", "really", "a", "tokenizer", "issue".
> 2. Count 'e's in each word:
> - "not": 0
> - "really": 1
> - "a": 0
> - "tokenizer": 2
> - "issue": 1
> 3. Sum the counts: 0 + 1 + 0 + 2 + 1 = 4.
>
> Answer: There are 4 E's in the phrase.
In the thought portion it broke the words up every which way you could think to check then validated the total by listing the letters in a number list by index and counting that compared to the sums of when it did each word.
alt Hacker News
Not 100% so for chain of thought models, they should recognize to spell the word letter by letter in some separated form and then count the tokens in that form. The Qwen distill seems to do exactly this really well:
> Step-by-step explanation:
> 1. Break down each word: "not", "really", "a", "tokenizer", "issue".
> 2. Count 'e's in each word:
> - "not": 0
> - "really": 1
> - "a": 0
> - "tokenizer": 2
> - "issue": 1
> 3. Sum the counts: 0 + 1 + 0 + 2 + 1 = 4.
>
> Answer: There are 4 E's in the phrase.
In the thought portion it broke the words up every which way you could think to check then validated the total by listing the letters in a number list by index and counting that compared to the sums of when it did each word.