logoalt Hacker News

scottcha • today at 4:30 PM • 0 replies • view on HN

Yes it looks that way. I pulled the energy data for GLM 5.3 as well just to check the core findings are roughly the same and they do seem to be so spec decode doesn't seem to have the same divergence as the prefill/decode energy (which is rougly a function of thinking/decode lenghts and less a property of individual tokens).

  Language      Strict  Lenient   Cap   Tok/req     J/req    J/tok   Req x EN  Tok x EN  J/tok x EN
   English        100%     100%     0      548.7    230.33   0.4198       1.00      1.00         1.00
   Chinese         96%      96%     1      754.0    694.47   0.9210       3.02      1.37         2.19
   Russian         96%      98%     2      738.8    592.15   0.8015       2.57      1.35         1.91
   French         100%     100%     0      631.0   1101.84   1.7462       4.78      1.15         4.16
   S. Pashto       88%      98%     6     1332.1    208.06   0.1562       0.90      2.43         0.37
   Tigrinya        56%      62%    19     3642.9   1445.84   0.3969       6.28      6.64         0.95
   Shan            44%      54%    27     3674.8   3791.59   1.0318      16.46      6.70         2.46
   Tibetan          2%      70%    15     2856.9   5306.68   1.8575      23.04      5.21         4.42