logoalt Hacker News

_ache_ • today at 5:00 PM • 1 reply • view on HN

Looks pretty legit to me at first. May be hard to run, I don't have 8TB of RAM.

The log line is also interesting.

> [2026-10-08T13:51:36Z step 1830001] loss=1.0550 lr=7.19012e-05 lr_phase=decay grad_norm=1.304 tokens/s=12.03M mfu=35.1% moe_balance_loss=0.0217 expert_gini=0.0231

Last line of log:

> [2026-10-08T14:37:21Z step 1830400] loss=1.0421 lr=7.16365e-05 lr_phase=decay grad_norm=1.266 tokens/s=11.25M mfu=32.9% moe_balance_loss=0.0218 expert_gini=0.0235

I'm still doubting it as legit.


Replies

yorwba • today at 5:04 PM

There is a ≈7 second gap between steps, suggesting the run had already been going for almost 5 months at that point...

➕ show 1 reply