alt Hacker NewsInteresting. New models are estimated at ~5T params, so 45,000x increase over BERT base (110m). But vocab size of 200k, so only an increase of 7x over BERT base (30k).
Replies
vlian2088 • yesterday at 11:39 PM
BERT is not an LLM, it's an encoder-only model, i.e. it doesn't generate new text. the first somewhat useful publicly accessible LLM was GPT-3 with 175B params, and it was also the last frontier model whose parameter count was disclosed.
Interesting that Grok 4.5 nears or exceeds OpenAI and Anthropic models in some benchmarks at only 1.5 trillion parameters per their announcement post.