Trinity large: An open 400B sparse MoE model

They trained it in 33 days for ~20m (that includes apparently not only the infrastructure but also the salaries over a 6 month period). And the model is coming close to QWEN and Deepseek. Pretty impressive

I'm particularly excited to see a "true base" model to do research off of (https://huggingface.co/arcee-ai/Trinity-Large-TrueBase).

Given that it's a 400B-parameter model, but it's a sparse MoE model with 13B active parameters per token, would it run well on an NVIDIA DGX Spark with 128 GB of unified RAM, or do you practically need to hold the full model in RAM even with sparse MoE?

What did they do to make the loss drop so much in phase 3?

Also, why are they comparing with Llama 4 Maverick? Wasn’t it a flop?

So refreshing to see open source models like this come from the US. I would love for a 100Bish size one that can compete against OSS-120B and GLM air 4.5

What exactly does "open" mean in this case? Is it weights and data or just weights?

The only thing I question is the use of Maverick in their comparison charts. That's like comparing a pile of rocks to an LLM.

Is anyone excited to do ablative testing on it?

This is a wonderful release.