Nvidia Nemotron 3.5 Lightning and NeMo Switchyard

One major consequence of the ramapocalypse, I think, is an even higher focus on small efficient models. I personally believe that the multi-trillion parameter models are fundamentally missing things and the push to smaller, more efficient will drive evolutionary structural changes that will lead to future gains

> NeMo Switchyard, an open source library for smart routing

> When deployed, NeMo Switchyard can intelligently direct each request to the most capable and suitable model for the job

How do routers like this handle prompt caching when you send the second request?

Sticky models per session? but then the second message of that session won't be sent to a suitable model, and will only be sent to the same model as previous one.

They conveniently decided not to include the Qwen range of models in the Artificial Analysis graph, except the out-of-league Max variant. At least be brave and honest.

I would like to propose something:

- problem: massive deluge of information because of AI

- solution: human beings should adopt a minimalist style of communicating in writing.

- e.g. this entire website page can be ten bullet points.

24 comments so far about Nemotron on this earlier submission: https://news.ycombinator.com/item?id=49257947

[dead]

The new Meta 30B models seems A LOT better:

https://aibenchy.com/compare/meta-muse-glimmer-30b-xhigh/nvi...