Good advice for all new LLM experimenters.

The bugs come from the downstream implementations and quantizations (which inherit bugs in the tools).

Expect to update your tools and redownload the quants multiple times over 2-4 weeks. There is a mad rush to be first to release quants and first to submit PRs to the popular tools, but the output is often not tested much before uploading.

If you experiment with these on launch week, you are the tester. :)