logoalt Hacker News

dannywtoday at 2:57 PM0 repliesview on HN

ARC Prize has retested Luna after the discount and validated identical performance.

(Also, quantization isn't inherently bad or damaging when done properly, e.g. QAT).

These APIs are used heavily by enterprises at scale; with lots of performance telemetry, live evals, etc. You can't really silently nerf API models at scale without people noticing.

Of course, what I said doesn't apply to non-API consumer sub models; there's many documented and officially confirmed instances of under-the-hood "juice/effort" adjustments. (Juice = a number your effort tier maps to underneath the hood; much like Inkling's effort=0.00 to 0.99).