logoalt Hacker News

yorwbayesterday at 6:44 PM1 replyview on HN

Without anchoring on the group average it's hardly Group-Relative and without likelihood there's no Policy; that doesn't leave much of the original GRPO unchanged...

Do you have a particular paper in mind that does likelihood-free best-of-K but just calls it GRPO?


Replies

zimablue22yesterday at 6:57 PM

Any of the Best of N papers that exploded in popularity after GRPO.

Likelihood is not fundamental to the spirit of GRPO, any exploratory mechanism would work.

That sequential LLMs have a step-wise probability is convenient but not critical to this approach (where rejection sampling is widely used in diffusion models).

show 1 reply