I'm skeptical that RLHF really works. Doesn't it just patch the obvious holes so it looks better on paper? If it can't reason then it will continue to get 2nd and 3rd order difficulty problems wrong.