A lot depends on what the models were specifically trained for using reinforcement learning. Modern models are generally OK and everything, but they're only really good at the things they have extensive RL for.