alt Hacker Newssemi-related: https://unsloth.ai/docs/get-started/reinforcement-learning-r...
i personally do finetuning in FP16 because then the LoRA merge at the end has less rounding error.
for most models it's lossless as well (when coming from a BF16 source), it's very quick to scan the weights and make sure they're all in range if you're concerned.
Yea that's pretty related. In this case, I'm looking at full finetunes (although we don't end up actually finetuning the whole model, just the small weights!)