Just quantizing takes seconds-to-minutes, llama.cpp provides a nice tool[0]. Improving quality is then a matter of picking specific tensors to maintain at higher accuracy, checking on representative data, and repeating.

[0]: https://github.com/ggml-org/llama.cpp/blob/master/tools/quan...