> Your credit score can decrease if you pay off your credit card balance or pay it down too quickly

It can have a short term impact, sure, but long term (a few years), your score will be fine. I paid my last student loans years ago and I did see a sudden score drop and not I'm sitting right below 800 with no debt.