I've not tried running it but while it defaults to NVIDIA hosted endpoints but the readme suggests you can host those yourself if you want.