You can distill closed weights models as well. (Just not logit-distillation)
Though it violates their terms of service
Though it violates their terms of service