That's why the major AI labs are really careful about the code they include in the training runs.

The days of indiscriminately scraping every scrap of code on the internet and pumping it all in are long gone, from what I can tell.