alt Hacker NewsLeague's Wikipedia page describes it as "inspired by Defense of the Ancients, a custom map for Warcraft III." I believe they also hired some of the core Dota devs to work on League. I guess if you want to be pedantic it was a custom map, but that was more a consequence of WC3 lacking support for mods. They ended up having to work around a lot of limitations to make Dota work in the custom map framework.
They hired a former dev, guinsoo. At the time when LoL was announced dota was being developed by icefrog for a few good solid years already. Most of dota's popularity happened during icefrog's years. Icefrog later joined valve and helped create Dora 2.