syncing URL lists should be done in parallel. Something like xargs+flock but under Python.
Contributions are welcome. I even believe that this whole code should be migrated from python-requests to curl.
alt Hacker News
Contributions are welcome. I even believe that this whole code should be migrated from python-requests to curl.