logoalt Hacker News

CraigJPerrytoday at 9:52 AM2 repliesview on HN

There are so many footguns[1] in async python but it really has stolen the zeitgeist of modern python web. Synchronous django has a lot to commend it. If you deploy it reasonably well (workers, behind a caching reverse proxy etc) it's easy to operate in production even under duress.

It's not going to be the right stack for long lived websocket connections or whatever but for a CRUD-ish or enterprise app, often very productive.

[1] buffer bloat is too easy to sleep walk into

    queue = asyncio.Queue()  # oops
unbounded concurrency

    await asyncio.gather(*(fetch(item) for item in items))  # look mum! no outbound sockets left or maybe even no file descriptors
accidental blocking

    data = requests.get(url).json()  # oops! we're blocking the event loop
and sure you can setup a separate task to measure the event loop latency and alert on that but this is the kinda thing you'll never find in a tutorial, you just need to experience this stuff and figure out a solution you like

I can go on and on about async python (cancellation bugs, leaking a task - that has a cataclysmic failure mode where if you forget to hold a reference to your asyncio.create_task() then it's all weak refs in that machinery so your task can get garbage collected before it ran or completed in production! super tricky forensics. Then there's all the obvious stuff like race conditions, new ways of creating deadlocks, blah blah i really can go on for days.


Replies

loevborgtoday at 8:28 PM

Please go on. I also feel like asyncio is a big hack. Even just accidentally blocking the event loop is way too easy. And I couldn't believe it when I read that you need to store a reference to a task in a set to keep it from accidentally getting canceled. How did this become the main stack of AI backends? It's like node but slower AND much easier to mess up because of synchronous IO.

show 1 reply
codethieftoday at 8:57 PM

I would love a blog post on this, in case you ever considered writing one!