alt Hacker NewsHow do you properly mix IO/compute (in any language)? In Rust what I’ve done in the past is have two Tokio runtimes, one for IO and one for compute. I know you can also use Rayon but the abstractions are not always flexible/convenient enough.
But in either case the boundary between the two types of async work is never easy to cross.
Replies
In Python Trio, you would have a thread pool for each type of computation (potentially a "pool" of 1 thread shared between all tasks if that works for your application). You would await trio.to_thread.run_sync(...) [1] (pass it a normal non-async function, and a token representing the thread pool). This is a pretty simple wrapper that takes care of queuing the work to the thread pool, waiting for it to complete and for a message to be passed back to the Trio thread.
It works for simple situations. It certainly preserves backpressure, although a slow CPU task can impact other users of the same thread pool.
[1] https://trio.readthedocs.io/en/stable/reference-core.html#tr...
You can use tokio::spawn_blocking to separate compute-heavy stuff, it gets handled in a separate thread pool. You shouldn’t need two tokio runtimes, that sounds problematic.
> How do you properly mix IO/compute (in any language)?
Good question, it’s a really hard problem.
In principle you are meant to use threads, the OS will automatically switch to other work while you are waiting for IO, like with async. But there are difficulties with threads too: memory overhead, switching overhead, thread limits… Although I am getting the feeling that they have a much worse reputation than they deserve. More projects should try switching to threads and actually measuring the difference.