alt Hacker NewsUPD.
I've just made a small but important clarification to the article. While in many cases it's easier and even preferred to calculate all results, accumulate them somewhere, then sort; this article focuses on memory bound algorithms that support infinite streams and backpressure.
Thanks, but I'd still use a queue over this solution.
Real-time Log Enrichment: perfect for my example [0], you're firing off endless tasks. RT logs have a timestamp.
Finding the First Match in a File List: Files tend to be static. I'd use a queue to first build an index and then a queue to process the index.
Time Series Data Processing: Break the data into chunks, you mention 600MB, which isn't that big at all given that Cloud Run memory maxes out at 32GB.
[0] https://news.ycombinator.com/item?id=45094387