logoalt Hacker News

Show HN: I built a fast RSS reader in Zig

89 pointsby superstarryeyeslast Tuesday at 7:55 PM31 commentsview on HN

Well, I certainly tried. I had to, because it has a certain quirk inspired by "digital minimalism."

The quirk is that it only allows you to fetch new articles once per day (or X days).

Why? Let me explain...

I want my internet content to be like a boring newspaper. You get it in the morning, and you read the whole thing while sipping your morning coffee, and then you're done! No more new information for today. No pings, no alerts, peace, quiet, zen, etc.

But with that, I needed it to be able to fetch all articles from my hundreds of feeds in one sitting. This is where Zig and curl optimisations come in. I tried to do all the tricks in the book. If I missed something, let me know!

First off, I'm using curl multi for the network layer. The cool thing is it automatically does HTTP/2 multiplexing, which means if your feeds are hosted on the same CDN it reuses the same connection. I've got it configured to handle 50 connections total with up to 6 per host, which seems to be the sweet spot before servers start getting suspicious. Also, conditional GETs. If a feed hasn't changed since last time, the server just says "Not Modified" and we bail immediately.

While curl is downloading feeds, I wouldn't want CPU just being idle so the moment curl finishes downloading a single feed, it fires a callback that immediately throws the XML into a worker thread pool for parsing. The main thread keeps managing all the network stuff while worker threads are chewing through XML in parallel. Zig's memory model is perfect for this. Each feed gets its own ArenaAllocator, which is basically a playground where you can allocate strings during parsing, then when we're done, we just nuke the entire arena in one go.

For parsing itself, I'm using libexpat because it doesn't load the entire XML into memory like a DOM parser would. This matters because some podcast feeds especially are like 10MB+ of XML. So with smart truncation we download the first few X mb's (configurable), scan backwards to find the last complete item tag, cut it there, and parse just that. Keeps memory usage sane even when feed sizes get massive.

And for the UI I just pipe everything to the system's "less" command. You get vim navigation, searching, and paging for free. Plus I'm using OSC 8 hyperlinks, so you can actually click links to open them on your browser. Zero TUI framework needed. I've also included OPML import/export and feed groups as additional features.

The result: content from hundreds of RSS feeds retrieved in matter of seconds, and peace of mind for the rest of the day.

The code is open source and MIT licensed. If you have ideas on how to make it even faster or better, comment below. Feature requests and other suggestions are also welcome, here or GitHub.


Comments

renegat0x0today at 10:51 AM

It is a fine project to limit 'doomscrolling', but I think the premise is wrong.

- I have created my own RSS readers, that contains 500+ sources. I do not doom scroll

- doom scrolling appears when social media algorithm feeds you data, even from a month ago

- I have various filters so I can browse whatever I want

So RSS should just have filters, categories, search extensive capabilities to solve doom scrolling, and on the other hand it will be able to provide you extensive amounts of data.

A1aM0today at 3:04 AM

The 'once a day' fetching limitation is a fascinating idea. It really captures the vibe of reading a physical newspaper in the morning rather than constantly checking for updates. I think many of us could use a tool that enforces a bit of 'digital silence' like this.

show 4 replies
lknuthyesterday at 7:52 AM

I think its cool that more people are building what I call "calm tech". More technology should try to serve a purpose quickly and then get out of the way instead of trying to artificially stay on your screen as long as possible.

Incidentally, I built my own calm RSS reader some time ago that has many similar ideas to yours: https://github.com/lukasknuth/briefly

show 2 replies
cefboudtoday at 11:21 AM

The human psychology is intriguing. The more I see "... Zig ..." the stronger the urge to learn and build something with it becomes. In practice, though, I've found that building things with languages I am familiar with is more enjoyable for me, personally.

A severe case of mimetic desire. I suspect a lot of devs suffer from it.

show 2 replies
vqtskatoday at 11:40 AM

This looks kinda scary to me considering that Zig isn't a safe language and it's being used here to parse untrusted data from the internet. Would the ReleaseSafe mode that Zig provides prevent any attempts of exploiting memory safety bugs?

show 2 replies
Zambytetoday at 1:39 PM

Very cool :)

I really like the culture around Zig right now. Ship simple, fast, correct, useful software feels like the general set of goals for people writing software in Zig.

renegat0x0today at 10:40 AM

If anyone is interested in RSS feeds, here are mine in SQLite table:

https://github.com/rumca-js/Internet-feeds

keyletoday at 6:33 AM

Nice work, I did mine in C, using Termbox2, in a very suckless fashion (config.h & make install)

I like the idea of the daily digest.

That gave me a good chuckle:

    Starts in milliseconds and parses hundreds of items in seconds.
Consider having a shortcut to load a feed item's comments in the browser, if that's not already there.
nhanbtoday at 11:44 AM

>And for the UI I just pipe everything to the system's "less" command.

I never thought of that as an option. Thanks for the tip haha.

show 2 replies
ekjhgkejhgkyesterday at 1:24 AM

Why MIT and not GPL3?

show 1 reply