alt Hacker NewsI've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.
Replies
I worked on a team that built a massive, high-performance internal service based on Riak. There are many things I learned from that system. Here is the best takeaway I can offer:
It does not matter what your technology is, or how theoretically superior it is. Getting it to actually work well "in production" is a whole separate thing than simply designing it and writing code. When it's a very small system, it will look like it's doing great. As it gets bigger, the seams will start to burst, and you will find out that promises and theory don't always match reality.
In the end, while its aims are great, it takes a whoooooole lot of work to smooth out the bumps in such a system. You need experts in that technology to address bugs in a timely manner. You need developers versed in the system to properly build apps utilizing it. You need competent operators to build, orchestrate, operate and maintain the whole thing.
All of that is made easier by using simple technology that everybody knows, that there's a huge support community for, professional services for, etc. A technology like MySQL or Postgres etc, has the corporate, development, support, etc to make it easy to work with at any scale. A little janky at times, limited, but dependable, predictable, controllable.
A small bespoke system with a small support community and virtually no corporate support is, comparatively, a hell of a lot more difficult/costly to support and harder to make work reliably.
My old team used Riak in production for time series data in a real-time system.
Our code was in Clojure, and we just wrapped the Java client. The conflict resolution was a steep learning curve, but overall, it was kind of nice (coming from Mongo).
But man, Clojure stack traces wrapping Java stack traces wrapping Erlang stack traces in a Kafka consumer... I wish that hell on no one.
> I've never met an engineering team that used Riak
I was part of a recent cloud migration. Part of on-prem (though unfortunately not migrated by my team) were this very first Riak Cluster I saw in production.
The engineering team used it as "kind of S3" for images, with 3 to 5 PHP scripts providing an interface to Riak and imageMagic. It seemed to me like a good abstraction and I think the migration to S3 was mostly painless.
Other than that I only had contact with Riak at university around 15 years ago, when we tested cluster setups of several NoSQL databases and tried to manually introduce faults to see if they could heal. Riak passed our test at that time, MongoDB didn't.
I'm pretty sure Stripe was a heavy user of this for a while. They used it due to their write-heavy system, if I recall.
I fondly remember writing a Go driver for it. Was a good experience: https://github.com/riaken/riaken-core
We used Riak at $dayjob at around 2014-2017 (iirc). I don't exactly remember it fondly. It was slow and unreliable. You could make it freeze/crash with the wrong SOLR query. (I was pretty good at that...)
Inscrutable erlang stack traces definitely played a part. They were horrible.
Companies would rather use something like dynamodb than self-host riak. You get an army of Amazon code monkeys to help you if something goes wrong, and it's a click away.
Speaking as a former tech evangelist/engineer at Basho, there were a few significant challenges.
Riak is horribly unfriendly as a database: no SQL, it exposes eventual consistency directly to the developer, it’s relatively slow, and Erlang is a fairly unusual language.
While you can run Riak on a single server, you’d have to really want to.
Its strength is the ability to scale massively, but not many projects need that scale, and by the time you do, you’re probably already using some friendlier database and you’d rather make that one work.