logoalt Hacker News

tombertyesterday at 6:02 PM2 repliesview on HN

I'm a little sad that I got laid off from a previous job for a variety of reasons, but one big one was that there were discussions of letting me open source some very big changes I had made to the Kafka Streams library.

I rewrote a lot of stuff while keeping the API mostly compatible, focusing on emphasizing non-blocking IO with backpressure semantics available if necessary. It was really cool and enabled a lot of interesting stuff involving the state store and mixing+matching blocking and non-blocking IO in a way that was still relatively performant. I think it was really neat and it's one of the projects I am most proud of because I was able to squeeze out performance in a lot of places that were non-obvious.

I was pushing to allow us to release it to Github and/or make a PR to the upstream Kafka Streams project, but sadly they did layoffs before that was completed and afterwards there was really no "champion" to do that, so it's stuck in proprietary land.

I might still do it from scratch and FOSS it, it's been long enough to where I think I wouldn't get in trouble if I rewrote it and released it (there weren't any patents or anything attached to it), and there are a few things I'd like to change anyway (like getting rid of the dependency of Vert.x). Maybe if I ever get a week off I'll do that.


Replies

sheepscreekyesterday at 6:18 PM

Fixing a bug in something open-source should be acceptable to most employers. However, if new functionality is being added, then it becomes an entirely different conversation.

I think it is good that you were taking the legal + compliance sign-offs before pursuing it.

show 1 reply
vibbixyesterday at 7:14 PM

This would be amazing to see upstreamed! I did some similar work for optimizing batch calls for the Quartz Scheduler. I left before it got merged, but was able to follow up and get it in as it was already PR'd.

show 1 reply