logoalt Hacker News

Show HN: Sourcebot, an open-source Sourcegraph alternative

259 pointsby bshzzle10/01/202459 commentsview on HN

Hi HN,

We’re Brendan and Michael, the creators of Sourcebot (https://github.com/sourcebot-dev/sourcebot). Sourcebot is an open-source code search tool that allows you to quickly search across many large codebases. Check out our demo video here: https://youtu.be/mrIFYSB_1F4, or try it for yourself on our demo site here: https://demo.sourcebot.dev

While at prior roles, we’ve both felt the pain of searching across hundreds of multi-million line codebases. Using local tools like grep were ill-suited since you often only had a handful of codebases checked out at a time. Sourcegraph (https://sourcegraph.com/) solves this issue by indexing a collection of codebases in the background and exposing a web-based search interface. It is the de-facto search solution for medium to large orgs, but is often cited as expensive ($49 per user / month) and recently went closed source (https://news.ycombinator.com/item?id=41296481). That’s why we built Sourcebot.

We designed Sourcebot to be:

- Easily deployed: we provide a single, self-contained Docker image (https://github.com/sourcebot-dev/sourcebot/pkgs/container/so...).

- Fast & scalable: designed to minimize search times (current average is ~73ms) across many large repositories.

- Cross code-host support: we currently support syncing public & private repositories in GitHub and GitLab.

- Quality UI: we like to think that a good looking dev-tool is more pleasant to use.

- Open source: Sourcebot is free to use by anyone.

Under the hood, we use Zoekt (https://github.com/sourcegraph/zoekt) as our code search engine, which was originally authored by Han-Wen Nienhuys and now maintained by Sourcegraph (https://sourcegraph.com/blog/sourcegraph-accepting-zoekt-mai...). Zoekt works by building a trigram index from the source code enabling extremely fast regular expression matching. Russ Cox has a great article on how trigram indexes work if you’re interested: https://swtch.com/~rsc/regexp/regexp4.html

In the shorter-term, there are several improvements we want to make, like:

- Improving how we communicate indexing progress (this is currently non-existent so it’s not obvious how long things will take)

- UX improvements like search history, query syntax highlighting & suggestions, etc.

- Small QOL improvements like bookmarking code snippets.

- Support for more code hosts (e.g., BitBucket, SourceForge, ADO, etc.)

In the longer-term, we want to investigate how we could go beyond just traditional code search by leveraging machine learning to enable experiences like semantic code search (“where is system X located?”) and code explanations (”how does system X interact with system Y?”). You could think of this as a copilot being embedded into Sourcebot. Our hunch is that will be useful to devs, especially when packaged with the traditional code search, but let us know what you think.

Give it a try: https://github.com/sourcebot-dev/sourcebot. Cheers!


Comments

peterldowns10/02/2024

Re-asking [0] as a top-level question, since it has gone unanswered: do you intend to make a business out of this project in some way, or is it a "real" open source project?

I know that intentions can change, but I'm curious how you see it. Sourcegraph was pretty clearly always going to be a business-type-of-project, and like most business projects, relicensed everything to their custom enterprise license. Originally it was Apache 2 [1].

I love open source and I write a lot of it myself [2]. I use the MIT license, just like you've done here, and I admire that. I don't think you owe me or anyone else anything, and the MIT license makes that clear.

I am very interested in this project and I'd love to extend and contribute to it, but only if it's an actual open source project. Seems like every devtools-focused startup these days calls themselves "open source" but fails to actually build a community, because in reality it's just a marketing gimmick. Because the project is actually a company, the people involved never try very hard to build a community of contributors. When the company invariably cannot make money with an open source product, the code gets relicensed to be closed-source. The few people who had contributed end up getting played. That's what happened to Sourcegraph!

So: open source, or open source "for now"?

[0]: https://news.ycombinator.com/item?id=41715776

[1]: https://github.com/sourcegraph/sourcegraph-public-snapshot/c...

[2]: https://github.com/peterldowns

show 2 replies
hanwenn10/02/2024

Hi!

sorry for not responding to your email, I was swamped.

I looked through the sourcecode, but I can only find UI (ie. browser) code. Does this do anything beyond delivering a more functional and prettier UI on top of an existing zoekt deployment? If no, everybody would be better served if you tried to improve the UI inside Zoekt, which currently is a live demonstration of (my lack of) web app programming skills.

Have you thought of how you will achieve your further goals (eg. semantic search)? That will require server-side changes, but you currently have no Go code at all.

show 2 replies
morgante10/01/2024

Awesome to see another open source player in the space, especially after Sourcegraph went closed source.

It looks like you're working on this full-time (and it's a lot of work to build great code search, as I know from working on my own product).

What are your plans for monetizing / building a sustainable business without inevitably going closed source like Sourcegraph?

show 1 reply
threecheese10/01/2024

Regarding your response to “why not use an IDE?”; do you have any other product-like use cases interest you? The one you mention - search across many repositories - makes a lot of sense for organizations with (for example) a Github Enterprise installation and want to investigate or make changes across multiple components. This is definitely relevant to me, and so I wonder what other cool things can I do with it?

show 1 reply
awenix10/02/2024

Another solid code search tool https://github.com/hound-search/hound.

Based on regexp

show 1 reply
mdaniel10/02/2024

I'll point out that you're missing a stellar opportunity to showcase your own champagne via

  --- a/README.md
  +++ b/README.md
  @@ -1,256 +1,256 @@
  - We do not collect or transmit [any information related to your codebase](https://github.com/search?q=repo:sourcebot-dev/sourcebot++captureEvent&type=code)
  + We do not collect or transmit [any information related to your codebase](https://demo.sourcebot.dev/search?query=repo%3Asourcebot-dev%2Fsourcebot%20captureEvent)
which regrettably currently says "No results found" :-(
show 2 replies
maxloh10/02/2024

Why not just fork Sourcegraph, instead of building the product from the ground up?

IshKebab10/01/2024

Nice! Still not quite as good as grep.app from an interface point of view. They have instant search-as-you-type results over all of GitHub.

It's not open source but I use it all the time. Far superior to Github's search.

show 1 reply
jmakov10/01/2024

Can somebody share the use case of this? Why not just use your IDE?

show 3 replies
zdw10/01/2024

Does this make a copy of each repo on ingest?

Can it work against in-place repos, for example if hosted on the same server as a code forge installation?

show 1 reply
metaroxx10/03/2024

How would this work if you want to index different branches of a repository only?

For example I’d like to index branches release1, release2, etc. but not have it index developer temporary gitlab MR branches.

I assume HEAD is referred to the head of the default branch when cloning the repository.

planb10/01/2024

Great work! Any plans to add Gitea/Forgejo (self-hosted) support?

show 1 reply
acloudbutwhy10/01/2024

What sort of effort is required for additional host types? I see an issue is opened for self-hosted Bitbucket which would be a blocker for me to try it.

schreiaj10/02/2024

Can you add repos after starting the container? What about persisting indexes across restarts?

Still, neat. Glad to have an easy to deploy open source tool like this.

show 1 reply
mattfat510/01/2024

This is well done thanks for the share.

ashobeiri10/01/2024

This is really exciting. Happy to see someone building an open source solution in this space

TavsiE9s10/01/2024

Any plans for non Github/Gitlab integrations? Gitea/Gogs/etc. maybe?

show 1 reply
ergocoder10/02/2024

What a milestone. SourceGraph is big enough to have its own open source clone

cprogrammer10/02/2024

Does it support Perforce? i couldn't find it in the schema in the repo.

show 1 reply
j4coh10/01/2024

Cool to see someone carrying on the dream after SourceGraph lost their way.

show 1 reply
asdev10/01/2024

sourcegraph is dead with advent of LLMs and AI coding tools right? Github cross repo search is also not bad anymore

show 2 replies