alt Hacker NewsShai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem
Comments
> """ I'm strongly in favor of blocking post-install scripts by default. :+1: This is a change that will have a painful adjustment period for our users, but I believe in ~1 year everyone will look back and be thankful we made it. It's nuts that a [pnpm|yarn|npm] install can run arbitrary code in the first place. """
- a pnpm maintainer 1 year ago
> stored in our database which was not compromised
Personally I don't really agree with "was not compromised"
You say yourself that the guy had access to your secrets and AWS, I'd definitely consider that compromised even if the guy (to your knowledge) didn't read anything from the database. Assume breach if access was possible.
Really appreciate the transparency here. Post-mortems like this are vital for the industry.
I'm curious was the exfiltration traffic distinguishable from normal developer traffic?
We've been looking into stricter egress filtering for our dev environments, but it's always a battle between security and breaking npm install
Very offtopic but this caught my eye:
> Total repos cloned: 669
How big is this company? All the numbers I can find online suggest well below 100 people, and yet they have over 600 repos? Is that normal?
> This is one of the frustrating realities of these attacks: once the malware runs, identifying the source becomes extremely difficult. The package doesn't announce itself. The pnpm install completes successfully. Everything looks normal.
Sounds like there’s no EDR running on the dev machines? You should have more to investigate if Sentinel One/CrowdStrike/etc were running.
That’s weird, pnpm no longer automatically runs lifecycle scripts like preinstall [1], so unless they were running a very old version of pnpm, shouldn’t they have been protected from Shai-Hulud?
I have been thinking about this. How do I make my git setup on my laptop secure? Currently, I have my ssh key on the laptop, so if I want to push, I just use git push. And I have admin credentials for the org. How do I make it more secure?
"The simultaneous activity from US and India confirmed we were dealing with a single attacker using multiple VPNs or servers, not separate actors."
Did it really? It's not clear to me why the possibility that the exfiltrated credentials were shared with other actors, each acting independently, is ruled out.
I am loving the ancient Lovecraftian horror vibe of these exploit names. Good for raising awareness, I guess!
The Torvalds commits were a common post infection signature, common in the random repos that published secrets (Microsoft documented https://www.microsoft.com/en-us/security/blog/2025/12/09/sha...)
It was a really noisy worm though, and it looked like a few actors also jumped on the exposed credentials making private repos public and modifying readmes promoting a startup/discord.
Would they detect this if the attackers just silently keep leaking the information, as opposed to go destructive about it?
The approach the attacker took makes little sense to me, perhaps someone else has an explanation for it? At first they monitored what's going on and then silently exfiltrated credentials and private repos. Makes sense so far. But then why make so much noise with trying to force push repositories? It's Git, surely there's a clone of nearly everything on most dev machines etc.
Given that all the stolen credentials were made public, I was hoping that someone would build a haveibeenpwned style site. We know we were compromised on at least a few tokens, but it would be nice to be able to search using a compromised token to find out what else leaked. We’ve rotated everything we could think of but not knowing if we’ve missed something sucks.
We don't have a clear explanation of the destructive behavior, right? It looks like it had no real purpose, and there were much more effective ways of destroying their repos. Very script kiddie-like, which does not really fit the main complexity of the virus. Very surprising.
I'm wondering why storing creds in env variables as plain text is acceptable - e.g. they better be dynamically fetched from a secret manager with 2FA in the way
Points for an excellent post-mortem.
> This incident involved one of our engineers installing a compromised package on their development machine, which led to credential theft and unauthorized access to our GitHub organization.
The org only has 4-5 engineers. So you can imagine the impact a large org will have.
It’s almost like Microsoft sells security products and runs the most insecure JavaScript package manager to build those security products and couldn’t switch off of it even if the engineers in the org recommended a more secure JavaScript execution context— and that’s realistically why anthropic bought an engine.
NPM post-install scripts considered harmful.
There has to be a tool that allows you (or an AI) to easily review post-install scripts before you install the package.
Surprised that people allow force-push on git. If it needs to be done, it should only be done after consultation and disabled after.
> Running npm install is not negligence.
I beg to differ and look forward to running my own fiefdom where interpreter/JIT languages are banned in all forms.
[dead]
>Running npm install is not negligence. Installing dependencies is not a security failure. The security failure is in an ecosystem that allows packages to run arbitrary code silently.
No, your security failure is that you use a package manager that allows third-parties push arbitrary code into your product with no oversight. You only have "secutity" to the extent that you can trust the people who control those packages to act both competently and in good faith ad infinitum.
Also the OP seemingly implies credentials are stored on-filesystem in plaintext but I might be extrapolating too much there.