Compromising OpenWrt Supply Chain

A vulnerability not mentioned in the article is the normalisation of executing code that has been especially targeted to a specific user or specific device with no validation of reproducibility and no ability for anyone to verify this custom build and download service hasn't been generating backdoored builds.

One should want to ensure use of the same build of xz-utils that Andres Freund is using, or at least a build of xz-utils that other security researchers can later obtain to figure out whether supply chain implants are present in open source software[1].

There's a write up at Mozilla[2] from years ago describing an abandoned attempt by Mozilla to ensure their release builds are publicly logged in a Merkle tree. Google has written up their implementation for Pixel firmware builds but apps delivered through the Google Play Store seem to be vulnerable (unless there is another log I have been unable to find).[3] Apple is seemingly worse than Google on binary transparency with Apple's firmware and app distribution system targeting builds to individual devices with no transparency of builds.

For an example of binary transparency done well, Gentoo's ebuild repository (being a single Git repository/Merkle tree containing source checksums) possibly remains the largest and most distributed Merkle trees of open source software.

[1] Post xz-utils backdoor, some researchers (including some posting to oss-security about their efforts) undertook automated/semi-automated scans of open source software builds to check for unexplained high entropy files which could contain hidden malicious code. This is not possible to achieve with customised per-user/per-device builds unless every single build is made publicly available for later analysis and a public log (Merkle tree) accompanies those published builds.

[2] https://wiki.mozilla.org/Security/Binary_Transparency

[3] https://developers.google.com/android/binary_transparency/ov...

Isn't the "".join also dangerous?

    get_str_hash(
        "".join(
            [
                build_request.distro,
                build_request.version,
                build_request.version_code,
                build_request.target,
                ...

That's why open source can never compete with business grade closed source stuff:

- they fixed the in 3 hours instead of making customers wait 6 months for a patch (if any)

- they did not try to sue the reporter of the issue

- they did not even tell the users to throw away the "outdated" but perfectly working devices, offering a small discount to buy new

First - open source tool adjusted to the task it wasn’t made for in short time, only because it is open source and written without BuilderFactoryProvider. (already mentioned so I’m sorry, but it’s killing me every day) Big company would take probably -1 years to fix this, because it would just sue the guy, try to arrest him ASAP and never release patch. OpenWrt after getting information just took the insecure service offline, checked the report (while clients were already safe, because of the shutdown), made patch and released in 3 hours. Wow!

Hi, I'm the one who created this services within a "Google Summer of Code" some 7 years ago and been maintaining it since. It's my first "larger" project and while it started as a short Python project, I eventually became a OpenWrt project member since the build system itself required so many changes.

I'd be very happy for further audits and reviews of the code, after multiple years of low interested, it now produces and caches about 1000 individual firmwares a day. I think it's only a question of time until other issues come up...

Loving this. I wonder how people even come up with an idea of truncating hashes. For what purpose or benefit?

First of all, nice writeup. I am a bit surprised that so much GPU power was needed to find such short collision but it was nice to see his implementation nevertheless.

Regarding the last section, is 40k a reasonable price for one month of security analysis? Does this mean that a good security researcher make about 500k/yr?

>I immediately noticed that the length of the hash is truncated to 12, out of 64 characters.

Ouch...

Killer write up, very clever bit of code reading and exploit development.

What's this about hashcat performance being orders of magnitude different depending on arg order? Is it scanning the argument line for target pattern with every execution?

The title uses the term "supply chain" but it appears nowhere in the blog post. I keep seeing this term used by "cybersecurity" researchers and software developers in ways that seem to differ from the definition I learned in school. .

From Wikipedia:

"A supply chain is a complex logistics system that consists of facilities that convert raw materials into finished products and distribute them^[1] to end consumers^[2] or end customers.^[3]"

    1. ^ Ganeshan, R. and Harrison, T. P., An Introduction to Supply Chain Management, updated 22 May 2005, accessed 29 June 2023

    2. ^ ^a ^b Ghiani, Gianpaolo; Laporte, Gilbert; Musmanno, Roberto (2004). Introduction to Logistics Systems Planning and Control. John Wiley & Sons. pp. 3-4.  ISBN 9780470849170. Retrieved 8 January 2023.

    3. ^ ^a ^b ^c Harrison, A. and Godsell, J. (2003), Responsive Supply Chains: An Exploratory Study of Performance Management, Cranfield School of Management, accessed 12 May 2021

I always assumed commercial suppliers compiled from source to add their own modifications, and then created their own images.

As a consumer of OpenWRT, I compile from source or use "official" images.

Very well written and easy to follow description of your attack.

OpenWrt is also very difficult to safely upgrade on some devices which I would also consider as a huge downside. I finally gave up and bought an old Dell off eBay and installed OpnSense and am much happier.

Kudos to the researcher for finding this, and to the openwrt team for the impressive response.

I know that opnsense is preferred over pfsense when it comes to performance, but does openwrt compete at speeds at or above 10gig?

Is there any way to fix the command injection solely in the Makefile?

I'm getting an error when I try to view this:

Secure Connection Failed

An error occurred during a connection to flatt.tech. SSL received a record that exceeded the maximum permissible length.

Error code: SSL_ERROR_RX_RECORD_TOO_LONG

No-one else?

Nicely done. Good write up too. I liked the bits about making hashcat do what you wanted.

Very cool article, security researchers are incredibly creative and scary smart.

That was an excellent report and a really decent technical explanation. Good to see how quickly OpenWrt (one of my favorite open-source projects) fixed and addressed this vulnerability!