alt Hacker NewsThey were almost certainly not doing the OCR on the device. Such a CPU would be too power-hungry and the process would still be too error-prone given the limited compute. They would just send the raw data to the C2, which could send it to an emulator, snapshot the output and send it to an LLM.
Replies
i don't think OCRing a bitmap font is very expensive or power hungry, if you know the exact pixel configurations of the glyphs ahead of time
Streaming the raw data would also be more flexible for other use cases. From what I've read, it was streaming off the generated recovery passphrase, but users aren't required to generate one--they can input one. That'd require a much more sophisticated algorithm than simply reading from the screen, since users are using two buttons to pick from hundreds of words in a randomized menu.
And besides, Ledger firmware can be updated to change fonts and such so it'd be long-term safer to just stream the raw data.
Well it doesn't need to be a generic purpose OCR, just one that works for this single UI layout and font.