The full error correction system involves qubits. This paper is mainly about the decoder, which is responsible for taking the symptom data produced by the quantum circuit and determining the most likely errors that caused those symptoms. In the blog post it's not stated what code is being run, but in the illustration it's clear it's a surface code [1] and this is confirmed in the paper's abstract [2].

Disclaimer: am one of the authors, but not a main contributor. I wrote the simulator they used and made some useful suggestions on how to use it to extract information they wanted for training the models more efficiently, but know nothing of transformers.

[1]: https://errorcorrectionzoo.org/list/quantum_surface

[2]: https://www.nature.com/articles/s41586-024-08148-8.pdf