Well it's in the books. O(n^2) algorithms are bad in the long run, transformers algorithm has such complexity, so not a big surprise we hit the limits.