I'm also sure that some kind of linear architecture is possible. After all, humans don't have N^2 perfect recall either.