The problem with DVI is twofold:

First, font support is purely by reference which means that you need some way of connecting the fonts used in the document with the DVI file. Use of the wrong font could produce some spectacularly bad output.

Second, graphical support, other than rectangular boxes is only handled through the xxx opcode which never had any standardized meaning (although I tried). This limitation also applied to colors. Really, it was only with the final victory of PDF as the universal document format that these limitations were finally ameliorated.