A server needs a lot more things to have a GUI compared to what's necessary to just run an SSH daemon that offers CLI/TUI access, where SSH is just providing secure network access for the most basic interface the server has anyway (the TTY).