they kinda tried that and it is my understanding that it sucks. One example of "direct linear mapping":

https://pdfs.semanticscholar.org/5080/78a97f4778eee488e11032...