It's not clear to me that these approaches aren't already being tried.
Firstly, by some researchers in the big labs (some of which I'm sure are funded to try random moonshot bets like the above), at non-product labs working on hard problems (eg World Labs), and especially within academia where researchers have taken inspiration from biology before, and today are even better funded and hungry for new discoveries.
Certainly at my university, some researchers are slightly detached from the hype cycle of NeurIPS publications and are trying interdisciplinary approaches to bigger problems. Though, admittedly less than I'd have hoped for). I do think the pressure to be a paper machine limits people from trying bets that are realistically very likely to fail.
>I do think the pressure to be a paper machine limits people from trying bets that are realistically very likely to fail.
Oh certainly. I also think it’s just a sweet spot of efficiency and scalability that transformers happen to occupy. A new paradigm will need to be more effective at similar cost.