alt Hacker NewsIf you want to experiment with hardcoding small programs into transformer weights, maybe try ALTA: https://arxiv.org/abs/2410.18077v2
If you want to experiment with hardcoding small programs into transformer weights, maybe try ALTA: https://arxiv.org/abs/2410.18077v2
I'm less interested in turning programs into transformers and more interested in turning programs into subnetworks within large language models.
Which the blog post brings up as a research direction, but never actually elaborates upon. And the interface between the two is a hard problem.
I'll check out the link though, thanks.