That UXN thing looks like it could be trivially jitted on-the-fly. Unless I am misreading their site, and the code doesn't have to reside in the ROM, of course.
There isn't ROM per se. The “ROM” image is always loaded into RAM and is free to self-modify as much as it feels like.
Actually no, there's no "ROM", that's just the name for the code that gets loaded into RAM. Even the article talked about self-modifying code...
It's a Von Neumann machine with no instruction cache coherence instructions. JITs for these are not trivial to produce due to every memory store potentially invalidating the JIT code, so you need clever solutions to make that invalidation extremely fast.