one form of very effective self-improvement that coding agents do all the time:
install or build stuff that they can then use
it changes the environment instead of the agent/harness but in a sense how separate is the agent from its environment and why do we apply this distinction re self-improvement?
animals and humans do the same thing and are great at it, without 'self-improvement' with emphasis on the 'self'
Blame Descartes? I don't know. Extended theories of cognition all get shot down, even analytic philosophers who are accused of being more "continental" still primarily hold mind-body dualisms