I think it's already game over if they have access to your home directory. They can also edit your path at that point.