I don't really understand the distinction here. Are you saying that it's not possible to harm someone by damaging their property?

Sure I destroyed their car and they weren't able to go to work and got fired, but I didn't physically attack them so no harm done.