> whereas a tiny change in a packfile shouldn't add much at all

Well to the extent that git is able to find a suitable delta, which is the difficult part.