I don't know a lot about this, but for example an would app side loaded or delivered via F-Droid be subject to this policy?

F-Droid notwithstanding using an alternative app is not a very attractive option right now as there no good alternatives. But if Valve can create an Android app store that competes with the Play store, the in principle situation is very different.