This rules out entire classes of app and would make using a computer a miserable experience.
For example let's say you want to make an app that every day writes a backup to a particular location e.g. 1Password can do a daily backup of your encrypted passwords to a backup location.
Or, let's say you want to make a GUI around a command line program that stores its config as a dotfile.
Without a way to save access to file system locations persistently, apps would be forced to constantly shove open panels in your face all the time.