I get the annoyance at "ask me again later", but "maybe later" or "not now" is helpful - there's no way to trigger dialogs by yourself, so this is a promise that even if you say "no" in the dialog, you can still go into settings to change your mind. Gives me less anxiety about the choice.
A much better way would be “Yes” and “No” with a small message of “you can change this later in the Settings”.
Or have three buttons: “Yes”, “No”, “Remind me Later”. Apple does stuff like that in other parts of the OS, like when asking for app permissions, so they clearly know how to do it.
How about a choice: both "no, maybe later" and "no, never"