I feel like the obvious thing to check here is that all chrome processes have terminated between tests. If it's still running some zombie process of course the data would persist.
At least on windows, closing chrome doesn't close the system tray process which keeps Chrome and Gemini prestaged and cached. Wonder if OP is experiencing a version of that on his Mac?
It would still be a (albeit convenient for google) bug, as the setting indicates closing all windows as the trigger for deletion.