But it is really hard to remember the shortcuts
Tool-tips, having an accelerator next to the item in the menu/context menu, accelerator underlines, god forbid documentation … UIs have long had ways of having a simple way and hinting at, documenting, or showing to the user how to accelerate if you want to get quicker.
You don't really need to remember everything though. Mouse navigation does not need to go away. The keyboard shortcuts should be available if you opt to use them, after which point you will be able to memorize them in short time.
The intuitive part of my argument also falls under this. By following known conventions as close as possible, we can eliminate the need to remember the most common actions.
Additionally, it is important to present your shortcuts in a shortcut window/dialog in a logical way for when the user needs to remember something.
In any case, an excesive number of keyboard shortcuts can have a negative effect on usability, so this is also important to keep in mind.