I think a first step would be a sane tab order, so you can easily navigate to a control using tab or cursor buttons. Second step are keyboard shortcuts.

I don't see why removing the buttons would be desirable. All that would do is to make the GUI unusable with the mouse and also make the discovery of the keyboard shortcuts much harder.