Did some debugging. It works after I disabled compression and control master.

I didn’t like CLI, since it didn’t pick up my default ssh key from the agent.