One of my best and most productive work situations was remote with a week[0] together every quarter. Key to this was scheduling the next trip while we were together to make sure it was on the books. We got to meet new team members, share some meals together, work through new architecture designs with a whiteboard, and plan. Not much got done during that week, but we sure got a lot done each quarter.

[0]: This was actually Monday-Thursday with travel on Friday