It's not an issue if your office is so small that no one is hanging out in front of it. :)

My wife only comes in to get printouts and supplies if I'm working, and if she's working (we share the "battle station" by switching out whose laptop is connected to the dock) I basically only go in there to quickly chat and walk around to the other side.