Yes, it's practically possible. But, there's a lot of gotchas there... probably need to schedule at home during a recess, not during sessions. Probably need to declare a specific topic, not just a chit-chat. Likelihood goes up if you bring a group of constituents with you so the Congressman can get more input in that block of time (which is now looking suspiciously like the "lobbyists" you decry).

That said, for single-person "I want to tell you about thing X", a staffer probably works just fine. That's why they're there.

For a local politician, who likely has much more impact on your day to day, this is all far easier. They're generally less busy, less travel, and more approachable. And, they're the ones setting real estate taxes, school policy, and other things that make places truly livable (or not).