Voting on policy directly would be taking up everyones time. It's just not practical for anything bigger than a small village.
The problem with today's system is we have MPs or representatives instead of delegates. They are not bound to their campaign promises in any way other than risking a small fine if someone fights them in court for months to prove they were lying. Once you get elected you can do what you want until the next election unless you really piss off your colleagues and get kicked out.
Voters should be able to call an early election in their district and override their reps/force them to vote a certain way, without having to personally show up and decide every single issue.