The fundamental problem is that, while the people grant power to representatives at elections, there is no mechanism to revoke it. For example, the Senate can impeach the president mid-term, but the people cannot. Nor can they remove their senators or congressional representatives.