Why do you think changing PM's is a bad thing?

The PM is selected from the party that has the majority in Parliament (typically the party leader), so if the PM changes without a national vote, it's still a MP from the majority party.