Term limits lead to institutional knowledge and skills being concentrated in unelected staffers and lobbyists. Effective legislating involves building relationships, negotiating skills, and deep subject matter knowledge in at least some areas.
Yes, we have tons of bad legislators: some just not good at their jobs, some actively harmful (leaving that vague on purpose—I think all partisans can agree that they exist, even if we disagree on who they are). In theory, they can be excised via the ballot box. However, we don't want to kick out the good ones just as they're getting to be their most effective—not only do we lose their direct skills, but we lose their ability to mentor the promising up-and-comers.
I place more blame on the way we do primaries and general elections: in most districts, the only thing that matters is the primary, and that produces some truly rotten results.
> Term limits lead to institutional knowledge and skills being concentrated in unelected staffers and lobbyists.
Who cares? They don't get to cast the votes.
I'm not sure I buy that. Sure, if everyone in Congress terms out at the same time, and your next Congress is full of fresh faces, you absolutely run into that problem.
As long as things are staggered, senior legislators will mentor junior legislators, and that institutional knowledge will be passed on.
And I don't think we're talking about limiting representatives and senators in the same way we do for the president. I would say it would be fair to allow them to serve for something on the order of 15-20 years.
But sure, I think there are many other problems that matter more: winner-takes-all elections that essentially require you have only a two-party system, the electoral college, and (as you mention) the primary system.