Unfortunately that is how it works. A vote for the green party is simply a vote not cast for D and favors R; and a vote for a libertarian is a vote not for R, so it benefits D.
A solution is Ranked Choice Voting where you can say, "Green, and if they don't win, D (or whatever)."
Fwiw, I vote my conscience, not to win. Not the best for my political positions maybe, but I hope to send a signal to others that maybe something other than R/D is one day possible. But, yeah, RCV would help with that conundrum.