I don’t think the Supreme Court as a whole wanted to overturn Griswold. At best, specific justices like Clarence Thomas wrote in a concurrence that he would want to overturn it, but that’s still far from the opinion of the court. Given the character of Thomas, it’s safe to say that most of his opinions don’t become opinion of the court.