FWIW, it seems pretty clear that while Fermat briefly thought he had a proof, he fairly quickly realized it was flawed since he never mentioned this publicly and later went on to develop a proof just for the simple n=4 case.

In 1847 Gabriel Lami presented a claimed (simple) general proof to the French Academy of Sciences, only for the flawed assumption in it to be pointed out immediately at the end of his presentation! This may have been the same proof that Fermat had in mind.