Beyond things that are provably carcinogen and specific genetic deficiencies its just calories. Every diet looks better compared to the standard American slop, but when tested against reducing caloric intake, every diet looks like the null distribution.
Dont be fooled. Just eat sensibly in amount and composition.
The composition of those calories impacts hormones in the body. Hormones do matter. If for nothing else, making it easier or more difficult to actually stay within that caloric budget.