It is mumbo jumbo (also people hearing “significant” treat this as “effect/change is large and important” which is in no relation to the actual amount of change).

Low p-value basically means how surprising your data would be if there were actually no effect (ie less than 5% of the time you’ll get this due to randomness if there is no change — which is rather impossible)

Sample size matters heavily. With more observations, estimates become more precise, so increasingly small differences can become statistically significant. With a large sample, you can therefore get a tiny, practically meaningless effect with a very small p-value.

Eg effect of $1 can be statistically significant (not random) which does not matter in practical terms if average is like $10000.

So the key point here is not only to look at the p-value but also at an actual change. If a drug gives you only 0.01% more hair, it doesn’t matter to you that it is guaranteed.