It's not so simple. Check this out: https://en.wikipedia.org/wiki/Freediving_blackout#Ascent_bla...
I never got "deep" into it but everybody told me that breath holding diving can be really dangerous.
I know part of the SCUBA story is that phenomena like nitrogen narcosis are particularly dangerous because you need your cognitive capacity to survive in the underwater environment.
In the surface world I can go to a party and drink eight beers and maybe throw up and act like a dumbass and embarrass myself and then wake up with a headache the next morning. That level of incapacitation under water would likely be fatal.
Barring really traditional (and now very rare) pearl/scallop divers, professional divers aren't doing it by holding their breath.