There is way more carbon in the ground as rocks than as oil. If you have plenty of energy, the difference is quite manageable.
Besides, as somebody already pointed out, there is that CO2 on the air that we actually want to get rid of. It's nothing compared to the rocks, and a little harder to get, but getting it first would improve things a lot.
The carbon isn’t valuable elementally as much as it is structurally and molecularly. I mean that as aromatic rings and other ready made building blocks that conveniently can be fractionally separated with pressure and temperature conditions in a column as a gross generalization. All of this is energy intensive but much less so than building up from three atom molecules with strong bonds. And much much less energy intensive than separating a trace % molecule from the atmosphere at low atmospheric pressure and translating that to complex molecules.
There needs to be more appreciation for the laws of thermodynamics when discussing technology. Everything is not a 1-dimensional reduced abstraction.