In Australia the answer is political lobbying, without a doubt.

We had an emissions trading scheme[0] in 2012 meant to help in a transition to clean energy sources that was aggressively lobbied against by Australia's largest polluters and lasted only 2 years before being repealed by the incoming government by labeling it a "tax" that citizens would pay for. This led to a decade of policy stagnation[1] where we could've been transitioning away from fossil fuels.

So while energy density is definitely a factor, political lobbying is absolutely a factor.

[0] https://en.wikipedia.org/wiki/Carbon_Pollution_Reduction_Sch...

[1] https://www.ft.com/content/0a453f5c-e859-4300-9355-46822c451...