How much of American society will get to share in the benefits of those new biological and medical discoveries when we don't have any health insurance because we lost our jobs to AI?

The problem is, at least in theory, that it entirely changes the calculus of how advancements take place. In the past, when the pace of advancement was stronger the primary factor was the cultivation of a culture that valued prestige and knowledge over monetary gains. It didn't really matter how much money you threw at a problem because the bulk of the people responsible for advancements weren't interested in obscene wealth. Obviously those people were well compensated but any number of entities could provide that compensation. It was about bringing prestige to your lab / school / town or even country.

If AI becomes a primary catalyst for advancement it further moves the needle in the monetary direction.

That redfines advancement to mean something different than what is beneficial to society to be what is monetarily best for the owners of said advancment.

Yes, that's very nice. But that's very different models from LLMs and slop image generators. AI as a term has been butchered beyond recognition; when mentioning the current harm of AI investor hype and job automation, people are talking about generative models using LLMs or prompt based input, which have seen little to no use in "accelerate biological and medical discovery"

Sure, the transformer is great for making larger neural networks with better learning potential, which are improving protein folding models a fair bit. But do we need the combined budget of the Apollo program or interstate highway system (adjusted for inflation) per year, to develop better molecular simulation models? (no, the most advanced ones run on mundane hardware and trained just fine on pre 2020 infrastructure).

So while it's true that; "AI" ((primarily) Neural network based deep learning techniques) are wonderful tools to make society better; slop generators absorbing the entire energy budget of a few small nations to generate infinite propaganda, linked-in posts and shrimp Jesus is only tangentially helping in that goal while destabilization civilization in the process.

I don't think that is realistic. But, it can write your child's essay for them.

But someone has to be able to buy the results, right?

I fear that at this rate the oligarchs will use medical breakthroughs to keep us alive and laboring against our and nature’s will like what industrial farming has done to chickens and other livestock

Superhuman abilities for the wealthy tech oligarchs, economic indentured servitude and slums for the rest of us.

Tens of millions of people in the US alone cannot obtain basic healthcare today, how would this outcome change for them because AI solved it? The only solid paths are regulation or prying the machine from the hands of those who hold it. GLP-1s are only widely available globally affordably because the patent expired, for example.

Does accelerating biological and medical discovery require over a trillion dollars of capital to be misallocated while Americans do not have medicare for all or universal childcare?

Who is exactly going to benefit here because Americans have been given a rotten deal by neoliberalism for the last 40 years.