The inflation fears are part of the vicious cycle. The typical mechanism by which a government would get rid of inconvenient amounts of debt denominated in their own currency is to monetize it and inflate it away. Bond purchasers know about this, and build inflation expectations into the interest rates they are willing to accept. This increases the amount of interest the government has to pay, which makes its financial position even more precarious, which builds in future expectations of either a default or a soft-default through inflation, which pumps interest rates even more.

It is possible to break this cycle, but it requires getting spending under control. As long as you operate in deficit, it requires finding private capital to finance future operations of the government. That private capital will require interest commensurate with the expected future devaluation of its principal to lend; otherwise they are just suckers. If you can bring spending into balance then you can hike rates and bring inflation under control and you'll be somewhat insulated from what bond purchasers are willing to accept, but if you can't then increases in rates just increase government spending as well.

Japan operated at extremely high levels of debt-to-GDP because their deflationary trap turned all major Japanese corporations into large net savers. Why would you hold debt when you have to pay it back in more valuable yen in the future? That created a very large oversupply of private capital, which crowded into government bonds as the spender of last resort.