The interest rate on long bonds does not affect the coupon on existing bonds. The bondholders lose, not the Treasury. An increase in short rates would be more damaging, to the extent that the government's debt is short-term, but nothing happens in a vacuum. The kind of demand that would cause significant inflation (and, therefore, a significant rise in short rates) would generate a dramatic increase in taxable incomes, as would the payment of higher interest by all borrowers, so it's not clear that the increased interest cost would not be offset by revenues. But maybe. I would be interested in that sort of dynamic analysis.