No it just means the person paying gets a discount on the tax, nobody else pays anything extra for that.

If the tax credit results in an increase of solar panel installations it’s still a net gain for society, even just financially.

While technically correct, assuming a balanced budget means less tax revenue will have to be offset either by reducing government expenses or raising taxes for everyone, so in practice all taxpayers 'pay for' a tax credit.

But you assume that 100% of homeowners will buy solar at full price no matter what, with such certainty that it can even be included as a given in the budget before it happens.

That's an unreasonable assumption IMO. Solar is not food, homeowners can choose to skip it.

That only works if the budget is balanced or the debt is paid off.

It is either today's or tomorrow's taxpayer.