I hate that, personally. I don't always have to agree with whoever downvoted, or sometimes I'm just curious about what it said.
Especially the lowest contrast, after several downvotes, that's just stupid, because it's pretty much unreadable unless you select the text. So you know there's text, but you can't really read it, then why show it at all?
Except that's not a downvoted comment or an old version as far as I can tell; it's the text of a Ask HN/Show HN thing. I never really understood why those are colored gray, either.
There's a fair amount of merit to downvoted comments having their contrast lowered.