An untargetted attacker cannot know that there is no authentication. Dragnetting connections where they don't recognize any authentication therefore risks detection.
That risk to the attacker is not present when observing plaintext connections.
.
I'm not safe from muggers because I have eyes in the back of my head to see them trying to sneak up on me. I'm (largely) safe because someone else is likely to see them (or catch them on camera) and get them caught.
One of the ways to detect MitM is authentication. It is a particularly good method, which I recommend whenever possible, but it is not the only method.
Suspicious changes in the environment might be another, as would detecting data that leak past the middleman. Key pinning would be an example of a change in the environment, unexpected changes in important network topology or routing could be another. An example of a leaking middleman might be detection of the real (non-poisoned) "duplicate" packet in a DNS-poisoning packet race.
These methods are nowhere near as good as proper authentication, of course. Reliability of detection is probably very low. The point is that it is better than the case of sending plaintext that anybody can trivially wiretap with zero chance of detection[1].
As always, it is important to define your threat model. If you are defending against any kind of targeted attack, then yes, authentication is a firm requirement. If your threat model is only concerned with avoiding the trivial surveillance that can be done in bulk, anything that forces the opponent to use a more complicated ("expensive") MitM attack is a success.
[1] modulo any still-very-hypothetical quantum communication methods. We can reevaluate our options if those technologies ever work well enough for common use,
