What you are suggesting sounds like a consensus mechanism. How do you safely choose which peers to assign this responsibility? How do you ensure it is resistant to a Sybil attack?

Suddenly the answers begin to look a lot like Proof of Work or Proof of Stake.

The answer could or could not look like that. In practice people have done this many times throughout history totally without blockchains (or computers) for that matter (e.g., money transfer systems, exchanges, etc.).

If this has been done many times in a way that solves my original question (decentralized escrow), there should be some concrete examples you can point to besides blockchain/crypto systems.

Obviously, non-digital assets, but does the Hawala system fit your analog version for decentralized escrow? Early stock exchanges were sometimes created to get around existing third parties ("auctioneers") and allow the brokers to directly transact p2p. What about a credit coop (it is central, but it is also owned by all the users)?

Edit: at it's most basic, reliable coinage was kind of way to create reliable p2p abilities without risk of "double spend". Once the coins were out there, central authority didn't matter so much, i.e., "good" coins were used fair and wide beyond the coining state (e.g., Athenian tetradrachma). Funny add. in some areas people actually allow temporary double spend (so that can be another solution)...

Perhaps, but my original question was around digital assets across the internet (and ideally on a global scale). If you are trading a plush toy for a coconut the "atomic swap" could be done with each party's two hands.

In the generic form, blockchain/DLT cannot solve the double spend there either. Selling a picture for a token does not magically delete all copies or makes it impossible for me to sell it again (yes, someone could look in some blockchain, but that might not even deter another buyer - even fake goods trade).

Basically, for on-chain assets, a blockchain solves double spend and also ensures that on-chain funds/assets are correctly delivered (malicious attacks aside). This also exists outside of blockchains, for example in payment vs payment settlement in FX. Banks created CLS precisely to avoid having one part of an FX transaction settle while the other was still outstanding - so other ecosystems with immaterial goods and risky settlement found other solutions/created their own "middle man mechanic".