Are you claiming that ’ (Unicode code point 002019) is not suitable as an apostrophe? In that case, what is? The classic ASCII 000027? Isn’t that one straight? (As opposed to ’, which is slanted correctly.)
Or are you perhaps thinking of ´ (Unicode code point 0000B4, ACUTE ACCENT)? Yes, that one is ugly and should not be used as either an apostrophe or a right quote.
This does not work, because the “right-skewed” apostrophe is not appropriate for every case of “straight” apostrophe placement.
However, making every “skewed” apostrophe “straight” is very easy for a computer to do (e.g. MusicBrainz Picard ships with a setting for this). This is why the MB guidelines say to prefer correct typographical apostrophes.
Note that they are preferred and not required, so you can still enter straight ones, but another editor may update them. I honestly don’t see the problem, but I am very pedantic about other things so I guess I can imagine how seeing the curlies could rub you the wrong way!
The reference glyph for ' (code point 000027) <https://www.unicode.org/charts/PDF/U0000.pdf> shows a straight rendering. Any Unicode font rendering it differently can reasonably be said to be rendering it incorrectly.
Or are you perhaps thinking of ´ (Unicode code point 0000B4, ACUTE ACCENT)? Yes, that one is ugly and should not be used as either an apostrophe or a right quote.