How about a delay before making the package publicly available so that there could be an undo window? I'm not sure what an appropriate duration would be, as there are obviously situations where someone might want to publish an urgent bugfix quickly.

This could also save publishers from their own "whoops" moments, akin to gmail's super-handy "undo send" feature[1].

[1]: https://support.google.com/mail/answer/1284885?hl=en