The screen would dim more aggressively and location-aware apps would be less accurate. Many users would be annoyed by this, though they might not know it was a result of the battery-saver setting.
When you turn battery saver on transition animations disappear (making everything feel less reponsive). The phone stops polling for updates as frequently. The screen dims. It's a noticeably worse experience because it does a lot to save battery. When I have plenty of battery or know that I'll be charging my phone soon I would much rather have all of those nice things than not. It's a tradeoff because the circumstances dictate how important saving battery is.