AFAIK Swift 3 was supposed to be the last version to introduce breaking changes. Have they changed that policy? I know that the last Swift update I did was very manageable compared to in the past (where I'd easily lose a day or two).
This project's still on Swift 3, so I don't have first-hand experience upgrading to Swift 4. It looks like there are breaking changes though, albeit not nearly as many as 3 had.
The most notable one looks like in the String API [1]. There are a few other minor ones that most projects won't hit, like in the dictionary/set API [2]. A few of these are also slightly more breaking than it initially seems because they've added temporary compatibility workarounds marked as deprecated in Swift 4 to avoid breaking as much working Swift 3 code, but these will go away for Swift 5, so you do still have to port anyway (just not immediately).