In my experience (admittedly only with recent versions) there aren't any problems that crop up when using XeTeX with the memoir class for simple stuff like ebooks. The only real downside is that the microtype support in XeTeX isn't as good as in pdfTeX, but LuaTeX is pretty close to providing the best of both worlds.