Any local kernel vulnerability will let you attack other containers on the same machine. This is a much bigger attack surface than Xen or KVM. It's nice in that it gives the same experience as "traditional" hypervisors (at least the basic features), but it's only applicable if you trust that the application inside the container will not be compromised.
On the flip side, when there is a kernel vulnerability, there's only 1 kernel to update! If you're running the Canonical Livepatch Service, for instance, critical security vulnerabilities are patch in real time, without reboot, and all containers running immediately benefit from the patch. Conversely, if you're running 50 Linux Xen or KVM machines, you have 51 kernels to update. So yeah, do think about what "attack surface" actually means, when comparing LXD and Full Virtualization.
On the same vein, there are plenty of Xen machines that are vulnerable despite the host OS being updated (which, honestly, I don't thing happens as quickly as it should considering it requires a restart for new kernel params). With things like live-patching, updating the kernel once, while running, makes sure all guests are also updated as well.
It's a trade off, but one that seems to trend towards more secure despite potentially a few quirks.
LXD supports using different id maps per container, which mitigates some of that.
I get that containers will always have a larger attack surface than Xen/KVM. Just thought it was worth mentioning that some container approaches are thinking about security more than others.