For normal VR/AR, definitely, since you want to have objects moving in the Z direction. For this usecase it should be enough to show the "flat" virtual screen at the focal distance.