That is my experience as well. Another case where the "clumsy proxy" issue happens is when I help a frontend dev debug something with the backend, especially online. Do collaborative IDEs exist? How usable are they? A collaborative terminal might be useful too (for when I want to run a few commands quickly to check something on my colleague's computer). Thinking about what I just wrote, maybe I could ssh into their machine?
People doubtlessly do it. I sometimes do it when I tutor somebody on a server we both access via SSH. If you use such a server as a jump server, you can setup SSH tunnels to get all the way to another desktop.
I think it would be wonderful to install a remote IDE with workspace on a remote server. Or to VNC/RDP into one.