No need to install it on remote systems. I'm more of a Vi guy and I have a decent selection of plugins on my primary system, but I appreciate being able to use the built in Vim features without installing stuff on remote machines.
Though Magit seems maybe a bit of a weird thing to be using on a minimal remote system.
And you can use the built-in emacs features via emacs -nw on a remote system. Core vim doesn't come with a git client I assume? So I'm not seeing the difference.
Though Magit seems maybe a bit of a weird thing to be using on a minimal remote system.