Yes, retrieval from disk caching is not as fast as one would expect. Detailed article: https://simonhearne.com/2020/network-faster-than-cache/

Speed is not the only criterion. Using network is a waste of energy and materials when the local cache was enough (even more when nobody sees the perf difference)

Since the svg is not inline but rather loaded externally it is being cached aswell. (source https://stackoverflow.com/questions/37832422/how-can-we-cach...)

That's a valid point, for subsequent loads in this case the inline svg is so small that when compared to the image tag it's replacing the difference is pretty small.

But the other side of this is cache latency, this depends on the caching policy defined in the http header, for example some modes require validating caches with the server which incurs a round trip even if it doesn't require always reloading the resource. If it's fully offline caching then as a sibling comment pointed out, disk caching is not free either, under some threshold (which definitely applies here) inline is going to be the fastest way to get an svg rendering.

When optimizing for lighthouse pagespeed metrics, yes.