"There are a couple of types of flaky tests that come to mind:
Tests that perform network operations
...
The first one is easy, ban these. You don’t need them in the blocking portion of your pipelines."
At a certain large FAANG company I worked at in the past, there were some end-to-end tests that used the network that were blocking. Most e2e tests were not blocking, but a few were considered critical enough to halt the whole build/deploy.
We had a system that checked both flakiness and running time for each of these tests. If they were consistently flaky, they were removed from the pool and the owner was informed and had to manually work to put it back in the pool and get sign-off, including showing things like load test results to prove that the same test had been run many hundreds of times in different environments.
"Your version control system not being able to handle the sheer number of references and operations done to the repo."
This was also an issue. For most companies, git is totally fine (unless you are filling it with petabytes of binary data or something else that is not a good idea without special mitigations) but the argument that "git is used by the Linux kernel! It should work for you!" falls down when your codebase is at least 20 times bigger than the Linux kernel.
Tests that perform network operations ... The first one is easy, ban these. You don’t need them in the blocking portion of your pipelines."
At a certain large FAANG company I worked at in the past, there were some end-to-end tests that used the network that were blocking. Most e2e tests were not blocking, but a few were considered critical enough to halt the whole build/deploy.
We had a system that checked both flakiness and running time for each of these tests. If they were consistently flaky, they were removed from the pool and the owner was informed and had to manually work to put it back in the pool and get sign-off, including showing things like load test results to prove that the same test had been run many hundreds of times in different environments.
"Your version control system not being able to handle the sheer number of references and operations done to the repo."
This was also an issue. For most companies, git is totally fine (unless you are filling it with petabytes of binary data or something else that is not a good idea without special mitigations) but the argument that "git is used by the Linux kernel! It should work for you!" falls down when your codebase is at least 20 times bigger than the Linux kernel.