Maybe not. If there is a force as strong as parts of government, it's other parts of government. Intelligence community is its own kingdom, with its own goals, but other areas of government run on a four year cycle and depend on public support. So if people will want restrictions in surveillance badly enough, politicians' pathetic pandering to public opinion could create a very powerful counter force.