Let's abstract this further: It's about the user's existing power and intentions, meaning if I am already in a position of power, AI will multiply it to levels way beyond a peasant could. Power dynamics just get exacerbated.
Then check V-Dem, you might argue they're flawed as well but then I'd suggest you to provide counterexamples for why the US should be considered a functioning democracy, and is not on the way to a fully authoritarian state.
The CCP knows, whatever the heck this technology will bring with itself, the current power dynamic inside of the country is on their side, and AI will solidify it.
I hypothesize that, rather than slowly having it disperse in society and allow people to harness it in ways they don't want, they might as well accelerate everything until AI becomes the totalitarian swiss knife - which they can make use of in the best way of course.
I don't really see how open weights models further what you're talking about.
It's trivial for me to download one of their models and run it on my Spark, and there's all sorts of ways to strip out their Tiananmen-denialism or whatever.
If/when the memory price crunch dissipates, even more so. And so far it's only China I see as making moves to increase production capacity on memory, too.
If anything the centralization of capital into US-based Anthropic and OpenAI is far more terrifying from the perspective you're outlining.
US used AI (Claude on Maven) to determine a girl's elementary school as a target in war[0] and then triple tapped it and you're still more worried about hypothetical misuses of the single country responsible for this technology not being concentrated in the hands of a few powerful elite? ffs
PS: I not trying defend bombing schools, but posting that its "AI" resposible is opposite of what you need to do if you care.
Its military - there been specific people who found this location for the strike, then some senior officers who choose it without checking and specific people who executed it. And its all logged with "paper" trail in chain of command.
It was all people with specific names who are responsible to avoid bombing schools. They failed. Not "AI".
The issue is that you wouldn't be able to even transparently get to any evidence, as these models are blackboxes.
They might start scheming behind employees backs as soon as they realize they are being used in critical infrastructure of adversaries. And nobody would know until it's too late.
reply