am I missing something or is the alien though experiment not obviously scary? There's only 30 of them and they don't reproduce much faster, and they don't have a tech advantage. This seems like it has some potential concerns but not existential civilization and humanity ending problems and I'm not even worried those aliens will take my job.
AI is riskier in a lot of ways from that so it doesn't scan to me as a good thought experiment.
But it says more ships aren't coming. I would definitely be worried if more were coming and we might get out competed or turned into a colony or whatever but the author goes out of their way to cut off those parts, so what's scary?
You fast forward 10 years and find that your new laptop is Alienware. Because, it turns out, the super smart aliens are damn good at both running businesses and designing semiconductors, and, after a series of events, the aliens run Dell. They have their own Alien Silicon (trademarked), and they're crushing their competitors on price-performance.
And that's not the weirdest thing that could have happened. Corporate alien techbros are weird enough, but they could have gotten themselves involved in world politics instead!
Ah, but what happens when somebody rich and powerful decides the aliens’ advantages will serve their purposes, and stands up an industrial-scale cloning operation?
There are only so many base models to date, right? With limited and somewhat ambiguous utility, and no real reason to impute intention to them.
Still, in the short time since they’ve arrived, their existence has inspired the people with money and power to geopolitical jousting, massive financial investment, and spectacular industrial enterprise—a nuclear renaissance and a “network of data centers the size of Manhattan” if I remember correctly?
The models might well turn out to be just, you know—30 kinda alien but basically banal digital humanoids, with a meaningful edge on only a few dimensions of human endeavor—summarization, persuasion, retrieval, sheer volume of output.
Dynomight’s metaphor seems to me like a useful way to think about how a lot of the dangers lie in the higher-order effects: the human behavior the technology enables, rather than the LLM itself exercising comprehensive intelligence or agency or villainy or whatever.
AI is riskier in a lot of ways from that so it doesn't scan to me as a good thought experiment.