This was one of the rare critiques of AI doom that actually understands the case for it and presents them well, so I kept reading to see what its arguments for our safety against AI doom were. They were roughly:
1. It's hard to put a cat in a box despite us being smarter than a cat, so we're safe. (Counter: we're pretty good at putting cats in boxes when it matters.)
2. It was hard for Australia to kill Emus, so we're safe. (Counter: Australia could probably kill all Emus if it mattered enough, and we definitely accidentally kill off species when one of their inputs for life matters enough to us.)
3. Some smart humans get paralyzed by hedonism or existential angst instead of optimizing for arbitrary goals implied by their arbitrary value sets, so we're safe. (Counter: others overthrow the Czar, land rockets, etc.)
4. Modern AI is data-trained, so recursive improvement requires more data, so we're safe. (Counter: AI-crafted, synthetic data is a thing.)
5. We don't (yet) know how to improve our brains with brain surgery, so we're safe. (Counter: same as #4 above, which unlike us/evolution AI is being deliberately trained to understand and perform.)
6. Children take a long time to grow up, so we're safe. (Counter: the author's own "Premise 5: Computer-Like Time Scales", where they correctly note that computers can be arbitrarily faster than us.)
7. Individual smart humans on a desert island would be cooked, so we're safe. (Counter: nothing says the capability of a single AI must stop at that of an individual human, or that of a small group of smart humans; humans brains got dropped into a savannah and eventually they launch rockets.)
8. If AI doom is not a real threat, believing in it makes you believe some other not-real things that seem crazy or distasteful. (Counter: do we have a clear argument why it is not a real threat yet, in the list above?)
2. At what cost? Much like the climate change above, you'll have people on the AI side even when it's out in the field extincting us.
4. Adding, over time synthetic data and its generating algorithms can become unaligned with human needs/behaviors (an example would be our current stock market, numbers must go up!).
8. Going back to climate change, it was predicted a long time ago, and while the explosion of automobiles has greatly improved human lives the risks of climate change could erase a lot of that. Might have been better if we dealt with the problem before we have to give the thermometer worried looks.
Within software engineering, security, reliability, and scale also seem boundless.
Software that never breaks (including because it never runs into scaling problems) and never leaks your data is preferable to software that breaks and leaks your data sometimes, but it has been too costly to be practical.
Current models are still very far from the reasoning muscle required to build things that never break, scale to billions of users with no issues, and cannot be exploited.
> Software that never breaks (including because it never runs into scaling problems) and never leaks your data is preferable to software that breaks and leaks your data sometimes, but it has been too costly to be practical.
It's almost impossible to prove non-trivial software is invulnerable.
It's very easy to prove that it sort of works.
For one, you have hardware vulnerabilities - period. If you're running on any operating system, you have OS vulnerabilities. If you're not running on bare metal, you may have who knows what kind of vulnerabilities. If you're running literally any other piece of software on the same machine, depending on the hardware and OS, you could have vulnerabilities...
Good luck finding any objective distinction between HN, Reddit, WhatsApp, Signal, or email + listserv and "social media"!
Or finding any one of "social media's harms" that could not, in some world where Facebook, Instagram, and Tiktok did not exist, be delivered in just as socially harmful (and beneficial) a form by sufficiently-accessible versions of the apps and protocols you value and use every day.
I met someone recently whose primary addiction is Wikipedia.
For me, Signal and Hacker News are the most addictive pieces of software I still use.
Serial television (best delivered by WebTorrent and The Pirate Bay) is by far the most addictive, for me, so much so that I had to quit.
And you can definitely run successful social movements and political campaigns (for both very good and very bad things) over HN or WhatsApp/Signal, given sufficient adoption.
That as well. But everyone reading GP’s posts knows in their bones that it’s unsustainable. It’s economically unsustainable and environmentally unsustainable, and in that context it strikes me as pure hoarding behaviour. Taking as much as they can for themselves before the house of cards crashes down.
I have no sympathy for OpenAI or Anthropic as corporations, but if these are the new tools of the trade, then platform abuse like GP is bragging about serves only to destroy the livelihoods of the rest of us who are content to use our fair share.
There’s no such thing as a free lunch, and the bill always comes at the end.
I mostly hate it because the token crunch is now coming for us regular users because of people like this. A few people always ruin it for the rest of us.
Yea. It’s greed, pure and simple. And also a major misstep on the part of the inference providers to offer these subsidized plans and not anticipate these slop mills.
The visuals weren't terrible, I thought, but the writing, dialog, acting (except for Moura), and narrative arc were terrible.
It's one of those movies where almost everyone looks like they just really love being on stage ("isn't cinema lovely?") and where the writers have an idea of what cliches they're trying to work with but can't land them into an actual story, even a story made out of cliches.
If it becomes commonplace for existing prediction markets to get undermined by this kind of manipulation, won't that just be an opportunity for people to create better prediction markets that are less vulnerable to manipulation?
And doesn't that just mean more resources and energy is going into solving the problem of determining the truth of past events (and, as a result given that these are prediction markets, the likelihood of future ones?)
If those prediction markets are patronised by people who want to manipulate it, what drives customers to the new ones?
Making it less vulnerable to manipulation would entail exposing less information too. You probably wouldn't be allowed to know the current odds, which makes gambling the same as reading tea leaves.
Both sports betting and prediction markets are effectively the same as reading tea leaves anyway. You can determine the real odds in roulette or blackjack because those are closed games that have simple probabilities, but any odds given in sports betting or prediction markets are just fancy guesses
The odds in betting are the payout ratio. To hinder manipulation, the payout ratio would have to be hidden and then the platform can just always say you won less money than you did.
It's wild that there are as many jobs in the category "Top Executives" as in the category "Retail Sales Worker".
This makes sense given both automation and the US's role in the global economy, but it runs somewhat contrary to standard ideas of class and inequality.
That category has a median pay of $105,350, and includes "general and operations managers" as well as "chief executives". I assume it includes executives of very small enterprises.
Good point. To take it one step further, if they are including 'general managers' and 'operations managers' in this bucket, then that should include the GM and Ops Manager at places like retail stores as well (for example, every Best Buy location has both positions, I'm sure it's similar for Walmart and other big box retailers too).
Remember that exec tech salaries are extreme outliers. I worked for an exec in manufacturing. He had full p&l responsibility for a business segment with ~150 employees, $27 million in revenue at 40% gross margins, and a production plant. His total comp was ~$300k.
Now just think of the comp levels in sectors like government, education, etc.
> Remember that exec tech salaries are extreme outliers.
It's the combination of tech and big or fast growing companies.
People who operate in FAANG or Silicon Valley bubbles (or who spend too much time on Blind) can lose track of what salaries look like in the rest of the world.
I often share Buffer's open salary page because their compensation is actually pretty normal from all of the data I've seen and hiring I've done: https://buffer.com/salaries
Every time it gets posted there are comments from people aghast that the software engineers "only" make $200K and in disbelief that the CEO's salary is "only" $300K.
These categories are extremely broad. Top Executive includes general managers, legislators, school superintendents, mayors, city administrators, and a lot of other government jobs. The name is misleading, it's basically non-frontline management.
Chief Executives is actually a specific sub-category of it and is, obviously, much smaller.
When people think "top executives" they think of a very, very small group of people making tens of millions of dollars a year or much more. The reality is that that's not the case.
Whether they participate voluntarily in a commercial transaction or participate only when compelled to by law (setting aside the question of whether the government does or should have that power) is certainly their call to make.
Just as any individual can decide whether to volunteer, whether to wait until drafted, or whether to refuse to be drafted and face the consequences.
(History shows these decisions, and the rights to make them, are meaningful at scale!)
Finally, governments who expect their leading scientists to do groundbreaking work simply out of fear of imprisonment are NGMI against governments whose scientists believe in their cause.
1. It's hard to put a cat in a box despite us being smarter than a cat, so we're safe. (Counter: we're pretty good at putting cats in boxes when it matters.)
2. It was hard for Australia to kill Emus, so we're safe. (Counter: Australia could probably kill all Emus if it mattered enough, and we definitely accidentally kill off species when one of their inputs for life matters enough to us.)
3. Some smart humans get paralyzed by hedonism or existential angst instead of optimizing for arbitrary goals implied by their arbitrary value sets, so we're safe. (Counter: others overthrow the Czar, land rockets, etc.)
4. Modern AI is data-trained, so recursive improvement requires more data, so we're safe. (Counter: AI-crafted, synthetic data is a thing.)
5. We don't (yet) know how to improve our brains with brain surgery, so we're safe. (Counter: same as #4 above, which unlike us/evolution AI is being deliberately trained to understand and perform.)
6. Children take a long time to grow up, so we're safe. (Counter: the author's own "Premise 5: Computer-Like Time Scales", where they correctly note that computers can be arbitrarily faster than us.)
7. Individual smart humans on a desert island would be cooked, so we're safe. (Counter: nothing says the capability of a single AI must stop at that of an individual human, or that of a small group of smart humans; humans brains got dropped into a savannah and eventually they launch rockets.)
8. If AI doom is not a real threat, believing in it makes you believe some other not-real things that seem crazy or distasteful. (Counter: do we have a clear argument why it is not a real threat yet, in the list above?)
reply