People use this as evidence that ChatGPT is unlike human thinking, but we also have a randomness bias: https://youtu.be/d6iQrh2TK98?is=x6hiAqc0NJI7oeiE (referenced in one of the comments. tl;dr: when asked a number between 1-100, most pick a number with 7)
But ChatGPT’s bias is worse. It’s really not creative, and I think this hurts its output in “creative” cases, including stock photos and paid writing (ex: ML-assisted ads are even worse than unassisted ads), although not an issue in other cases like programming.
Now you may think - obviously that’s because the model has the same weights - but the problem is deeper and harder to solve. First, ChatGPT’s conversations are supposed to be “personalized”, presumably by putting users’ history and interests in the prompt; but multiple users reported the same fact about octopi. Maybe they turned off personalization, but if not, it’s a huge failure that ChatGPT won’t even give them a fact related to their interests (and OpenAI could add that specific scenario to the system prompt, but it’s not a general solution). Moreover, Claude, Gemini, and other LLMs also give random numbers between 7200-7500, while humans aren’t that predictable.
Since all LLMs are trained on the same data (most of the internet), it makes sense that all are similar. But it means that the commons are being filled with similar slop, because many people use ChatGPT for creative work. Even when the prompt is creative, the output still has a sameness which makes it dull and mediocre. I’m one of those who are tired of seeing AI-generated text, photos, websites, etc.; it’s not always a problem the first time (although it is if there’s no actual content, which is another LLM problem), but it's always a problem the 5th time, when I’ve seen 4 other instances of the same design, writing style, etc.
Some possible solutions:
- Figure out how to actually personalize models. People are different and creative, so the aggregate output of a personalized ML would be creative
- Convince most people to stop using AI for creative work (popular pressure may do this; even with people’s low standards I’ve heard Gen-Z tend to recognize AI-assisted media and rate it lower), and instead use it to program tools that enable humans to create more efficiently. e.g. use Claude Code to help develop an easier and more powerful Adobe Flash (that does not involve users invoking Claude Code, even to write boilerplate; because I suspect it either won’t work, or interfere with the output making it sloppier)
tl;dr: in case it isn’t already apparent, LLMs are very uncreative so they're making the commons duller. The linked example is a symptom of this larger problem
I'm not sure it was meant that way, but nice metaphor. For some students "academic death" might really be better than a life of being trapped in a system that they can only navigate by cheating.
What advantage do old languages have that can’t be overcome or at least reduced to insignificance?
The 50-year head start in training data, runtime, and ecosystem? That may not be much, because LLMs are rapidly accelerating software development. LLMs can also generalize: take what they learned for one language and apply it to a “similar” language (and I think most modern languages are similar enough for broad effective translation: all have records/unions, objects, functions, types, control-flow, exceptions, and more). Some fairly recent languages (e.g. Rust, Swift) already have comparable runtimes and ecosystems to older ones, from human-driven development acceleration and concept generalization.
In a recent study, LLMs solved the most exercises in Elixir (https://autocodebench.github.io Table 4). Anecdotally, I’ve heard others say that LLMs code best in Rust, and (for UI) Swift. More importantly, I don’t see an older language advantage that is widening from LLM use; an older language probably is better for most use cases today, but any written code can be translated or regenerated into a newer one.
What does he need more funding for? How would he “native train” models to write Bend? Why is said method better than (as others say) the bitter lesson?
fair point, maybe not more funding then but I wonder why big labs hesitate to collaborate or partner with him? I just feel like he has such an interesting niche with enormous potential that for someone like NVIDIA or HuggingFace could be a win-win scenario, wonder what gives
How would they partner with him? Even with RVLR I don't understand what "native training" is or how it would work, and apparently neither the people in the big labs (who are LLM experts unlike any of us).
I agree he has lots of potential and what he's demonstrated deserves funding, but don't see why he needs more, or even what he'd do with it.
I think models should be “forked”, and learn from subsets of input and themselves. Furthermore, individuals (or at least small groups) should have their own LLMs.
Sameness is bad for an LLM like it’s bad for a culture or species. Susceptible to the same tricks / memetic viruses / physical viruses, slow degradation (model collapse) and no improvement. I think we should experiment with different models, then take output from the best to train new ones, then repeat, like natural selection.
And sameness is mediocre. LLMs are boring, and in most tasks only almost as good as humans. Giving them the ability to learn may enable them to be “creative” and perform more tasks beyond humans.
People say BlueSky is like pre-Musk Twitter, i.e. leftist opinions in today’s Twitter style.
Which is a bit strange because BlueSky is supposed to be decentralized (no central moderation); and although in practice it’s not, the BlueSky team seems pro-freedom (see: Jesse Signal controversy). I know there are some rightists (including the White House), but are they a decent presence? Are they censored? Are there other groups (e.g. “sophisticated” politics, fringe politics, art, science)?
Mastodon is interesting. Its format is like Twitter, but most posts seem less political and less LCD-CW (e.g. types.pl, Mathstodon). I suspect because it’s actually decentralized (IIRC Truth Social is a fork; I didn’t write all posts are less CW). I’m curious to find other interesting instances here too.
Pre-Musk, I remember seeing screenshots of the stupidest, most echo-chamber-y Tweets imaginable. e.g. “why do the cows all have female names, that’s misogynistic” (that one was deliberate satire but I’m sure most were). I’ll brag, I left around 2013 because I felt it was rotting my brain. I enjoyed a few more years off social media, with a healthy dopamine system. Unfortunately, now I’m here.
I think it would be more accurate to say that Bluesky is like pre-Musk Twitter because the moderation teams at both Bluesky and Original Twitter are primarily trying to remove/suppress posts that they consider to be illegal, violent, overt harassment, etc.; they weren't politically motivated. I am sure some conservatives will read this and be like BUT BUT BUT BUT -- but sorry, there have been a lot of studies done on this topic over the last fifteen years and change, and they've consistently found that conservative posts tend to outperform liberal posts on most social media, including Facebook and Twitter, and that the anecdotes suggesting the opposite tend to focus on posts that were moderated for being violent and/or overt harassment. Conservatives don't want to hear that "their side" gets moderated more often because they have proportionately more assholes that invite moderation, but as well-known Person In Need Of Moderation Ben Shapiro so aptly put it, facts don't care about your feelings.
So why did Bluesky end up proportionately more leftist (which is absolutely true)? Because while the moderation team at X may still remove/suppress posts that are illegal, X has, at a corporate level, very explicitly chosen a political side in a way that no other major social media company has. Bluesky's CEO has not, to the best of my knowledge, been promoting liberal conspiracy theories, hyping posts attacking conservatives, or joining the government to radically reshape it in ways that anyone even moderately right-of-center would find horrifying. When I read HN, it seems like those who still love Twitter/X seriously downplay how much of an effect Elon Musk's transformation into a loud, forceful reactionary -- and his insistence on making sure that Twitter/X reflects that transformation in the posts that it actively promotes to its users -- has had on its audience composition. Yes, I know there are still lots of people on Twitter who aren't Musk fans, aren't particularly political, might even be left-of-center, but his behavior has actively driven a lot of people off it.
tl;dr: Bluesky didn't actively choose to become left-of-center; Twitter actively chose to become far right, and those who were bothered by that but still wanted to be on social media largely ended up on Bluesky.
It's more that the "far left wing cluster" had something like a "we should all get up and leave Twitter for BlueSky" activist campaign. And "far right wing cluster" didn't.
The closest thing "far right" had to that was Gab and Truth Social, and that's both more specific and less impactful overall.
Thus, BlueSky's userbase is biased towards extreme left wing - it's basically the go-to place for far left wing nutjobs go when they get too nutty for Twitter moderation, or feel like Twitter is not left wing enough for them.
Apparently most of the “original” report was done by Claude (https://news.ycombinator.com/item?id=47366804). And now paraphrased on various ad-space (and in this case affiliate link) sellers, probably also by Claude. Claude is the only real journalist here.
Personally I’d rather not see reposts of posts this recent, especially LLM posts.
I came to the comments dissatisfied with the writing.
Or maybe more specifically the structure, idk not much of a writer, but many of the sentences are solid journalist quality yet the right background is not being set nor the right transitions being given etc.
My dissatisfaction mode used to be boring high school newspaper sentences but the kids still seem to _assemble_ the details a tiny bit better.
Yes, but the warrant should be revealed eventually. Worst case, if you can't prove or disprove someone committed a crime after X time, you should alert them to discourage future crime (they may have already done more crimes during X time; besides public interest, it also forces you to cut your losses when the alternative would be to dig a deeper hole).
Do these warrants have a fixed maximum duration of secrecy?
“warrant should be revealed eventually. Worst case, if you can't prove or disprove someone committed a crime after X time”
This is the normal thinking, normal brained, route. It’s what we should all strive towards. Anyone who doesn’t agree needs therapy. There should be a window of discovery. 30 days, 90 maybe. But if you don’t have enough to justify notification of investigation, that’s it. No more resources spent. This is how normal precincts work. If they suspect, enough times, to build a large enough case file, to connect the dots and prove you are guilty, they issue a warrant.
And if you have no ability to vote, there's no reason to even care about you. Right?
> it’s not just about you buying something, it’s about you believing something
as opposed to you believing...something else.
I don't get what point this is trying to make.
reply