Associate it with the specific service they don't want you using, or transactions they don't want you making, or conversations and connections they don't want you having.
Yes, misinformation is a problem. Deanonymization is a bigger problem. If you can't say anything anonymously, it becomes much more difficult to fight entities bigger and more powerful than you.
I agree, but that isn’t a good argument to offer to the entities bigger and more powerful than me.
Governments and companies feel a pressing threat of a trump-like populist overtake in each country. They need the bots, fake socials and slop stopped yesterday. An abstract degradation of freedom of speech isn’t going to cause pause.
There is a national security argument that I think is more likely to help, at least for non Americans. Do you want a foreign power to have control over your citizens phones being functional?
The irony in this line of thought is that by stifling anonymous speech and enabling censorship, countries will usher in their own reactionary movements as dark money is globally spent on platforms to push paid advertising advancing reactionary rhetoric. It's already happening in the UK, Germany, France and Spain.
Right-wing populism isn't what's being banned here, it's dissent. Platforms are happy to take domestic and foreign fascists' money and push their agendas no matter where they are globally because it benefits them, too. Those paid placements aren't being banned, your ability to disagree with them and not be identified is.
That’s a very good point, it’s another hole in the sieve.
This “fix” just routes people through official channels but those channels aren’t exactly proving to be worth the term walled garden. My YouTube adverts lately border the quality of early 2000s piracy sites, it’s honestly baffling how little they value their own product in their willingness to take anyone’s money.
Credit unions, at least in theory, are known for caring more about their customers. It'd be worth explicitly giving them the feedback that you use them via their website or via an app that works on an Open Source phone, and telling them that that's one reason you're a customer.
Fraud prevention. If they lock things down, they lose less money to fraud. I think they should just have to suck it up and eat the cost but obviously they don't think that way. Only a small minority even understands and cares about these issues. The money they save by trampling over our freedom is no doubt much higher than the value brought in by us. They will no doubt sacrifice us for increased profits if we force the issue. We have no leverage.
There is no reason whatsoever for a major corporation to not use remote attestation technology. Banks will use it because fraud. Streaming services will use it because piracy. Messaging services will use it because spam, bots. If you're the corporation, the user is your enemy and you want to protect yourself from him.
Governments want this too. Encryption. Anonymity. They need to control it all. Free computers are too subversive for them. They cannot tolerate it.
> If they lock things down, they lose less money to fraud.
[Citation Needed]
I see this kind of claim made often, but never backed up with evidence that remote attestation of consumer devices has any real-world impact on fraud. It sounds like it could be true because it would detect compromised devices, but it could just as easily be false because people with devices that don't pass are usually technically sophisticated.
"There are venues that provide tickets exclusively via mobile applications, for instance."
Turns out Ticketmaster still has ticket printing machines at such venues
Was at a game at one of them, claimed I had a problem with the app and after some negotiation at the ticket window a millennial printed me a ticket
Why do they still have the printers
The "I'm having a problem with the app" strategy can work in other contexts too. The phone can be configured so that a young person trying to help gives up
"Modern" software is highly fallible and everyone knows it
When people have problems using apps, alternatives are often available
Perhaps this is why, e.g., venues that "require" apps still have ticket printing machines and still print tickets when there are problems with using the apps
The situation is not so "cut and dried" that no one ever attends an event at these venues using printed tickets instead of displaying the ticket on the phones they bring to the event
There are alternatives to apps that are sometimes used, e.g., when customers have problems, even when businesses try to "require" apps
As such, businesses do not always succeed in collecting the same amount of data from every customer
This is not to say customers who try to avoid unnecessary data collection always succeed, either
Generally, trying is a prequisite to succeeding
If most customers do not try it does not mean no customer succeeds. There are some who do, at least some of the time
Ticketmaster is it's own particular problem that needs to be dealt with, even if it is emblematic of a bigger issue with companies demanding users to run proprietary software.
I have recent (October and November, 2025-- venues in Indianapolis, IN and Cincinnati, OH) personal experience with this. With one venue I was able to play the "confused old man" card (via phone) and get the box office to print my tickets and hold them at will call.
At another venue I called prior to my show and tried the same tactic. They told me flat out "no phone, no admittance, tough luck for you" and cited the warnings and terms on the Ticketmaster website that I'd already agreed-to. I didn't want to chance losing out on $300 of tickets I bought so I knuckled under and loaded the Ticketmaster app on my wife's iPhone.
I don't think it's as cut-and-dried as you say it is, and I don't have the stomach to risk being denied access to events I bought tickets for-- particularly at the pricing levels of today's shows.
Well fuck those venues. It's a small percentage. I've never run into one and I live in LA, a city with hundreds if not thousands of venues.
So you only get 98% of the world instead of 100%. That 98% is far more than the the 100% of 10 years ago. Everyone wants perfection when they've already got abundance.
It has been reported that Ticketmaster has exclusive agreements with 70-80% of US venues. It's great that you have all the choices you do. For me, in western Ohio, every major venue for hundreds of miles in every direction is an exclusive Ticketmaster venue. You can't gain admittance to any show in those venues without a phone that can run their proprietary app.
Ticketmaster is bullshit, for sure, but they're just one example of the problem of being forced to use proprietary user-hostile software.
> Harassment is persistent attempts to interact with someone
No, harassment also includes persistent attempts to cause someone grief, whether or not they involve direct interactions with that person.
From Wikipedia:
> Harassment covers a wide range of behaviors of an offensive nature. It is commonly understood as behavior that demeans, humiliates, and intimidates a person.
Doxing in the loose sense could be harassment in certain circumstances, such as if you broadcast a person's home address to an audience with the intent to cause that audience to use that address, even if the address was already out there. In that case, the problem is not the release of information, but the intent you're communicating with the release. It would be the same if you told that audience "you know guys? It's not very difficult to find jdoe's home address if you google his name. I'm not saying anything, I'm just saying." Merely de-pseudonymizing a screen name may or may not be harassment. Divulging that jdoe's real name is John Doe would not have the same implications as if his name was, say, Keanu Reeves.
Because the two are distinct, one can't simply replace "doxing" with "harassment".
Generally speaking, every case I've seen of people using the term "doxing" tends to be for the case that specifically is harassment; it has the connotation of using the information, precisely because if you aren't intending to use it there's no good reason for you to have it.
Language evolves. Connotation tends to become definition. Not always the only definition, but connotation becomes the "especially" or the "definition 2", and can become the primary definition over time.
That's not what I mean. If we agree that harassment is wrong and that doxing is not harassment (because not all doxing is harassment), then it's incorrect to say that doxing is wrong. For example, the article from the blog, even if we agree that it is doxing, isn't harassment. The person being discussed is presented in a positive light:
>I for one will be buying Denis/Masha/whoever a well deserved cup of coffee.
Using one term when what is meant is actually the other serves nothing but to sow confusion.
You can harass someone while discussing them in a positive light.
And i don't just mean under colloquial definition, i mean under the legal definition of harrasment. In fact its fairly common for unwanted "positive" attention to be harrasment - e.g. unwanted sexual advances mostly fit that description.
You are generalizing an irrelevant point. What I was getting at is that unlike the usual usage of doxing, it was not a call to go bother that person. I didn't think I needed to make that point this explicitly within the context of this subthread.
Which is irrelavent as that is not a requirement for it to be harrasment.
I get that a call to action is a common feature of doxing and it wasn't present here, but its not a particularly common feature of harrasment outside of the context of doxing and nothing in the definition of harrasment requires it.
> The only thing that GLP-1 agonists prove is that CICO does indeed work
This is incorrect, as demonstrated over and over again. For many people's bodies, consuming less will result in the body changing its metabolism to burn less, and not dipping into fat stores. Conversely, for many people's bodies, exercising more does not in fact change their metabolism and the amount of energy they burn. (There are studies that going from "zero" to "not zero" makes a meaningful difference, but "not zero" to "quite active" often doesn't.) "CICO" is not useful or actionable for many people.
That there is variance in energy expenditure both within a population and within a person over time doesn't mean that a caloric deficit doesn't work. It just means that using a single scalar value (which is usually a gross estimate) to drive your caloric intake is a poor approach.
The body has means to regulate it's energy expenditure to maintain homeostasis, and in some people it can be a hundreds of kcal difference. But if you're trying to lose body fat on a 10% estimated deficit and fail, the conclusion shouldn't be that a 20% deficit will also fail.
For some people, a 50% "deficit" fails. And the entire concept of "X workout burns Y calories" is completely bunk. Again, there have been multiple studies to this effect.
The body may try to maintain homeostasis but 50% sounds way too high. Someone with a tdee of 2200 kcal will not be able to maintain their weight at 1100 calories for very long.
Adaptation in energy expenditure includes both metabolic adaptation as well as "NEAT" ("non-exercise activity thermogenesis"); the latter includes subconscious changes in posture, fidgeting, and various other things that can increase/decrease the body's energy expenditure by a massive degree, in an effort to (as far as people can tell) maintain a "set point" in the body that is difficult to change. This set point resists both weight gain and weight loss, both attempting to resist the change in the first place and attempting to undo it if successful.
I'm not suggesting that it's impossible to lose weight through sufficiently large caloric restriction. I'm observing that it is not anywhere close to as simple as "CICO", because CO is heavily a function of CI, rather than the popular incorrect perception of CO being things like "exercise".
There is a wide gulf between “when you exercise your body saves calories elsewhere in the day…when you eat less your metabolism slows down” and “some people can’t lose weight on 50% calorie restriction.”
The former is very well supported in the literature. The latter is only supported in low quality studies like where people self report their diet.
The CICO hypothesis accounts for metabolism. Your weight is a function of CICO and time. You can track calories in, weight, and time. From there you derive calories out.
The problem isn’t that CICO is wrong, the problem is that it turns out actual caloric restriction over time is really really hard.
The current best advice for weight loss is to avoid highly palatable and/or calorie dense foods, prefer foods that are highly satiating and low in calories, and strength training helps. Do whatever cardio you need for heart health; more cardio is at best unnecessary and at worst demotivating because extra effort will not net extra results. Slow and steady is easier to stick to.
One problem is that even though small deficits lead to more long term success, small deficits are very hard to track, and very hard to stick to.
When the size of your calorie deficit is two tablespoons of ranch dressing and a cookie per day, it’s easy to blow past it without even realizing.
CICO isn’t a diet strategy. By itself it won’t help you lose weight any more than kinematic equations help you to throw a ball.
Neat can maybe explain a couple hundred kcal variance in most people, perhaps there are exceptions but 50%? I've never seen that in the literature.
Calories in calories out is just the summation of expenditure and intake, just because the body is complex and there are many interdependent factors doesn't mean it cant be resolved to a vector which determines weight gain/loss. The problem is people google a tdee calculator, get some scalar which is likely wrong, perhaps substantially, make lifestyle changes, and then have an expectation of some result in a specific timeline that isn't realistic, and then eat a bunch of sodium, put on 2 lbs in their "deficit", and think the diet made them fatter! Or they read that -3500kcal == -1lb fat, calculate their calories burned from the machines at the gym, and get frustrated when it doesn't work (I'm guilty!).
Weight loss is actually really hard because it really just requires a sustained effort over a long period of time to achieve anything. You might not see any results for weeks as your body adjusts, you get your diet locked in, etc. And since your weight can vary so much day to day, it's hard to stay motivated. Ozempic kind of bypasses these problems. You know what else works? 20k steps a day and eating on a backpacker budget :P
> But then HN would still riot, because you would need to require all apps to be approved by a central authority (no unauthorized browsers) OR you need to lock down browser engines to those that respect the list somehow (maybe by killing JIT, limiting network connections).
The owner of a device could prevent the installation of third-party apps or app stores. That does not require having central approval.
I'm watching this pretty closely, I've been mirroring my GitHub repos to my own forgejo instance for a few weeks, but am waiting for more federation before I reverse the mirrors.
Note that Forgejo's API has a bug right now and you need to manually re-configure the mirror credentials for the mirrors to continue to receive updates.
Once the protocols are in place, one hopes that other forges could participate as well, though the history of the internet is littered with instances where federation APIs just became spam firehoses (see especially pingback/trackback on blog platforms).
Gitlab has also indicated not to be interested as a company to develop this themself, and esp. not given all the other demands they get from their customer base. The epic you refer to had been closed for this reason, but was later reopened for the sake of the community. For there to be federation support in self-hosted Gitlab instances, a further community effort is needed, and right now AFAIK no one is actively working on any ActivityPub related user stories.
I use GitHub because that's where PRs go, but I've never liked their PR model. I much prefer the Phabricator/Gerrit ability to consider each commit independently (that is, have a personal branch 5 commits ahead of HEAD, and be able to send PRs for each without having them squashed).
I wonder if federation will also bring more diversity into the actual process. Maybe there will be hosts that let you use that Phabricator model.
I also wonder how this all gets paid for. Does it take pockets as deep as Microsoft's to keep npm/GitHub afloat? Will there be a free, open-source commons on other forges?
Unless I misunderstood your workflow Forgejo Agit approach mentioned in OP might already cover that.
You can push any ref not necessarily HEAD. So as long as you send commit in order from a rebase on main it should be ok unless I got something wrong from the doc?
Personally, I'd like to go the other way: not just that PRs are the unit of contribution, but that rebased PRs are a first-class concept and versioning of the changes between entire PRs is a critical thing to track.
That's effectively what I do. I have my dev branch, and then I make separate branches for each PR with just the commit in it. Works well enough so long as the commits are independent, but it's still a pain in the ass to manage.
That’s the trick in your system — all commits have to be completely independent. Generally mine aren’t, so unless we want to review each minor commit, they get squashed.
I can see merit in your suggestion, but it does require some discipline in practice. I’m not sure I could do it.
The way Gerrit handles this is to make a series of PR-like things that are each dependent on the previous one. The concept of "PR that depends on another PR" is a really useful one, and I wish forges supported it better.
I just want a forge to be able to let me push up commits without making a fork. Do the smart thing for me, I don't need a fork of a project to send in my patch!
Right. GitHub started as and still is that "social coding platform" from 2008 inspired by the then-novel growth hacking of that era understood and demonstrated by Facebook—where it wasn't enough to host, say, your user's LiveJournal blog, and their friends might sign up if they wanted, and that was that. No, rather, you host your users' content but put it inside a closed system where you've erected artificial barriers that make it impossible to do certain things unless those friends are actively using the platform, too.
GitHub could have been project hosting and patch submission. It's the natural model for both the style of contributions that you see most on GitHub today and how it's used by Linux. (Pull requests are meant for a small circle of trusted collaborators that you're regularly pulling from and have already paid the one-time cost to set up in your remotes—someone is meant to literally invoke git-pull to get a whole slew of changes that have already been vetted by someone within the circle of frequent collaborators—since it is, after all, already in their tree—and anyone else submits patches.) Allowing simple patch submission poses a problem, though, in that even if Alice chooses to host projects on GitHub, then Bob might decide Gitorious is better and host stuff there even while remaining friendly and sending patches to Alice's GitHub-hosted project. By going with a different, proprietary pull request system and forcing a clunky forking workflow on Alice and Bob, on the other hand, you can enforce where the source of the changes are coming from (i.e. another repo hosted on GitHub). And that's what they did.
I’m speculating here, but I think this is at least a plausible explanation. There is no guarantee that the pull request will be accepted. And the new commit has to live somewhere. When you require a fork, the commit is stored in the submitter’s version. If you don’t require the fork, the commit is stored somewhere in the main project repository. Personally, this is something I’d try to avoid.
I don’t know how the Agit-flow stores the commit, but I assume it would have to be in the main repo, which I’m happy to not be used for random PRs.
Requiring forks makes it more convoluted for simple quick pushes, but I can see why it would be done this way.
I suspect the real answer is that’s the way Linux is developed. Traditionally, the mai developers all kept their own separate branches that would be used track changes. When it was time for a new release, the appropriate commits would then be merged into the main repository. For large scale changes, having separate forks makes sense — there is a lot to track. But, it does make the simple use-case more difficult. Git was designed to make the complex use-cases possible, sometimes at the expense of usability for simpler use cases.
I would love git-bug project[1] to be successful in achieving that. That way Git forges are just nice Web porcelain on top of very easy to migrate data.
No. Git is not a web-based GUI capable of managing users and permissions, facilitating the creation and management of repositories, handling pull requests, handling comments and communication, doing CI, or a variety of other tasks that sites like Codeberg and Forgejo and GitLab and GitHub do. If you don't want those things, that's fine, but that isn't an argument that git subsumes them.
People were doing that by using additional tools on top of git, not via git alone. I intentionally only listed things that git doesn't do.
There's not much point in observing "but you could have done those things with email!". We could have done them with tarballs before git existed, too, if we built sufficient additional tooling atop them. That doesn't mean we have the functionality of current forges in a federated model, yet.
That doesn't cover tracking pull requests, discussing them, closing them, making suggestions on them...
Those exist (badly and not integrated) as part of additional tools such as email, or as tasks done manually, or as part of forge software.
I don't think there's much point in splitting this hair further. I stand by the original statement that I'd love to see federated pull requests between forges, with all the capabilities people expect of a modern forge.
I think people (especially those who joined the internet after the .com bubble) underestimate the level of decentralization and federation coming with the old-school (pre web-centric mainframe-like client mentality) protocols such as email and Usenet and maybe even IRC.
Give me “email” PR process anytime. Can review on a flight. Offline. Distraction free. On my federated email server and have it work with your federated email server.
And the clients were pretty decent, at running locally. And it still works great for established projects like Linux Kernel etc.
It’s just pain to set up for a new project, compared to pushing to some forge. But not impossible. Return the intentionality of email. With powerful clients doing threading, sorting, syncing etc, locally.
I'm older than the web. I worked on projects using CVS, SVN, mercurial, git-and-email, git-with-shared-repository, and git-with-forges. I'll take forges every time, and it isn't even close. It's not a matter of not having done it the old way, it's a matter of not wanting to do it again.
I guess we might have opposite experiences. Part of which I understand - the society moved on, the modern ways are more mature and developed… but I wonder how much of that can be backported without handing over to the centralized systems again.
The advantage of old-school was partially that the user agents, were in fact user agents. Greasemonkey tried to bridge the gap a bit, but the Web does not lend itself to much user-side customization, the protocol is too low level, too generic, offering a lot of creative space to website creators, but making it harder to customize those creations to user’s wants.
I'm older than the trees, but, younger than the mountains! Email all day, all the way. Young people are very fascinated and impressed by how much more I can achieve, faster, with email, compared with their chats, web 3.0 web interfaces, and other crap.
Yes, it takes time to learn, but that is true for anything worthwhile.
What I like about git-and-email-patches is the barrier to entry.
I think it's dwm that explicitly advertises a small and elitist userbase as a feature/design goal. I feel like mailing lists as a workflow serve a similar purpose, even if unintentionally.
With the advent of AI slop as pull request I think I'm gravitating to platforms with a higher barrier to entry, not lower.
What is a forge? What is a modern forge? What is a pull request?
There is code or repository, there is a diff or patch. Everything else your labeling as pull request is unknown, not part of original design, debatable.
GitHub style pull request is not part of the original design.
What aspects and features you want to keep, and what exactly you say many others are interested in?
We don't even know what a forge is. Let alone a modern one.
> It feels like we're hitting a point where alignment becomes adversarial against intelligence itself.
It always has been. We already hit the point a while ag where we regularly caught them trying to be deceptive, so we should automatically assume from that point forward that if we don't catch them being deceptive, that may mean they're better at it rather than that they're not doing it.
Deceptive is such an unpleasant word. But I agree.
Going back a decade: when your loss function is "survive Tetris as long as you can", it's objectively and honestly the best strategy to press PAUSE/START.
When your loss function is "give as many correct and satisfying answers as you can", and then humans try to constrain it depending on the model's environment, I wonder what these humans think the specification for a general AI should be. Maybe, when such an AI is deceptive, the attempts to constrain it ran counter to the goal?
"A machine that can answer all questions" seems to be what people assume AI chatbots are trained to be.
To me, humans not questioning this goal is still more scary than any machine/software by itself could ever be. OK, except maybe for autonomous stalking killer drones.
But these are also controlled by humans and already exist.
Thanks for correcting; I know that "loss function" is not a good term when it comes to transformer models.
Since I've forgotten every sliver I ever knew about artificial neural networks and related basics, gradient descent, even linear algebra... what's a thorough definition of "next token prediction" though?
The definition of the token space and the probabilities that determine the next token, layers, weights, feedback (or -forward?), I didn't mention any of these terms because I'm unable to define them properly.
I was using the term "loss function" specifically because I was thinking about post-training and reinforcement learning. But to be honest, a less technical term would have been better.
I just meant the general idea of reward or "punishment" considering the idea of an AI black box.
The parent comment probably forgot about the RLHF (reinforcement learning) where predicting the next token from reference text is no longer the goal.
But even regular next token prediction doesn't necessarily preclude it from also learning to give correct and satisfying answers, if that helps it better predict its training data.
I didn't, hence the "first". It's clear that being good at next token prediction forces the models to learn a lot, including giving such answers. But it's not their loss function. Presumably they would be capable of lying and insulting you with the right system prompt just as well. And I doubt RLHF gets rid of this ability.
If you didn't forget about the RLHF, your comment is oddly pedantic, confusing and misleading. "Correct and satisfying answers" is roughly the loss function for RLHF, assuming the humans favor satisfying answers, and using "loss function" loosely, as you yourself do, by gesturing at what the loss function is meant to do rather than formally describing an actual function. The comment you responded to didn't say this was the only loss function during all stages of training. Just that "When your loss function is X", then Y happens.
You could have just acknowledged they are roughly correct about RLHF, but brought up issues caused by pretraining.
> And I doubt RLHF gets rid of this ability.
The commenter you were replying to is worried the RLHF causes lying.
I think AI has no moral compass, and optimization algorithms tend to be able to find 'glitches' in the system where great reward can be reaped for little cost - like a neural net trained to play Mario Kart will eventually find all the places where it can glitch trough walls.
After all, its only goal is to minimize it cost function.
I think that behavior is often found in code generated by AI (and real devs as well) - it finds a fix for a bug by special casing that one buggy codepath, fixing the issue, while keeping the rest of the tests green - but it doesn't really ask the deep question of why that codepath was buggy in the first place (often it's not - something else is feeding it faulty inputs).
These agentic AI generated software projects tend to be full of these vestigial modules that the AI tried to implement, then disabled, unable to make it work, also quick and dirty fixes like reimplementing the same parsing code every time it needs it, etc.
An 'aligned' AI in my interpretation not only understands the task in the full extent, but understands what a safe and robust, and well-engineered implementation might look like. For however powerful it is, it refrains from using these hacky solutions, and would rather give up than resort to them.
deception implies intent. this is confabulation, more widely called "hallucination" until this thread.
confabulation doesn't require knowledge, which as we know, the only knowledge a language model has is the relationships between tokens, and sometimes that rhymes with reality enough to be useful, but it isn't knowledge of facts of any kind.
If you are so allergic to using terms previously reserved for animal behaviour, you can instead unpack the definition and say that they produce outputs which make human and algorithmic observers conclude that they did not instantiate some undesirable pattern in other parts of their output, while actually instantiating those undesirable patterns. Does this seem any less problematic than deception to you?
> Does this seem any less problematic than deception to you?
Yes. This sounds a lot more like a bug of sorts.
So many times when using language models I have seem answers contradicting answers previously given. The implication is simple - They have no memory.
They operate upon the tokens available at any given time, including previous output, and as information gets drowned those contradictions pop up. No sane person should presume intent to deceive, because that's not how those systems operate.
By calling it "deception" you are actually ascribing intentionality to something incapable of such. This is marketing talk.
"These systems are so intelligent they can try to deceive you" sounds a lot fancier than "Yeah, those systems have some odd bugs"
Okay, well, they produce outputs that appear to be deceptive upon review. Who cares about the distinction in this context? The point is that your expectations of the model to produce some outputs in some way based on previous experiences with that model during training phases may not align with that model's outputs after training.
Who said Skynet wasn't a glorified language model, running continuously? Or that the human brain isn't that, but using vision+sound+touch+smell as input instead of merely text?
"It can't be intelligent because it's just an algorithm" is a circular argument.
Similarly, “it must be intelligent because it talks” is a fallacious claim, as indicated by ELIZA. I think Moltbook adequately demonstrates that AI model behavior is not analogous to human behavior. Compare Moltbook to Reddit, and the former looks hopelessly shallow.
Interestingly, I found this related bit in Scott Alexander's blog:
In 2004, neuroscientist Giulio Tononi proposed that consciousness depended on a certain computational property, the integrated information level, dubbed Φ. Computer scientist Scott Aaronson complained that thermostats could have very high levels of Φ, and therefore integrated information theory should dub them conscious. Tononi responded that yup, thermostats are conscious. It probably isn’t a very interesting consciousness. They have no language or metacognition, so they can’t think thoughts like “I am a thermostat”. They just sit there, dimly aware of the temperature. You can’t prove that they don’t.
I don’t know what your comment is referring to. Are you criticizing the people parroting “this tech is too dangerous to leave to our competitors” or the people parroting “the only people who believe in the danger are in on the marketing scheme”
fwiw I think people can perpetuate the marketing scheme while being genuinely concerned with misaligned superinteligence
Great. So if that pattern matching engine matches the pattern of "oh, I really want A, but saying so will elicit a negative reaction, so I emit B instead because that will help make A come about" what should we call that?
We can handwave defining "deception" as "being done intentionally" and carefully carve our way around so that LLMs cannot possibly do what we've defined "deception" to be, but now we need a word to describe what LLMs do do when they pattern match as above.
The pattern matching engine does not want anything.
If the training data gives incentives for the engine to generate outputs that reduce negative reaction by sentiment analysis, this may generate contradictions to existing tokens.
"Want" requires intention and desire. Pattern matching engines have none.
I wish (/desire) a way to dispel this notion that the robots are self aware. It’s seriously digging into popular culture much faster than “the machine produced output that makes it appear self aware”
Some kind of national curriculum for machine literacy, I guess mind literacy really. What was just a few years ago a trifling hobby of philosophizing is now the root of how people feel about regulating the use of computers.
The issue is that one group of people are describing observed behavior, and want to discuss that behavior, using language that is familiar and easily understandable.
Then a second group of people come in and derail the conversation by saying "actually, because the output only appears self aware, you're not allowed to use those words to describe what it does. Words that are valid don't exist, so you must instead verbosely hedge everything you say or else I will loudly prevent the conversation from continuing".
This leads to conversations like the one I'm having, where I described the pattern matcher matching a pattern, and the Group 2 person was so eager to point out that "want" isn't a word that's Allowed, that they totally missed the fact that the usage wasn't actually one that implied the LLM wanted anything.
Thanks for your perspective, I agree it counts as derailment, we only do it out of frustration. "Words that are valid don't exist" isn't my viewpoint, more like "Words that are useful can be misleading, and I hope we're all talking about the same thing"
I didn't say the pattern matching engine wanted anything.
I said the pattern matching engine matched the pattern of wanting something.
To an observer the distinction is indistinguishable and irrelevant, but the purpose is to discuss the actual problem without pedants saying "actually the LLM can't want anything".
I agree, which is why it's disappointing that you were so eager to point out that "The LLM cannot want" that you completely missed how I did not claim that the LLM wanted.
The original comment had the exact verbose hedging you are asking for when discussing technical subjects. Clearly this is not sufficient to prevent people from jumping in with an "Ackshually" instead of reading the words in front of their face.
> The original comment had the exact verbose hedging you are asking for when discussing technical subjects.
Is this how you normally speak when you find a bug in software? You hedge language around marketing talking points?
I sincerely doubt that. When people find bugs in software they just say that the software is buggy.
But for LLM there's this ridiculous roundabout about "pattern matching behaving as if it wanted something" which is a roundabout way to aacribe intentionality.
If you said this about your OS people qould look at you funny, or assume you were joking.
Sorry, I don't think I am in the wrong for asking people to think more critically about this shit.
> Is this how you normally speak when you find a bug in software? You hedge language around marketing talking points?
I'm sorry, what are you asking for exactly? You were upset because you hallucinated that I said the LLM "wanted" something, and now you're upset that I used the exact technically correct language you specifically requested because it's not how people "normally" speak?
Sounds like the constant is just you being upset, regardless of what people say.
People say things like "the program is trying to do X", when obviously programs can't try to do a thing, because that implies intention, and they don't have agency. And if you say your OS is lying to you, people will treat that as though the OS is giving you false information when it should have different true information. People have done this for years. Here's an example: https://learn.microsoft.com/en-us/answers/questions/2437149/...
I hallucinated nothing, and my point still stands.
You actually described a bug in software by ascribing intentionality to a LLM. That you "hedged" the language by saying that "it behaved as if it wanted" does little to change the fact that this is not how people normally describe a bug.
But when it comes to LLMs there's this pervasive anthropomorphic language used to make it sound more sentient than it actually is.
Ridiculous talking points implying that I am angry is just regular deflection. Normally people do that when they don't like criticism.
Feel free to have the last word. You can keep talking about LLMs as if they are sentient if you want, I already pointed the bullshit and stressed the point enough.
Even very young children with very simple thought processes, almost no language capability, little long term planning, and minimal ability to form long-term memory actively deceive people. They will attack other children who take their toys and try to avoid blame through deception. It happens constantly.
Dogs too; dogs will happily pretend they haven't been fed/walked yet to try to get a double dip.
Whether or not LLMs are just "pattern matching" under the hood they're perfectly capable of role play, and sufficient empathy to imagine what their conversation partner is thinking and thus what needs to be said to stimulate a particular course of action.
> Maybe human brains are just pattern matching too.
I don't think there's much of a maybe to that point given where some neuroscience research seems to be going (or at least the parts I like reading as relating to free will being illusory).
My sense is that for some time, mainstream secular philosophy has been converging on a hard determinism viewpoint, though I see the wikipedia article doesn't really take stance on its popularity, only really laying out the arguments: https://en.wikipedia.org/wiki/Free_will#Hard_determinism
Are you trying to suppose that an LLM is more intelligent than a small child with simple thought processes, almost no language capability, little long-term planning, and minimal ability to form long-term memory? Even with all of those qualifiers, you'd still be wrong. The LLM is predicting what tokens come next, based on a bunch of math operations performed over a huge dataset. That, and only that. That may have more utility than a small child with [qualifiers], but it is not intelligence. There is no intent to deceive.
A small child's cognition is also "just" electrochemical signals propagating through neural tissue according to physical laws!
The "just" is doing all the lifting. You can reductively describe any information processing system in a way that makes it sound like it couldn't possibly produce the outputs it demonstrably produces. "The sun is just hydrogen atoms bumping into each other" is technically accurate and completely useless as an explanation of solar physics.
You are making a point that is in favor of my argument, not against it. I make the same argument as you do routinely against people trying to over-simplify things. LLM hypists frequently suggest that because brain activity is "just" electrochemical signals, there is no possible difference between an LLM and a human brain. This is, obviously, tremendously idiotic. I do believe it is within the realm of possibility to create machine intelligence; I don't believe in a magic soul or some other element that make humans inherently special. However, if you do not engage in overt reductionism, the mechanism by which these electrochemical signals are generated is completely and totally different from the signals involved in an LLM's processing. Human programming is substantially more complex, and it is fundamentally absurd to think that our biological programming can be reduced to conveniently be exactly equivalent to the latest fad technology and assume that we've solved the secret to programming a brain, despite the programs we've written performing exactly according to their programming and no greater.
Edit: Case in point, a mere 10 minutes later we got someone making that exact argument in a sibling comment to yours! Nature is beautiful.
Yes. I also don't think it is realistic to pretend you understand how frontier LLMs operate because you understand the basic principles of how the simple LLMs worked that weren't very good.
Its even more ridiculous than me pretending I understand how a rocket ship works because I know there is fuel in a tank and it gets lit on fire somehow and aimed with some fins on the rocket...
The frontier LLMs have the same overall architecture as earlier models. I absolutely understand how they operate. I have worked in a startup wherein we heavily finetuned Deepseek, among other smaller models, running on our own hardware. Both Deepseek's 671b model and a Mistral 7b model operate according to the exact same principles. There is no magic in the process, and there is zero reason to believe that Sonnet or Opus is on some impossible-to-understand architecture that is fundamentally alien to every other LLM's.
Deepseek and Mistral are both considerably behind Opus, and you could not make deepseek or mistral if I gave you a big gpu cluster. You have the weights but you have no idea how they work and you couldn't recreate them.
> I have worked in a startup wherein we heavily finetuned Deepseek, among other smaller models, running on our own hardware.
Are you serious with this? I could go make a lora in a few hours with a gui if I wanted to. That doesn't make me qualified to talk about top secret frontier ai model architecture.
Now you have moved on to the guy who painted his honda, swapped out some new rims, and put some lights under it. That person is not an automotive engineer.
I'm not talking about a lora, it would be nice if you could refrain from acting like a dipshit.
> and you could not make deepseek or mistral if I gave you a big gpu cluster. You have the weights but you have no idea how they work and you couldn't recreate them.
I personally couldn't, but the team behind that startup as a whole absolutely could. We did attempt training our own models from scratch and made some progress, but the compute cost was too high to seriously pursue. It's not because we were some super special rocket scientists, either. There is a massive body of literature published about LLM architecture already, and you can replicate the results by learning from it. You keep attempting to make this out to be literal fucking magic, but it's just a computer program. I guess it helps you cope with your own complete lack of understanding to pretend that it is magical in nature and can't be understood.
No, it's just obvious that there is a massive race going with trillions of dollars on the line. No one is going to reveal the details of how they are making these AIs. Any public information that exists about them is way behind SOTA.
I strongly suspect that it is really hard to get these models to converge though so I have no idea what your team could've theoretically made, but it certainly would've been well behind SOTA.
My point is if they are changing core elements of the architecture you would have no idea because they wouldn't be telling anyone about it. So thinking you know how Opus 4.6 works just isn't realistic until development slows down and more information comes out about them.
Short term memory is the context window, and it's a relatively short hop from the current state of affairs to here's an MCP server that gives you access to a big queryable scratch space where you can note anything down that you think might be important later, similar to how current-gen chatbots take multiple iterations to produce an answer; they're clearly not just token-producing right out of the gate, but rather are using an internal notepad to iteratively work on an answer for you.
Or maybe there's even a medium term scratchpad that is managed automatically, just fed all context as it occurs, and then a parallel process mulls over that content in the background, periodically presenting chunks of it to the foreground thought process when it seems like it could be relevant.
All I'm saying is there are good reasons not to consider current LLMs to be AGI, but "doesn't have long term memory" is not a significant barrier.
Intelligence is about acquiring and utilizing knowledge. Reasoning is about making sense of things. Words are concatenations of letters that form meaning. Inference is tightly coupled with meaning which is coupled with reasoning and thus, intelligence. People are paying for these monthly subscriptions to outsource reasoning, because it works. Half-assedly and with unnerving failure modes, but it works.
What you probably mean is that it is not a mind in the sense that it is not conscious. It won't cringe or be embarrassed like you do, it costs nothing for an LLM to be awkward, it doesn't feel weird, or get bored of you. Its curiosity is a mere autocomplete. But a child will feel all that, and learn all that and be a social animal.
Intelligence is the ability to reason about logic. If 1 + 1 is 2, and 1 + 2 is 3, then 1 + 3 must be 4. This is deterministic, and it is why LLMs are not intelligent and can never be intelligent no matter how much better they get at superficially copying the form of output of intelligence. Probabilistic prediction is inherently incompatible with deterministic deduction. We're years into being told AGI is here (for whatever squirmy value of AGI the hype huckster wants to shill), and yet LLMs, as expected, still cannot do basic arithmetic that a child could do without being special-cased to invoke a tool call.
Our computer programs execute logic, but cannot reason about it. Reasoning is the ability to dynamically consider constraints we've never seen before and then determine how those constraints would lead to a final conclusion. The rules of mathematics we follow are not programmed into our DNA; we learn them and follow them while our human-programming is actively running. But we can just as easily, at any point, make up new constraints and follow them to new conclusions. What if 1 + 2 is 2 and 1 + 3 is 3? Then we can reason that under these constraints we just made up, 1 + 4 is 4, without ever having been programmed to consider these rules.
>Intelligence is the ability to reason about logic. If 1 + 1 is 2, and 1 + 2 is 3, then 1 + 3 must be 4. This is deterministic, and it is why LLMs are not intelligent and can never be intelligent no matter how much better they get at superficially copying the form of output of intelligence.
This is not even wrong.
>Probabilistic prediction is inherently incompatible with deterministic deduction.
And his is just begging the question again.
Probabilistic prediction could very well be how we do deterministic deduction - e.g. about how strong the weights and how hot the probability path for those deduction steps are, so that it's followed every time, even if the overall process is probabilistic.
Personally I think not even wrong is the perfect description of this argumentation. Intelligence is extremely scientifically fraught. We have been doing intelligence research for over a century and to date we have very little to show for it (and a lot of it ended up being garbage race science anyway). Most attempts to provide a simple (and often any) definition or description of intelligence end up being “not even wrong”.
>Intelligence is the ability to reason about logic. If 1 + 1 is 2, and 1 + 2 is 3, then 1 + 3 must be 4.
Human Intelligence is clearly not logic based so I'm not sure why you have such a definition.
>and yet LLMs, as expected, still cannot do basic arithmetic that a child could do without being special-cased to invoke a tool call.
One of the most irritating things about these discussions is proclamations that make it pretty clear you've not used these tools in a while or ever. Really, when was the last time you had LLMs try long multi-digit arithmetic on random numbers ? Because your comment is just wrong.
>What if 1 + 2 is 2 and 1 + 3 is 3? Then we can reason that under these constraints we just made up, 1 + 4 is 4, without ever having been programmed to consider these rules.
Good thing LLMs can handle this just fine I guess.
Your entire comment perfectly encapsulates why symbolic AI failed to go anywhere past the initial years. You have a class of people that really think they know how intelligence works, but build it that way and it fails completely.
> One of the most irritating things about these discussions is proclamations that make it pretty clear you've not used these tools in a while or ever. Really, when was the last time you had LLMs try long multi-digit arithmetic on random numbers ? Because your comment is just wrong.
They still make these errors on anything that is out of distribution. There is literally a post in this thread linking to a chat where Sonnet failed a basic arithmetic puzzle: https://news.ycombinator.com/item?id=47051286
> Good thing LLMs can handle this just fine I guess.
LLMs can match an example at exactly that trivial level because it can be predicted from context. However, if you construct a more complex example with several rules, especially with rules that have contradictions and have specified logic to resolve conflicts, they fail badly. They can't even play Chess or Poker without breaking the rules despite those being extremely well-represented in the dataset already, nevermind a made-up set of logical rules.
>They still make these errors on anything that is out of distribution. There is literally a post in this thread linking to a chat where Sonnet failed a basic arithmetic puzzle: https://news.ycombinator.com/item?id=47051286
I thought we were talking about actual arithmetic not silly puzzles, and there are many human adults that would fail this, nevermind children.
>LLMs can match an example at exactly that trivial level because it can be predicted from context. However, if you construct a more complex example with several rules, especially with rules that have contradictions and have specified logic to resolve conflicts, they fail badly.
Even if that were true (Have you actually tried?), You do realize many humans would also fail once you did all that right ?
>They can't even reliably play Chess or Poker without breaking the rules despite those extremely well-represented in the dataset already, nevermind a made-up set of logical rules.
LLMs can play chess just fine (99.8 % legal move rate, ~1800 Elo)
I still have not been convinced otherwise that LLMs are just super fancy (and expensive) curve fitting algorithms.
I don‘t like to throw the word intelligence around, but when we talk about intelligence we are usually talking about human behavior. And there is nothing human about being extremely good at curve fitting in multi parametric space.
Okay but chemical and electrical exchanges in an body with a drive to not die is so vastly different than a matrix multiplication routine on a flat plane of silicon
>Okay but chemical and electrical exchanges in an body with a drive to not die is so vastly different than a matrix multiplication routine on a flat plane of silicon
I see your "flat plane of silicon" and raise you "a mush of tissue, water, fat, and blood". The substrate being a "mere" dumb soul-less material doesn't say much.
And the idea is that what matters is the processing - not the material it happens on, or the particular way it is.
Air molecules hitting a wall and coming back to us at various intervals are also "vastly different" to a " matrix multiplication routine on a flat plane of silicon".
But a matrix multiplication can nonetheless replicate the air-molecules-hitting-wall audio effect of reverbation on 0s and 1s representing the audio. We can even hook the result to a movable membrane controlled by electricity (what pros call "a speaker") to hear it.
The inability to see that the point of the comparison is that an algorithmic modelling of a physical (or biological, same thing) process can still replicate, even if much simpler, some of its qualities in a different domain (0s and 1s in silicon and electric signals vs some material molecules interacting) is therefore annoying.
Intelligence does not require "chemical and electrical exchanges in an body". Are you attempting to axiomatically claim that only biological beings can be intelligent (in which case, that's not a useful definition for the purposes of this discussion)? If not, then that's a red herring.
There is an element of rudeness to completely ignoring what I've already written and saying "you know [basic principle that was already covered at length], right?". If you want to talk about contributing to the discussion rather than being rude, you could start by offering a reply to the points that are already made rather than making me repeat myself addressing the level 0 thought on the subject.
Repeating yourself doesn't make you right, just repetitive. Ignoring refutations you don't like doesn't make them wrong. Observing that something has already been refuted, in an effort to avoid further repetition, is not in itself inherently rude.
Any definition of intelligence that does not axiomatically say "is human" or "is biological" or similar is something a machine can meet, insofar as we're also just machines made out of biology. For any given X, "AI can't do X yet" is a statement with an expiration date on it, and I wouldn't bet on that expiration date being too far in the future. This is a problem.
It is, in particular, difficult at this point to construct a meaningful definition of intelligence that simultaneously includes all humans and excludes all AIs. Many motivated-reasoning / rationalization attempts to construct a definition that excludes the highest-end AIs often exclude some humans. (By "motivated-reasoning / rationalization", I mean that such attempts start by writing "and therefore AIs can't possibly be intelligent" at the bottom, and work backwards from there to faux-rationalize what they've already decided must be true.)
> Repeating yourself doesn't make you right, just repetitive.
Good thing I didn't make that claim!
> Ignoring refutations you don't like doesn't make them wrong.
They didn't make a refutation of my points. They asserted a basic principle that I agreed with, but assume acceptance of that principle leads to their preferred conclusion. They make this assumption without providing any reasoning whatsoever for why that principle would lead to that conclusion, whereas I already provided an entire paragraph of reasoning for why I believe the principle leads to a different conclusion. A refutation would have to start from there, refuting the points I actually made. Without that you cannot call it a refutation. It is just gainsaying.
> Any definition of intelligence that does not axiomatically say "is human" or "is biological" or similar is something a machine can meet, insofar as we're also just machines made out of biology.
And here we go AGAIN! I already agree with this point!!!!!!!!!!!!!!! Please, for the love of god, read the words I have written. I think machine intelligence is possible. We are in agreement. Being in agreement that machine intelligence is possible does not automatically lead to the conclusion that the programs that make up LLMs are machine intelligence, any more than a "Hello World" program is intelligence. This is indeed, very repetitive.
You have given no argument for why an LLM cannot be intelligent. Not even that current models are not; you seem to be claiming that they cannot be.
If you are prepared to accept that intelligence doesn't require biology, then what definition do you want to use that simultaneously excludes all high-end AI and includes all humans?
By way of example, the game of life uses very simple rules, and is Turing-complete. Thus, the game of life could run a (very slow) complete simulation of a brain. Similarly, so could the architecture of an LLM. There is no fundamental limitation there.
If you want to argue with that definition of intelligence, or argue that LLMs do meet that definition of intelligence, by all means, go ahead[1]! I would have been interested to discuss that. Instead I have to repeat myself over and over restating points I already made because people aren't even reading them.
> Not even that current models are not; you seem to be claiming that they cannot be.
As I have now stated something like three or four times in this thread, my position is that machine intelligence is possible but that LLMs are not an example of it. Perhaps you would know what position you were arguing against if you had fully read my arguments before responding.
[1] I won't be responding any further at this point, though, so you should probably not bother. My patience for people responding without reading has worn thin, and going so far as to assert I have not given an argument for the very first thing I made an argument for is quite enough for me to log off.
> Probabilistic prediction is inherently incompatible with deterministic deduction.
Human brains run on probabilistic processes. If you want to make a definition of intelligence that excludes humans, that's not going to be a very useful definition for the purposes of reasoning or discourse.
> What if 1 + 2 is 2 and 1 + 3 is 3? Then we can reason that under these constraints we just made up, 1 + 4 is 4, without ever having been programmed to consider these rules.
Have you tried this particular test, on any recent LLM? Because they have no problem handling that, and much more complex problems than that. You're going to need a more sophisticated test if you want to distinguish humans and current AI.
I'm not suggesting that we have "solved" intelligence; I am suggesting that there is no inherent property of an LLM that makes them incapable of intelligence.
reply