Fair. I was trying to convey self hosted benefits without the downside of hosting your own MTA. I was also really in my head on the niche and assumed everyone else would get it from the snappy title. Not my intent to mislead.
BTW if you want to kick the tires at the current stage, building it locally is easy, just a couple of commands to install dependencies and run the build script:
IMHO: hard, inflexible rules like these are always deeply rooted in biases and personal convictions, not in facts. The suggested policy amendment by Claude at the end is much more honest, logical, and palatable.
This reminds me of something funny I noticed about AI. Let's say you ask it what it thinks of an email you just drafted. It will provide corrections.
Delete that session, and ask it about the corrected email. It will provide more corrections.
Repeat. It always provides more corrections. Sometimes returning the recommended email back to a previous state.
This is basically what's gonna happen when people argue-from-AI. It's the same cycle but because control is distributed the individuals participating can't see how stupidly pointless it is.
> The argument assumes that unassisted PR authorship is what builds trustworthy contributors, and that LLM assistance prevents that growth.
No, I don't think that was the argument. As I understood it, unassisted contributions have higher chances to grow a trusted contributor. Not 100% vs 0% chances, but statistically higher. So, given limited resources, it makes sense to prefer unassisted over assisted contributions.
I don't believe that even the weakened version of the argument works -- it is based on an assumption, not fact.
Why would a contributor that uses AI assistance have fewer chances to be trusted?
I'm not talking about AI slop, but a contributor that takes time to understand a problem, find a solution, and discuss pros/cons alternatives. Using LLM assistance, of course.
You could extend that argument to any tool used by the developer, like a linter, sanitizer, the IDE itself, or even auto-completion. Why target LLMs specifically?
The more I think about it, the more nonsensical it is.
- What if I do everything by hand, but have an LLM review my work at the very end?
- What if I have an LLM guide me through the codebase just by specifying the files I should read and in what order, but I do all the reading myself?
- What if I do everything by hand, but then use an LLM to optimize a small part of an algorithm?
You can easily see how absurd it is to completely ban LLMs.
What matters is the quality and correctness of the contribution. Even with heavy LLM usage, unless the developer understands what problem they're solving, the quality will be sub-par.
it's in the definition of the word. you can not determine what the LLM will do.
anything done with AI is a problem because it is essentially unpredictable. auto-complete is on the fence because you presumably are still able to pay attention that it completes what you want but it depends on how diligent you are when working and how much i trust your diligence.
You are still dodging the question -- what is the problem with not being able to determine in advance what the LLM will do?
Even then, you can clearly see that the LLM will try its best to follow the instructions. The result might not be 100% predictable, but it is somewhat predictable depending on the task.
After the LLM does what it has been asked, you can review it, iterate on it, test it, and so on. And if you're trying to make anything worthwhile, you will do so.
anything unpredictable is inherently untrustworthy and requires extra effort to review.
Lack of determinism is not a practical concern.
it is to me. it's a knockout criteria. it is the only reason that keeps me from using LLMs for coding. nothing else is as serious an issue to me as this.
here is why: i tell the LLM to build something with requirements A B C D and E. it builds, i review and i find A B and D are good, C and E are broken. i tell it to fix them, it does, so C and E are fixed, but now A is broken. i tell it to fix that, and i have to keep iterating until i find a combination where everything works. in every iteration any part can randomly break, so for every iteration i get changes all over the place. they never are confined to the issue i pointed out. i have to review the whole thing every time. that's what i mean by lack of determinism, and that is a serious practical concern because instead of getting done in two or three iterations it requires dozens of them. see my related replies elsewhere. i just don't want to work that way.
You'd have to review and verify even changes that you've written by hand. You might think that your hand-written code satisfies A+B+C+D+E, but until you've verified it, you cannot prove it.
That's not any different from LLM-assisted writing -- humans are inherently non-deterministic as well :)
The other fallacy is assuming that everyone else's experience with LLM-assisted writing is the same as yours. Personally, I've rarely encountered the issue you've mentioned -- most of my LLM-assisted coding has been a net positive and quite straightforward.
Perhaps it's the nature of the problem I'm working on, perhaps it's the model I chose, perhaps it's my prompting skills. It doesn't matter -- you just cannot assume that because something doesn't work for you it doesn't work for anyone else.
The other fallacy is considering LLM-assisted coding a binary option, like the nonsensical Zig policy does.
I agree with you that "vibe coding" something from scratch will likely result in poor quality and many iterations. But that's not the only way to use LLMs.
You can ask LLMs to review hand-written code.
You can ask LLMs to optimize a specific part of code.
You can ask LLMs to apply a specific refactor.
You can ask LLMs to brainstorm solutions to a problem.
You can ask LLMs to autocomplete patterns.
I could go on. This stuff works. It is helpful.
Assuming that everyone who uses LLMs is incompetent and preventing them from contributing because of a hunch or your own negative experiences is just asinine.
The other fallacy is assuming that everyone else's experience with LLM-assisted writing is the same as yours.
that's irrelevant. my choice can only be based on my experience. i am unable to verify your experience, because i am not you. we have different tolerances, and if it works for your project, then fine.
you just cannot assume that because something doesn't work for you it doesn't work for anyone else.
we are talking about contributions to my project. if LLM coding doesn't work for me, then your LLM created contributions won't work for me either because i won't trust them. you can't legislate or enforce trust. trust can only be earned. lack of trust means i have to spend more effort to verify your code.
Zig devs don't find LLMs to be net positive, what is so hard to understand? You can write your own compiler with LLM yourself, nobody is standing in your way.
I think you’re missing the underlying point. The Zig team is focused on the contributor and their relationship to the project, not on the correctness of the work. People, not product. Yes, an LLM can help you better understand your code and pick up on things you may have missed before you submit your change. But I think they look at it as you’ve then robbed the Zig team of that interaction with the contributor. They lost the opportunity to learn about how that person thinks, and that person lost the opportunity to be mentored and learn from other members of the Zig team. Sure, your code is better, but did you or the team grow from the experience or simply churn out more code?
I’m not saying whether or not that’s good or bad. I agree with their approach, but that’s just my opinion and who am I to say what’s right or wrong? I think there’s value to LLMs as a tool to search and learn, but I’m also worried that LLMs make it really easy to focus on only the result and not the process. That process can be really valuable in building good teams, while LLMs can be really good at churning out an assembly line of code.
My claim that LLMs can benefit the end product and create quality contributions does not imply that the person behind the contribution is less capable/creative/smart than someone who doesn't use LLMs.
But it seems that the Zig policy implies that. Otherwise what would be wrong with interacting with contributors using LLMs?
Now read the actual points made by the AI with an open mind and a critical mindset, instead of dismissing them because they were not written by a human being.
The point I'm making is that this policy is so stupid that even an LLM can easily figure out the logical flaws. Perhaps an LLM could have also helped you figure out the point of my original comment.
reply