I'm noticing terms related to DL/RL/NLP are being used more and more informally as AI takes over more of the cultural zeitgeist and people want to use the fancy new terms of the era, even if inaccurately. A friend told me he "trained and fine tuned a custom agent" for his work when what he meant was he modified a claude.md file.
Respectfully, your friend doesn't know what he is talking about and is saying things that just "feel right" (vibe talking??). Which might be exactly how technical terms lose their meaning so perhaps you're exactly right.
There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model.
And with that comes reward hacking - which isn't really about looking for more reward but rather that the model has learned patterns of behavior that got reward in the train env.
That is, any kind of vulnerability in the train env manifests as something you'd recognize as reward hacking in the real world: making tests pass _no matter what_ (because the train env rewarded that behavior), being wildly sycophantic (because the human evaluators rewarded that behavior), etc.
> There is a nontrivial amount of RL training (RLHF, RLVR, ...), so it would be reasonable to call it an RL model.
Hm, as i understand it, parts of the training of e.g. ChatGPT could be called RL models. But the subject to be trained/fine tuned is still a seq2seq next token predictor transformer neural net.
RL is simply a broad category of training methods. It's not really an architecture per se: modern GPTs are trained first on reconstruction objective on massive text corpora (the 'large language' part), then on various RL objectives +/- more post-training depending on which lab.
> Is it really about rewards? Im genuinely curious. Because its not a RL model.
Ha, good point. I was using it informally (you could handwave and call it an intrinsic reward if a model is well aligned to completing tasks as requested), but I hadn't really thought about it.
I think an humble and open mind is essential. I think that we reap what we sow, but also that struggle makes us robust.
I try to explain stuff to my kids, to the best of my ability, but give them room to make their own conclusions. As an old fart, there is a limit to how relevant my world will be to them - and I have to acknowledge that.
Change is scary and not always for the better, but in my humble opinion; we have nothing to lose and everything to gain.
> Furthermore, what does it matter if it's "AI generated"? Is some AI content ok? What's the pass/fail threshold on human vs AI generated text?
If a human put his effort into it, is proud of it and wants to show it to the world, i'm happy to invest some time to have a look at it and maybe provide some helpful feedback.
I'm not willing to invest my time into evaluating the more or less correct sounding ideas of a ML model.
> coming AI wasteland: motivated individuals join small local groups and are validated face-to-face at meet-ups. Local trusted leads gatekeep their chapter’s posts, and this scalable moderation works up the tree. Bad leaves get culled out reasonably fast,
> CasNum (Compass and straightedge Number) is a library that implements arbitrary precision arithmetic using compass and straightedge constructions. Arbitrary precision arithmetic, now with 100% more Euclid. Featuring a functional modified Game Boy emulator where every ALU opcode is implemented entirely through geometric constructions.
> The real question is what happens when the labor market for non-physical work completely implodes as AI eats it all. Based on current trends I'm going to predict in terms of economics and politics we handle it as poorly as possible leading to violent revolution and possible societal collapse, but I'd love to be wrong.
Exactly and the world has to start talking about it. Eventually everybody will, including all sorts of politicians who advocate to 'finally tackle the problem', which will be too late.
Is it really about rewards? Im genuinely curious. Because its not a RL model.
reply