LLM output doesn't have the variety of human output, since they operate in fixed fashion - statistical inference followed by formulaic sampling.
Additionally, the statistics used by LLMs are going be be similar across different LLMs since at scale its just "the statistics of the internet".
Human output has much more variety, partly because we're individuals with our own reading/writing histories (which we're drawing upon when writing), and partly because we're not so formulaic in the way we generate. Individuals have their own writing styles and vocabulary, and one can identify specific authors to a reasonable degree of accuracy based on this.
It's a bit like detecting cheating in a chess tournament. If an unusually high percentage of a player's moves are optimal computer moves, then there is a high likelihood that they were computer generated. Computers and humans don't pick moves in the same way, and humans don't have the computational power to always find "optimal" moves.
Similarly with the "AI detectors" used to detect if kids are using AI to write their homework essays, or to detect if blog posts are AI generated ... if an unusually high percentage of words are predictable by what came before (the way LLMs work), and if those statistics match that of an LLM, then there is an extremely high chance that it was written by an LLM.
Can you ever be 100% sure? Maybe not, but in reality human written text is never going to have such statistical regularity, and such an LLM statistical signature, that an AI detector gives it more than a 10-20% confidence of being AI, so when the detector says it's 80%+ confident something was AI generated, that effectively means 100%. There is of course also content that is part human part AI (human used LLM to fix up their writing), which may score somewhere in the middle.
> LLM output doesn't have the variety of human output, since they operate in fixed fashion - statistical inference followed by formulaic sampling.
This is the wrong thing to look at; your chess analogy is much stronger, the detection method similar (if you can figure out a prompt that generates something close to the content, it almost certainly isn't human origin).
But to why the thing I'm quoting doesn't work: If you took, say, web comic author Darren Gav Bleuel, put him in a sci-fi mass duplication incident make 950 million of him, and had them all talking and writing all over the internet, people would very quickly learn to recognise the style, which would have very little variety because they'd all be forks of the same person.
Indeed, LLMs are very good at presenting other styles than their defaults, better at this than most humans, and what gives away LLMs is that (1) very few people bother to ask them to act other than their defaults, and (2) all the different models, being trained in similar ways on similar data with similar architectures, are inherently similar to each other.
An LLM is just computer function that predicts next word based on the input you give it. It doesn't make any difference what the input is (e.g. please respond in style X) - the function doesn't change, and the statistical signature of how it works will still be there.
If you don't believe me, try it for yourself. Ask an AI to generate some text and give it to the AI detector below (paste your text, then click on scan). Now ask the AI to generate in a different style and see if it causes the detector to fail.
I can't use that linked app, paywall immediately. Unlike the person you were replying to here[0], I do not claim that this is impossible:
LLM is indeed just computer function that does stats. And our brains are just electro-chemistry that does stats. This is why stylometric analysis of human writing is a thing.
My previous experience with things such as you have linked to, is they used to be quite poor. I assume they're better since then, but then again so are the models.
> I assume they're better since then, but then again so are the models.
Yes, but "better" means different things for each of these.
Detectors are trying to get better at distinguishing human from LLM-generated text.
LLMs are being improved to generate more useful (and benchmark maxxing) outputs, not to attempt to avoid detection.
LLMs are in fact explicitly trained to be as predictable as possible. The training goal is to minimize continuation prediction errors, which means they are in effect being trained to generate output where each word can be predicted by what came before it (which we can contrast to a human who tries to spice it up and keep it interesting by not being too predictable!).
RL post-training, which is especially used for computer code and math, is going to change this word-by-word predictability (detectability) a bit since the focus is now on a longer term goal rather than next word, but to some extent you could also view it as just steering/narrowing the output of the model towards that goal, not totally overriding the next-word statistics.
I don't know if there are AI detectors specifically trained to detect AI code rather than prose, but I'd expect that is more difficult to do, both because of the RL factor, and because computer code is so predictable in the first place - adhering to rigid syntax etc.
A human can easily produce output that looks like anything an LLM can produce, therefor an LLM detector that can say "this is 100% written by AI" cannot exist. It's really that simple.
> Can you ever be 100% sure? Maybe not
The commenter I was replying to claimed exactly this. Their AI detector showed that the text was "100%" AI generated.
I was just expressing some caution. Saying you are 100% certain of anything when the evidence is statistical seems a bit too certain, especially if it was just from a short text sample.
Compare to flipping a coin, counting heads vs tails, and trying to assess if it's a fair or biased coin. After 1000 flips if it's not close to 50/50 you would rightfully be suspect, and if it was 10/90 you should be almost certain it's biased. But you can never be 100% sure.
What performance issues do you usually run into on the current iteration of the app? Just out of personal curiosity so I can take note next time I'm using the application.
Not who you asked, but I notice a lot of lag when I'm searching/navigating/etc in Spotify. I understand that it's fetching a lot of this data from their servers, but the UI itself is also laggy when I'm browing local files, scrolling through playlists, etc, so it's not just serverside.
Not to mention the SSD-destroying & CPU-hogging habits of the application. That being said, I still use Spotify because I really value the music discovery and algorithms... and nobody else offers a streaming service with a dark theme :)
Mostly it's millisecond lag when changing songs or opening up new playlists. I get that everything is on the cloud but maybe more aggressive caching would be helpful.
If it were any other application, I wouldn't care. But because this is the second most frequently piece of software I use (browser being first), I'd love for it to be as performant as possible.
OP here - this was a personal collection I had been keeping up to date and decided to make it public given the number of people that have asked me how to get started with VR.
There's a lot going on in the space so if I missed an important resource, would be happy to know.
But people like to pay for convenience, and this is convenient. You pay a monthly fee to receive a predetermined project, all the parts necessary to build it, an entire walk through equipped with videos/photo, and customer support.
Sure the price is inflated (for someone with the know-how) but for people interested in learning electronics, the all-in-one-delivered-monthly is a great sell until you grow out of it.
NotionTheory | http://notiontheory.com/ | Full Stack Engineer | Washington, DC | Remote - Full-Time
We’re a team of talented engineers helping startups deliver their web, mobile, wearable, virtual reality, and hardware products to market in record time. We’re looking to round out our troupe with a full stack developer who can continue to elevate the quality of our web and mobile products for clients.
The web stack typically consists of Ruby on Rails, postgreSQL, and heroku. For the mobile stack, we use Cordova, Ionic framework (built on angularjs), and a firebase or rails server for the backend depending on the project needs.
A deep love for javascript in either stack is a must and you should be comfortable using third party APIs such as stripe, google, twilio, pusher, etc.
Any interest/experience in wearables, virtual reality and hardware/robotics is a plus.
-----
The perks of working at NotionTheory:
- “Take The Time You Need” vacation policy
- “Flex Fridays” - every Friday we work on open source or internal company projects
- Frequent company trips, local events and team activities
NotionTheory | http://notiontheory.com/ | Full Stack Engineer | Washington, DC | Remote - Full-Time
We’re a team of talented engineers helping startups deliver their products to market in record time. We’re looking to round out our troupe with someone who can continue to elevate the quality of our work and relationships with our clients.
The web stack typically consists of Ruby on Rails, postgreSQL, and heroku. For the mobile stack, we use Cordova, Ionic framework (built on angularjs), and a firebase or rails server for the backend depending on the project needs.
A deep love for javascript in either stack is a must and you should be comfortable using third party APIs such as stripe, google, twilio, pusher, etc.
-----
The perks of working at NotionTheory:
- “Take The Time You Need” vacation policy
- “Flex Fridays” - every Friday we work on open source or internal company projects
- Frequent company trips, local events and team activities
We’re a team of talented engineers helping companies rapidly build and deliver their products to market in just 4 - 6 weeks. We're looking to round out our team with someone who can continue to help elevate the quality of our work and relationships with our clients.
As part of the core team, you’ll be expected to help play the role of stand-in CTO to our clients.
In addition to being a kickass developer, you should also:
- Have a track record of developing and delivering products to market (while maintaining quality)
- Have a sense of product ownership and bring innovative ideas to the table for our company and our clients
- Not be afraid to say “I don’t know”. You routinely figure things out.
- Be seriously committed to helping build a startup, even though things won’t always be easy.
- Have an insatiable thirst to always be learning and experimenting
Began a startup senior year of college (Thryv). Managed to build a team, get sales, generate revenue and score a large partnership. Long story short, I was a customer of my own product and assumed I knew what other customers wanted. I spent the next 6 months having a product built without validating my initial assumptions before launching. The product was too niche only fitting customers who had my specific workflow and I ultimately found this out too late when we ran out of steam/resources. Total timeline (including fuckups) was 3 years.
I was non-technical at my first startup (domain expert & business/marketing/sales) and after I shut it down, I spent the next 8 months teaching myself how to build software programs, becoming a now tech co-founder.
I then started my second startup (http://notiontheory.com/) by getting $20k pre-sales in 2 weeks to validate my idea before committing to it.
Attempt 2 has been far more successful and we're already beginning hiring in month 5 bootstrapped.
The process is the same for converting doge to USD (which I should make clear in the guide, thanks).
There are a few exchanges with plans to offer direct exchange from Doge to USD in the coming weeks, which I will also be making a guide for when it's released.
You could alternatively sell your Doge for USD through a service like eBay and add a nice markup for convenience (doge is selling for 2x the value in some bids), although you chance dealing with scammers.
edit* thanks for the props, specifically in comic sans :)
YW and thanks. I'm the person who asked on reddit about converting to JPY. A Japanese shibe and I are trying to help Atsuko Sato, the owner of the doge, learn how to convert dogecoin donations. Since your tutorial is real clear I'll point my Japanese friend to it to translate.
