When you press the report button, a confirmation message is shown explicitly saying that the last 5 messages will be sent to WhatsApp without the other user knowing. You can then accept or cancel.
You can also copy the whole conversation and send it to the police, or post them on a forum, etc.
WhatsApp moderators can't read your messages, but they can read what the person you talk to send them. E2E encryption means that only the ends should be able to decrypt and read the contents, but what they do with them is another different topic.
"AI" running on every client can automatically flag messages and send them to moderators.
> Most can agree that violent imagery and CSAM should be monitored and reported; Facebook and Pornhub regularly generate media scandals for not moderating enough. But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.
The problem here is that the third party controls the software on both ends of the communication. And that software can send the messages to this party without the participants knowingly triggering it.
The article says that by reporting a user, the software on the site of the reporting user silently sends data to WhatsApp. The reporting user does not know what data is sent.
The article quotes the terms of service which say that they send:
>"the most recent messages”
and
>“information on your recent interactions with the reported user.”
which is unclear, but not unknown, and as far as this article claims they don't actually send anything else, though they do combine it with whatever metadata they have on the users involved.
> Seated at computers in pods organized by work assignments, these hourly workers use special Facebook software to sift through streams of private messages, images and videos that have been reported by WhatsApp users as improper and then screened by the company’s artificial intelligence systems.
> Instead, WhatsApp reviewers gain access to private content when users hit the “report” button on the app, identifying a message as allegedly violating the platform’s terms of service. This forwards five messages — the allegedly offending one along with the four previous ones in the exchange, including any images or videos — to WhatsApp in unscrambled form, according to former WhatsApp engineers and moderators. Automated systems then feed these tickets into “reactive” queues for contract workers to assess.
From the actual ProPublica report. If their published understanding is correct, E2EE is not broken, but rather end users who are one of the ends of E2EE are sending the decrypted content to be moderated. The AI bit is a filter to reduce the amount of content passed on to human moderators.
From near that second quote:
> Artificial intelligence initiates a second set of queues — so-called proactive ones — by scanning unencrypted data that WhatsApp collects about its users and comparing it against suspicious account information and messaging patterns (a new account rapidly sending out a high volume of chats is evidence of spam), as well as terms and images that have previously been deemed abusive.
That part is AI driven, but my reading is that the moderators do not get access to the encrypted data (the actual messages) only the behavior patterns, and from that make a determination of what to do.
It sounds like the only unencrypted data that the moderators see is sent from an endpoint (a user clicking "report"). After that an AI looks at the report and prioritizes ones that looks like it might be CSAM.
> But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.
It's not just when a recipient reports them it seems but also when they have been flagged by their algorithm. If that were true, the claim that the conversation is e2e encrypted simply cannot be true, unless the algorithm runs on the client.
just think about the sheer number of people that report stuff.
given that facebook has less than 1k moderators, do you honestly think that they'd just let the moderators sift through everything manually?
obviously you'd classify stuff first, checking against known images is easy. Classifying new images is a lot harder, plus the ethics of training and labelling a dataset for accurate detection is pretty hard, also almost impossible to do legally.
I suspect the next best thing is detecting nudity and age of the subject, and taking the hit that you're going to prioritise a lot of malicious reports, rather than genuine.
I don't feel like it's unreasonable for a company to have a system where a user can say "Hey someone is sending me something unwanted using your service" and for that company to use technology to sort those complaints for humans to review and action appropriately based on their terms of service.
I read this as a user, whom has received and decrypted an e2ee message, can report it. This will send it, e2ee, to WhatsApp HQ. no breakage here. E2ee is no panacea.
Boggles my mind that people see this as Facebook reading their encrypted messages. The person you sent the message to chose to forward the message to Facebook and are even told that this is happening when they report the message.
You can't stop someone who knows what a message says from sharing the contents of that message without physically restraining and gagging them. They can send a screenshot out, take a photo of the screen, write down the message on paper, or just memorize it and tell others.
Boggles mine that people believe proprietary platforms blindly. It can't be proved that there's no backdoor added after the initial consulting for OpenWhisper.
Gee, seeing how they defended Whatsapp tooth and nail time and time again, I'm inclined to not be surprised if it was added _during_.
Whether or not they're siphoning information some other way is not the point here. The accusations here are directed at how reporting content sends the decrypted messages to Facebook. The ridiculousness is that people are treating that reporting functionality as a secret data leakage channel.
If you think that they have backdoor access to messages, please investigate that, submit your findings as a different HN post, and then we can discuss it.
Exactly. Your e2e message is encrypted from your end to the recipents end. They then decrypt the message and forward it to someone else.
Nothing new here. I really dont get what the expectation of these things are for people. Are they not aware, someone can forward a message to a thrid party, people can and do screenshot text and forward it. Writing a message, and sending it encrypted to someone else doesnt protect you from what they might do with the message they now have.
> But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.
It seems like whatever the AI decides is questionable gets sent to moderators automatically, also revealing the thread history (at least partially) as well.
THose who are claiming that they are running Machine learning at the edge, think about this:
how on earth can they trust the data coming from it?
We all know that ML is hard, we also know that to get accurate classification requires serious horsepower.
None of this can be run on phone hardware without people noticing.
If you were to design a system for handling reported data, who's hard constraint is _human_ eyeball time, would you run ML at the edge? No.
You'd make it so the "report this message" sent the last n lines and the attachment, and then run the ML optimised for accuracy (not speed or memory) on your hardware that you control and trust explicitly.
I would have thought all of this is obvious, given that facebook is all about avoiding spending money on human moderators. Edge ML is not going to help you do that.
When you need moderators in private chat something is messed up socially and technologically.
If you and I are texting, privately, and you say something to me I don't like, I'll tell you., or I will stop talking to you. That's healthy social interaction. If I report it instead, that's not healthy social interaction, and it appears to me the tech companies designing these solutions to problems that were solved a decade ago are more than happy to oblige because it gives them an excuse to surveil and censor. They're encouraging unhealthy social interaction because it serves their ends to do so.
If an instant messaging application has moderators it's not an instant messaging application.
2 billion people use Whatsapp, some of them for purposes that are not as pedestrian as yours.
Private Whatsapp groups can include thousands of people and the topics can be anything from retail discount alerts, to COVID updates, to political organizing. That absolutely merits some kind of reporting capability, particularly if criminal activity is being conducted or is about to be conducted.
Alright well I'm not a WhatsApp user, so I have a couple of questions:
In these large groups that obviously need moderation, are the moderators appointed in some way by WhatsApp, or are they like classical rooms in that the founders of the room pick the moderation team.
Also, do one to one rooms have reporting functionality and who picks the moderator that it is reported to?
Is your email private when recipient can take a screenshot and report it to the police? Is your snail mail private if recipient can take it to the police? The end to end encryption (or the envelope you put your snail mail into) only means that nobody but you and recipient(s) can read the content. It doesn’t mean that recipients can’t share said content with 3rd parties.
I don't think this is as complicated or hard to understand as this journalist is making out. Any content in a messaging system can be shared by either party with third parties, even if it's encrypted end to end. Hitting the "report" button is just one example of that, sharing the last few messages with WhatsApp moderators.
The part that aggravates me is that alternatives have their own issues. I managed to get my mom from whatsapp to signal, but when ( I wanted to say if, but that is wishful thinking ) it turns out it is no longer viable I know I will have a hard time migrating her. And even if I do manage to move her, it will be a painful process of re-learning new system for her.
They were critical of the client side scanning. So implementation as opposed to the goal. And also a marketing opportunity to take a pop at Apple due to their hubris over privacy and security.
No, that's not what the article is saying. There are two ends, but if one of the ends decides to forward the content to WhatsApp, then WhatsApp can read it. Which isn't shocking or breaking E2EE in any way.
How does one confirm how many “ends” are there in a conversation? If WhatsApp secretly set themselves (or another third party) up as an “end” in every conversation , that would also not break break E2EE.
Ok, but if we use this definition, then “somebody took a screenshot of my Signal chat and posted it on Twitter” is proof that “Twitter can read your Signal messages”. That doesn’t seem like a very practical definition.
A screenshot can be forged, meanwhile Facebook has reasonable certainty about the authenticity of the message. This third end is within the WhatsApp platform, so we're not really comparing the same thing.
It's essentially a forward to the party that supposedly can't see your messages, breaking what people understand by E2E.
Today this behavior is triggered when it's reported, tomorrow it'll be when the government, or worse, some AI, flags a user. You share a groupchat with the wrong person, or they have you in their contact list, boom, eavesdropped.
But some people will still claim it's only 2 ends just because it's not forwarded 100% of the time. I guess it's matter of semantics.
Can those who claimed that WhatsApp was infallible because it was E2E-encrypted take a step back and see that incentives matters as much as technology?
The sadness of my life is that I'm probably bound to be repeating this until I let out my final breath:
If you're not running the service yourself and taking care of end-to-end encryption, somebody can (and probably will) read your messages. Actual privacy of messaging does not exists today. Don't even bother looking for that.
EDIT: just to clarify, it might just be that somebody can access your data for a legitimate purpose (say an operator of the service) and then leak/sell the data.
This is the simplest thing that can happen and that DID happen. The united arab emirates paid a Twitter employee from UAE to leak data about anti-UAE-government from Twitter. People died because of this.
You can also copy the whole conversation and send it to the police, or post them on a forum, etc.
WhatsApp moderators can't read your messages, but they can read what the person you talk to send them. E2E encryption means that only the ends should be able to decrypt and read the contents, but what they do with them is another different topic.