Actually, you might find this interesting: we wrote a blog post a while back about how we gave our agent a way to voice its suffering. The first example is from the screenshot updating subagent, where it got stuck after 84 browser interactions trying to log into a customer's app :'(

https://promptless.ai/blog/technical/i-must-scream

Funnily enough, one of our early users (a solo technical writer) told us that the biggest value he got from Promptless was just finding out when the engineering team forgot to tell him about a change that would impact docs. The drafts from Promptless were just the cherry on top for him.

So yes, there's a workflow where you really just use Promptless for the notifications. That being said, the drafting is now quite good, so it's probably at least a good starting point to get past writer's block!

That's fantastic! I may very well bring this up at our next all hands, since we recently just wrote a ton of documentation for our modeling code.

Yep. You give Promptless access to a test account and it navigates and interacts with your product to capture new/updated screenshots.

Here's an example from our own docs, where you can see the captured screenshots if you scroll down to the PNGs that Promptless added https://github.com/Promptless/docs/pull/233/changes

Getting the browser navigation to work was predictably hard, but the other thing that's very hard to get right is timing when Promptless takes the screenshots. Some teams want Promptless capturing screenshots on a staging environment, and sometimes it's on prod, and different teams have different release cycles and feature flagging processes.

For that reason, we still have to do a bit of manual set-up for each user that wants screenshot updating.

Agreed though that just asking nano banana to predict a new screenshot is negative value.

I've built something similar to this internally and absolutely agree that timing is the real challenge.

Getting an AI to navigate a process is now easy. However, the difficult problem is having it understand: 1. How to screenshot or, even better, capture a GIF of exactly the right points in time 2. How to move the mouse and pause at the right points in time such that a human understands it

tl;dr. I'm one of the founders of Promptless, the AI agent that automatically updates docs. 12 months ago, we decided to give our agent a tool to let it "scream" when it was struggling, and we wanted to share our learnings and some cool examples of issues that it caught.

AMA!

A (charitable) interpretation of this is that the model understands "stuff that would embarrass Anthropic" to just be code for "bad/unhelpful/offensive behavior".

e.g. guiding against behavior to "write highly discriminatory jokes or playact as a controversial figure in a way that could be hurtful and lead to public embarrassment for Anthropic"

In this sentence, Anthropic makes clear that "be hurtful" and "lead to public embarrassment" are separate and distinct. Otherwise it would not be necessary to specify both. I don't think this is the signal they should be sending the model.

Promptless tries to check its generated docs against other sources to identify errors, but there are also some very cool documentation testing tools that we might integrate in the near future!

Happy to clarify! There are two main ways people are using the Slack integration: (1) You click the Promptless Slack message action (video below) on a thread that might warrant a doc update—Slack Connect channels are an extremely common use-case for this, and (2) You tag @Promptless in a DM or a channel with a brief instruction—people sometimes use this for simple requests, but after signing up for Promptless, you can also send Promptless a link to an old PR, and it'll use the Github API to read the content of the PR and draft an update.

Here's a video we made a month or so ago that shows how (1) works, hope that helps! https://www.tella.tv/video/cm5x8uj9f001d0al11vyxcnsc/view

Thank you!

Yeah, even with lots of full-time resources, companies' documentation regularly fall out of date. It's particularly ubiquitous in AI just because of how fast the general space is changing.

Thank you!