In the article the author says they are doing reinforcement learning with LLMs.

Seems like they just want to play PewDiePie after making tons of money from their salaried job and have a bunch of spare time now.

How does this differ from dottxt's Outlines[0] on the technical level? Are you using some JSON grammar to force the LM head distribution to follow it?

[0]: https://github.com/dottxt-ai/outlines

I only just skimmed it, but will try to dive deeper in a bit.

I think we share a lot on tool definitions/schemas. Forge will let a consumer define a tool, set of tools, pydantic schema for each, etc. outlines seems to be similar with their task definition.

I think where we differ is what happens when that doesn't work...and the model still doesn't get the contract right. Something like a pydantic-valid string path for glob, that points to a non-existent thing. Glob will error, forge catches, and nudges the model. Forge does very little model output manipulation (just a basic regex parse to try to find json/XML), the core of it is in the retry mechanisms.

Once I dig into it more I'll try to highlight other deltas.

it's using webgpu kernels, probably a false positive

I really like QBE but I hope they will make it a true library because I don't want to launch a subprocess to compile a program.

This is exactly my gripe unfortunately, it feels like needless fragility. IIRC the author has said they believe it wouldn't be too difficult to patch QBE to work as a library, but from what I've seen the code is somewhat terse and eccentric.

There is a library conversion fork: https://github.com/sgraham/sqbe.

IMO when the intended usage is AOT with an external assembler, which is another subprocess, text-based IO is actually the more natural approach.

There are at least 2 Go libraries written in Go, based on QBE. One is used by this project: https://github.com/xplshn/gbc

Why not? Processes are cheaper than you might think. GCC invokes several as an example.

Because this concept only works for offline compilers, but not for dynamic languages. It's about 100x slower.

I'm just converting the call to an external assembler in my compiler rcc to assemble the bytes directly. No need for strings and external files. The cost of the external call is outrageous.

Tangentially: Is there a Rust alternative similar to this?

Cool but why would anyone write an agent in C? Since the language is verbose it seems to be a painful choice to use C for this task.

Aside from context length, verbosity is a strength for LLMs. However, what's not a strength is the closing braces, so I'd say it's a tie.

They have added a lot of optimization focussing on the KV-cache, so they can have a much larger window without eating all the VRAM.

The 1M window might be usable, but it will probably underperform against a smaller window of course.

Mollie might be a direct competitor

Isnt Mollie Europe only?

Mollie seems to only provide services to business based in European Economic Area, Switzerland and the UK [0], so yes?

[0]: https://help.mollie.com/hc/en-us/articles/115002116105-Can-I...

And they also have a minimum turnover limits. They rejected us as too small.

Is there anyone that feels that LLMs are wrong for computer use? It's like robotic, if find LLMs alone are really slow for this task

> find LLMs alone are really slow for this task

Faster LLMs will be here by next year.

Some people are speculating that Opus 4.7 is distilled from Mythos due to the new tokenizer (it means Opus 4.7 is a new base model, not just an improved Opus 4.6)

The new tokenizer is interesting, but it definitely is possible to adapt a base model to a new tokenizer without too much additional training, especially if you're distilling from a model that uses the new tokenizer. (see, e.g., https://openreview.net/pdf?id=DxKP2E0xK2).

Not impossible, but you have to be at least a little bit mad to deploy tokenizer replacement surgery at this scale.

They also changed the image encoder, so I'm thinking "new base model". Whatever base that was powering 4.5/4.6 didn't last long then.

Yes, I was thinking that. But it could as well be the other way around. Using the pretrained 4.7 (1T?) to speed up ~70% Mythos (10T?) pretraining.

It's just speculative decoding but for training. If they did at this scale it's quite an achievement because training is very fragile when doing these kinds of tricks.

Reverse distillation. Using small models to bootstrap large models. Get richer signal early in the run when gradients are hectic, get the large model past the early training instability hell. Mad but it does work somewhat.

Not really similar to speculative decoding?

I don't think that's what they've done here though. It's still black magic, I'm not sure if any lab does it for frontier runs, let alone 10T scale runs.