Why so negative lol. The speed and very reduced power use of this thing are nothing to be sneezed at. I mean, hardware accelerated LLMs are a huge step forward. But yeah, this is a proof of concept, basically. I wouldn't be surprised if the size factor and the power use go down even more, and that we'll start seeing stuff like this in all kinds of hardware. It's an enabler.
It's not negativity, it's like building a flying car but it's some crap from the 60s that nobody wants to drive anyway, it doesn't even have airbags. If you're so advanced w millions of dollars of funding why didn't you pick up a simple Prius? It's not that hard. They could instantly snatch up a ton of market share by being 100x cheaper/faster, but now instead they've given all the other LLM providers a heads up. Either the tech doesn't work on newer models, or they just want to get bought out ASAP.
