Fwiw - I did a fairly large comparison of Gemini Nano (the in browser ai model) vs a comparable free hosted model of Gemma (from OpenRouter) and the hosted model absolutely trashed the local model on every aspect of speed, reliability, availability, etc. [1]
I'm not particularly happy about that outcome as I wish we had more locally run AI models for reasons of privacy and efficiency, so this is more just a warning that at present there are some severe tradeoffs.
Thanks for the write-up and the comparison, but more importantly for using the API in production!
You’re highlighting the "state of the art" gap we’re working to close. Cloud models will always have the advantage of massive parameter counts, but our bet is that for a huge class of simpler or high-volume tasks, the upsides of on-device (e.g. zero-cost, permission-less start with no quotas/infra, network-resilience, privacy) make it a compelling trade-off.
The models have been getting better at a rapid clip, and the team is heads-down on optimizing performance and reliability. To that end, we're always grateful for feedback. If you hit specific bugs, crashes, or quality regressions, filing a report with repro steps is the best way to help us improve. You can file those on crbug.com under the "Chromium > Blink > AI" component.
There's a push and pull here, Typescript + React + Vercel are also very amenable to LLM driven development due to a mix of the popularity of examples in the LLMs dataset, how cheap the deployment is and how quick the ecosystem is to get going.
Memory makers make capital investements (build different factories, convert physical production lines, etc.) to meet orders that have been place for the next ~5 years.
OpenAI (or whoever) crashes and can't pay for the order leaving the memory makers in a tough spot.
The amount of money flowing both from the AI bubble and from quite literally scalping both the server and consumer market... They gambled on the opportunity and if they fail - it's their problem.
And how is that a problem and more importantly how's that a problem of Average Joe?
Capitalists did their gamble things. If they fail in that gamble what forbids them to sell the regular RAM they made for AI bubbleists to the regular consumers? Besides HBM it's just the regular chips which are exactly the same for the consumer/server market, why it would be any different?
Partially this was solved by the ZipDrive being designed for portability as well (I'm not even sure if there was ever a built in model). So if you needed to copy a large file and take it to a friends house you just took the drive with you.
Hang on, tell me how, because I am not picking up what you are putting down. At a minimum, wouldn’t this require working from a perfectly written spec that has already accounted for the discovery of changes that would need to be made from the original perfect spec?
1. "Mythical Man Month" which is the shorthand for a whole book + concept that you can't just throw more people at a software development project and get linear productivity improvements as the communications overhead (meetings, emails, mistakes due to poor assumptions, etc.) deeply eat into the raw number of productive hours that a new person added to the team brings.
2. AI automation tools (Claude Code) are often described as a "junior developer" which is an imperfect comparison as while you could potentially sort of set them up that way many people use them as more of a singular force multiplier.
I use them to work on many more projects in many more ways and ship far more than I could even if I had a "junior developer" sitting alongside of me as there's not the same level of communication needed.
The way I see it, because you can spin up additional AI employees at will (and spin them back down), when the problem with the spec is found, it's no big deal to redo all of that work from before, adjusting for that change.
Ironically, people keep saying this, but then gloss over the core problem of coordinated between these agents... For completely independent codebases with no dependencies, sure thing right on...go for it. But the vast majority of F500 companies I work with have wild and undocumented dependencies between almost every system that will take years to "agentify" (assuming they ever figure out that it's an organization and governance problem.... which they might not ever realize)
What's weird though is the bifurcation in pricing in the market: aka if your app can function on a non-frontier level AI you can use last years model at a fraction of the cost.
This is one of those slippery slope things where Grammarly did "just" Grammar and then slowly got into tone and perception and brand voice suggestions and now seems to more or less just want to shave everything down to be as bland as possible.
I tried using an LLM to help me write some stuff and it simply didn't sound like I'd written it - or, it did but in a kind of otherworldly way.
The only way I can describe it is like when I was playing with LPC10 codecs (the 2400bps codec used in Speak'n'Spells, and other such 80s talking things). It didn't sound like me, it sounded like a Speak'n'Spell with my accent, if that makes sense.
No? Okay, if not, if you want I could probably record another clip to show you.
All you have to do is prompt your AI with a writing sample. I generally give it something I wrote from my blog. It still doesn't write like I do and it seems to take more than that to get rid of the emdashes, but it at least kicks it out of "default LLM" and is generally an improvement.
reply