What a weird time for our industry. On one hand, small teams have never been able to move faster than right now.
On the other, the economy and market conditions are brutal for the little guys. Incumbent behemoths hoovering up value, talent and financing.
Instead of shaking things up as usual when a major paradigm shift hits, AI has mostly been a centralizing, consolidating force. Not that I was expecting it to be otherwise, but it's certainly dismaying to witness.
Or am I being too pessimistic / glorifying the past?
It's easier than ever to make your own furniture. IKEA is bigger than ever.
It's easier than ever to publish a video game. Steam is bigger than ever.
It's easier than ever to 3D-print tractor parts. John Deere is bigger than ever.
It's easier than ever to switch to solar power. The petroleum industry is bigger than ever.
One person reverse-engineered Coca Cola, made an exact taste-alike and published the formula. You can make some at home. Coca Cola is bigger than ever.
The hidden cost to competing in these industries is insane. Its so hard to build a physical product that can compete against a giant like IKEA. You need to make some with less r&d, less automation, less infrastructure and you're going to sell less units and all that needs to be price competitive against something that is made on an production line with a team of experienced engineers and sold to millions at fine margins.
In a reductive sense, yeah it's a bit silly. But zooming out, I can understand. Sucks to have your hand forced. Sucks to be let down. Sucks to watch something that was great fall from grace.
Thanks for Ghostty, been my daily driver for awhile now. Hope the rest of your day/week goes much better!
Perhaps you could generate a few tokens before the entire model is downloaded, but since every token takes a potentially different "path" through an MoE model, you'd still need to wait for the entire download before getting deeper than a handful of tokens... which is not really a UX improvement imo.
Even at its worst, it's a minor UX improvement compared to having to download everything prior to getting to the first token. Ultimately we will complete the download, but we can still pick the best priority so that the first handful of tokens goes through.
How well does it work with Godot? Engines like Unity and Godot are very focused on using the editor UI, so I've always wondered if there's any better workflow than generating code snippets. Unless you're going full .NET/GDExtension...
> I would also expect to see it taking exponentially longer to process a prompt. I don't believe LLMs work like that.
Try this out using a local LLM. You'll see that as the conversation grows, your prompts take longer to execute. It's not exponential but it's significant. This is in fact how all autoregressive LLMs work.
Yesterday I was playing around with Gemma4 26B A4B with a 3 bit quant and sizing it for my 16GB 9070XT:
Total VRAM: 16GB
Model: ~12GB
128k context size: ~3.9GB
At least I'm pretty sure I landed on 128k... might have been 64k. Regardless, you can see the massive weight (ha) of the meager context size (at least compared to frontier models).
> As a user, I _expect_ the cost of resuming X hours/days later to be no different to resuming seconds or minutes later.
As an informed user who understands his tools, I of course expect large uncached conversations to massively eat into my token budget, since that's how all of the big LLM providers work. I also understand these providers are businesses trying to make money and they aren't going to hold every conversation in their caches indefinitely.
I'd hazard a guess that there's a large gulf between proportion of users who know as much as you, and the total number using these tools. The fact that a message can perform wildly differently (in either cost, or behaviour if using one of the mitigations) based on whether I send it at t vs t+1 seems like a major UX issue, especially given t is very likely not exposed in the UI.
Haven't had a chance to test 4.7 much but one of my pet peeves with 4.6 is how eager it is to jump into implementation. Though maybe the 4.7 is smarter about this now.
On the other, the economy and market conditions are brutal for the little guys. Incumbent behemoths hoovering up value, talent and financing.
Instead of shaking things up as usual when a major paradigm shift hits, AI has mostly been a centralizing, consolidating force. Not that I was expecting it to be otherwise, but it's certainly dismaying to witness.
Or am I being too pessimistic / glorifying the past?
reply