Alternate headline: C-Suite Execs Come To The Realization That AI Costs Money
And it’s going to cost a whole lot more once the investments dry up and they’re going to have to actually pay for all the data centers under construction.
Prolly not. When the bubble pops these data centers will be sold off for cents on the dollar and a bunch of the companies go bankrupt. There will be a glut of equipment, so prices will fall. People will buy up the scraps and provide compute services cheaper.
Unlikely. They said that during the bitcoin and blockchain bubble too, and that didn’t pop, it just deflated and everything got moved over to AI. AI is starting to deflate, but there will be a new bubble.
Why even bother at all? What can you do with those tokens? Ask for the weather twice?
The Haiku 5.5 model is actually quite inexpensive. I’ve used it to implement multiple features this month and have only spent $4 of my $20 monthly budget.
Are you using api or subscription?
Technically neither - I use it via Microsoft Co-pilot subscription, and it’s a la carte for model usage from every provider.
Ah co pilot was much worse than claude code when I used it. Subscription is far far better the api prices are outrageous
Microsoft’s copilot branding is all over the place. Copilot CLI and GitHub copilot (IDE plugin) both let you choose which model you want to use.
Let’s not forget Copilot is also an office suite lol
Everything is copilot. No more windows, copilot is my operating system. Microsoft has changed its name to Copilot, Inc., and Satya Nadella has updated his CopilotedIn profile name to Copilot Copilot. Copilot is my copilot.
If you’re spending $1K per month on LLMs – putting aside whether this is productive or ethical for a moment – wouldn’t it make more sense to just pony up the $5K-$10K or so that it would cost to provision a private, local model? Might that not actually be superior in terms of corporate security?
Definitely. We do this at my job. But a server big enough to provide a comparable service would cost at least $30k. And you likely need more than one. And you still need someone to maintain it.
I’ve heard that GPU’s for this work are exceedingly hard to get ahold of as well. Glm 5, which i think is near frontier model capabilities, takes around 1.5 Tb of vram un-quantized.
Not hard. Just really fucking expensive and might have a long lead time.
