OpenAI and Anthropic cut their LLM prices by up to 80% within a few months. Why?

If we assume it's due to competition, then their usage has precisely not plummeted because they cut prices by 80%.

They are too big to implode.

The sword that protects is the exact same sword that attacks. One idiot attacks the wrong party and the show is over.

It's nice because I can keep going from anywhere, other team members can see and interject in my Slack threads, and Hermes auto updates its skills with any adjustments that we make so we don't have to ask twice.

Comiong to harnesses, even Pi seems like a weird choice. I mean Pi, until it's "your own customised Pi", is pretty useless and pointless. Besides they are also starting to have their own models/infra business (or is it mostly proxy like OpenCode?).

I have also given up on Herdr and came back to Tmux (and doing a bit of customisation slowly, as I need it). I wanted to try dsh but somehow couldn't brave it. Maybe once I have a good container/vm setup is ready. I know I am biased/discriminatory here, but there are things about which I'd be better safe than sorry.

As for OpenCode: I used V1 a lot and I am struggling with V2 currently. Yes, it just doesn't feel right. It's neither here, nor there. I am not a harness reviewer, so I won't be able to put that into industry accepted words and tests but it doesn't feel useful and right to me. At least Pi is completely stripped (it might be changing though) and then you can make it as you want it. Even Claude Code is better than OpenCode and that's saying something. I find Codex to be quite effective but it doesn't feel familiar. I am still in the middle of some work so I am using OpenCode v2 a lot (and then there's OpenCode Go and I won't want to play around a lot with it) but I will move away. Where? I am not sure yet.

What I feel will work for me? I guess a harness of my own! Forked something smaller/similar? I don't know. No, really! Just like I have never found the "my" notes app and journal app yet, I don't think I should use any of these harnesses if it doesn't work for me, especially now that I can kinda make one (hail LLM/vibe coding :P). It doesn't even have to be a TS/Node monsters most of these harnesses are. More I am reading about, digging deeper (haven't been quite far yet), how models and the "local setup" interact, the more I end up scratching my head – why all this drama and why all these 5 billion GitHub star "harnesses" with trillions of lines of code! Why? It doesn't make sense!

And I’m slowly writing my own. But it’s garbage at the moment.

However, Anthropic is pickier about the harnesses you can use with Claude Code and I noticed discrepancies when using Zed's External Agent feature with Claude Code, so I just use the Claude app. It's fine.

Sonnet 5.5 is 16x more expensive at $10 per million output tokens. Kimi K3.1 is also $10 per million output tokens, and its number of active parameters is only 6.5x larger than V4.1 Flash (104B compared to 16B). These things are not directly comparable of course, but just to give an idea.

Even if inference of those models needs 16x, 32x, or 48x the number of GPUs compared to V4.1 Flash, still seems like potentially great revenue per GPU.

> The current service throughput of a single cluster is 155 trillion tokens per day, and the overall revenue is $38 per hour (at the lowest price) per card.

With Kagi translate (best), it says GPU

> A single cluster currently serves 155 trillion tokens per day, generating overall revenue of $38 per GPU per hour (at off-peak electricity rates).

https://zhuanlan.zhihu.com/p/2095560295930508953

No, it isn't. It's revenue per hour per GPU.

They are meant to help the consumer.

Unfortunately neither of the two parties in the US even pay lip service to free markets anymore.

Regulate the bad actors, not the neutral tools. Lockpicks aren't illegal, but breaking and entering is. Imagine if MasterLock used a lockpick to break into a government building and steal stuff, then tried to lobby to make lockpicks illegal. That's what OpenAI is doing, and you're falling for it.

In the hands of 6th party system Republicans, namely Ronald Reagan and Robert Bork, the government worked hard to usher in a a new era of Robber Barons. We live in the culmination of this project.

History says the result depends on the hands, not the tool. 6th party system Democrats have largely failed to undo the Reagan/Bork policies, but that's an act of omission not comission and even then Lina Khan was a big step in the right direction. This issue was on the ballot in 2024 and will be again in 2028. Hopefully we make better choices in 2028.

Government regulations exist to regulate activity, to prevent, punish, and/or remediate harms.

Environmental regulations exist because businesses wouldn't clean up their own messes. Financial regulations exist almost exclusively because of harms and abuses committed by members of the financial industry. Building codes are updated to reflect lessons learned about people died, to prevent more people from dying the same way.

If AI regulations are passed, it will be because the AI industry decided not to self-regulate itself and stop illegally hacking computer systems, generating CSAM, or various other illegal activities that should easily be preventable if their systems were actually intelligent.

Also there are real trade offs. Food would be cheaper if, for instance, it were legal to cut it with any old cheap substance regardless of health effects.

But I totally agree that some regulations are dumb or are regulatory capture monopoly schemes.

Unfortunately like many other things there is no absolute simple rule to save us from thinking. Each case must be evaluated in depth hopefully with domain expertise.

API usage being profitable is completely plausible, but almost nobody pays API prices for Claude and GPT, nor would they be able to afford them if the subscription suddenly stopped existing.

Because all these newly trained models using the compute are doing really well and growing!

Even if something is a reasonable infrastructure project with a sound long term business plan, investors can still lose money if they pay too much for it. For the Telco bubble & crash, there were a few of these Telco companies engaging in financial shenanigans / securities fraud to juice their quarterly financials by leasing capacity to artificially inflate their operating revenue etc. E.g. we booked $100m of revenue by leasing X units of our capacity to $competitor! Sure, we're still extremely cashflow negative, but look at our operating revenue grow! Fine print: we also incurred a symmetric $100m expense to lease X units of capacity back from the same $competitor, but for some reason we didn't classify that as an operating expense, we've thrown it into a different accounting bucket.

That said, overbuilding fiber optic capacity by laying undersea cables is likely different to overbuilding GPU datacenter capacity in that most of the cost of laying a cable is the physical cost to lay the cable -- given you're already laying an undersea cable, the incremental cost to add a bit more capacity to it (more optical fibers) is negligible -- at least before you've sunk the cable! Whereas for a data center full of GPUs, maybe 50%-60% of the total cost is the GPUs, so it isn't cheap to massively overbuild capacity.

[1] https://en.wikipedia.org/wiki/Telecoms_crash

Language modeling is fundamentally compression: https://arxiv.org/abs/2309.10668

The entire endeavor is ultimately fundamentally coupled to reducing compute

Eventually we will compress models further by removing outdated knowledge like Python and Cobol cause who needs 1960s like languages to then layout data in the screen when the model has knowledge of geometry and can mutate the screen geometry without needing a Python interpreter or C compiler

This is the same mechanism used in unlimited mobile internet for example.

I expect they will be "unlimited" in the same sense the mobile / ISPs do that 99% of people never reach the limits.

We could have "unlimited" electricity too, if we choose, by printing solar panels (like the Chinese) and hopefully fusion before long.

Maybe, maybe not. There could be a saturation point of compute demand from ordinary consumers. (See: hot-water analogy.)

Tell that to my dial up modem as I played Active Worlds in 1996.

The cost of processing tokens however does not. Throughput and efficiency are improving around 10x per year. Hardware is not the only factor.

People don't like complex pricing. This would be for businesses running larger workloads. For individuals, the unlimited plan seems the most probable to me. This stuff is going to get real cheap in the coming years.

The energy needed per token is in dropping rapidly, even though the energy per-GPU is going up. We are seeing something like 10x improvements per year.

[1] https://dealroom.co/news/165341-anthropic-backs-its-first-eu...

To give a simplistic example: if a city as a ton of available AirBnBs, you might say, "Lots of people are willing to rent out their apartments, it must be profitable for them to do so." But the fact that all of those rooms are not already booked could also mean that way too many people have attempted to rent out their apartments.

And for what it's worth, it's possible for supply to outstrip demand while still leaving inference profitable. The question is whether it's (a) profitable enough to service massive debts and (b) sustainably profitable over the long run.

Edit: to add one note, it's possible for the LABS to be profitable on inference without inference itself being profitable. We don't know what the contracts look like, but it's entirely possible that labs received sweetheart deals from neoclouds so that the latter could flaunt big utilization numbers.

Electrification will drive the demand for oil down, making the overhead more expensive per gallon, and eventually we will largely move off of fossil fuels, though I don't expect completely. It will be many decades due to the turnover times for industrial/commercial transportation systems.

Meanwhile, Oracle is handing Tencent 100.000 GPUs at 1.60$ hour, dangerously close to breakeven...

What does this tell you?

Maybe the FDA is a good analogy, you wouldn’t want to buy meat at the store not FDA certified would you? OpenAI and Anthropic want you to feel the same about AI.

There will likely be requirements like registering with the government (China already does this, every business already does everywhere), certain usages require more or have deeper regulations (this is already done with most technologies, depending on application)

Commercial airlines are of great value to our lives. Who's questioning or denying that?

Horrible business to be in though.

That's not how you grab market share. Early on, any profit you aren't reinvesting into the business is basically hurting your growth rate.