IMHO, API prices are likely to drop over time as we scale up and efficiencies are found, combined with general pricing pressure from the competition… SOTA models will continue to have SOTA prices, of course, but reasonable priced distilled and open weights models are already common.
Some of this seems like wishful thinking to me.
“as we scale up” We already have large data centers, and, if anything, “scaling up” is just driving up expenses for debt and hardware. The real problem with AI, relative to the internet boom, is that the more you use it, the more expensive it is. The economies of scale to be taken advantage of are minimal.
I do think price of compute will come down, but exactly because people will be running their own open source models on their own hardware instead of Sam Altman to stack GPUs in a warehouse, but it will still be too expensive and inaccurate.
And we’re still headed for a financial cliff in this scenario.
Of course they want to turn the corner and ideally keep all their infra, but they might need to restructure their debts if/when the creditors come knocking.
BUT: they have a money printing machine with inference… Enterprise is where most of the money is made, as they pay (closer to) API prices… consumer & small business plans are heavily discounted in comparison, with mostly underutilized use-it-or-lose it plans. They’d be stupid if they were making an opex loss on inference.
As for why API prices are dropping? Competition is a bitch… but they seem willing and able to play the game for now.
Like a fraction of a percent of people will run their own models. It won’t influence price. Remember that coders/tech people are only a minority of users.
Oh, I see. I was mistaken. I thought tech companies were OpenAI’s major customers, and they certainly could make their own data centers for internal work, and that coders–that minority of users–were the only ones likely to pay full price for tokens, the rest of us wouldn´t be willing to pay full fare for a machine that is right most of the time sort of. Thanks for setting me straight on that.
Some large companies do it … but I think you greatly overestimate the willingness of large enterprises to build and maintain their own infra. Once a company has lost the competency (and many have), it’s hard to get it back. There are always pockets of people with the right skillset in a large org, but they mostly don’t want to do DC work anymore. Team/personal servers and beefy laptops, otoh…
Some of this seems like wishful thinking to me.
“as we scale up” We already have large data centers, and, if anything, “scaling up” is just driving up expenses for debt and hardware. The real problem with AI, relative to the internet boom, is that the more you use it, the more expensive it is. The economies of scale to be taken advantage of are minimal.
I do think price of compute will come down, but exactly because people will be running their own open source models on their own hardware instead of Sam Altman to stack GPUs in a warehouse, but it will still be too expensive and inaccurate.
And we’re still headed for a financial cliff in this scenario.
Of course they want to turn the corner and ideally keep all their infra, but they might need to restructure their debts if/when the creditors come knocking.
BUT: they have a money printing machine with inference… Enterprise is where most of the money is made, as they pay (closer to) API prices… consumer & small business plans are heavily discounted in comparison, with mostly underutilized use-it-or-lose it plans. They’d be stupid if they were making an opex loss on inference.
As for why API prices are dropping? Competition is a bitch… but they seem willing and able to play the game for now.
If we’re talking about the survivability of AI as a technology, I agree.
If you’re talking about the viability of the hyperscaling business model, I think we are looking at wildly different sets of information.
Like a fraction of a percent of people will run their own models. It won’t influence price. Remember that coders/tech people are only a minority of users.
Oh, I see. I was mistaken. I thought tech companies were OpenAI’s major customers, and they certainly could make their own data centers for internal work, and that coders–that minority of users–were the only ones likely to pay full price for tokens, the rest of us wouldn´t be willing to pay full fare for a machine that is right most of the time sort of. Thanks for setting me straight on that.
Some large companies do it … but I think you greatly overestimate the willingness of large enterprises to build and maintain their own infra. Once a company has lost the competency (and many have), it’s hard to get it back. There are always pockets of people with the right skillset in a large org, but they mostly don’t want to do DC work anymore. Team/personal servers and beefy laptops, otoh…