What if AI compute had a stock exchange?
A friend of mine has recently started a company trying to build a marketplace for AI inference. Very very cool. Though my mind starts to boggle at the implementation complexity of it.
What do I mean by an AI inference marketplace? Ultimately this is a platform that would connect people who need inference done (buyers) with people who own & provide inference capacity (sellers). Think of 'inference' as the AI actually doing the task you want done.
Say I need an AI to do a task for me. I could specify what sort of quality I would need (i.e. the capability of the model), how quickly I need the job done, what sort of cost I'd entertain, and even where the data needs to be processed (residency/sovereignty).
The marketplace would receive this request - and then start to connect it to the sellers (i.e. those offering inference capacity). The sellers could then bid against each other for my AI task - competing to fulfill all of my various quality/speed/cost/residency requirements - and trying to be selected to go actually do the job.
The marketplace would then pick the winning bid, connect the buyer and the seller, and allow the inference to take place. Once inference is done, payment is made & everybody's happy.
I don't have an economics degree, but marketplaces are generally good things. Introducing competition into inference transactions helps to push prices down & accelerates innovation because suddenly frontier labs can't just sit back and enjoy complete consumer brand lock-in - they need to compete for customers at the point of inference demand.
This is still very early days. Nowadays, people generally align to a single model - in business I've worked with companies that appear to have enterprise AI strategies aligned to a single frontier model (no comment...), and in personal life to be honest I generally use chatGPT rather than Claude or Gemini. Could I be getting my AI tasks done more cheaply & efficiently if I used an inference marketplace that could fine-tune the inference capacity to my specific needs with competitive pricing? Yes probably.
More companies looking into this probably do exist, and marketplaces for GPU capacity do already exist in various guises - but here GPU capacity is more like a marketplace for the hardware underlying inference than a marketplace for choosing between inference providers & models. Semantics...
Either way - maybe in a year's time or less, I think this will be launched & commonplace. And maybe by then I'll have understood it a bit better...