Keep pulling the thread on Jared Quincy Davis.
Using a method of parallel model calls with early stopping can be faster, more accurate, and cheaper on average than a single model call.
By composing calls to cheaper models, it is possible to achieve the performance of a frontier model with cost reductions of over 1000x.
A study in the "Networks of Networks" paper found that composing architectures from calls to a frontier model achieved over 9% performance gains on difficult, verifiable benchmarks where generational model improvements were only around 1%.
The cost to achieve a baseline level of performance, such as GPT-4 on the MMLU benchmark, has decreased by approximately 10x per year over the last three years, for a total reduction of 1000x.
There is a massive cost dispersion among AI models, exemplified by O1 Pro at $150 per million tokens compared to DeepSeek R1 at 3 cents per million tokens.
Research from the "LM selector" paper demonstrated that for multi-step tasks, a hybrid system using different models for different steps outperforms the single best monolithic model applied to all steps.
Foundry's cloud platform can reduce compute costs by a factor of 12x to 20x for specific workloads that are preemptible, can be checkpointed, or can run in a flexible batch mode.
Many AI companies, including Google, Microsoft, and startups, now have compute costs that exceed their personnel costs.
For reasoning models like DeepSeq R1, a longer thinking time for a given problem correlates with a higher likelihood of producing an incorrect answer.
Foundry is a cloud platform built from scratch specifically for machine learning workloads.
Foundry's strategic goal is to enable a broader range of companies to achieve AI capabilities currently limited to organizations like OpenAI and DeepMind.
The "laconic decoding" method can improve performance by creating multiple model replicas and returning the response from the first one to complete its computation.