Keep pulling the thread on Fully Connected Tokyo.
OpenAI and Meta use the Weights & Biases "Models" product to track the training of their large language models.
Theo from Mercari states that rapid iteration, enabled by seeing results within minutes, is the key to their success in AI development.
The team at Mercari, led by Theo, has successfully shipped numerous AI applications by adopting a rapid iteration development approach.
Weights & Biases hosts open models such as Quinn, Kimi, and some Llama models on its inference server.
The Weave platform is designed for production use, with features to handle large scale, operate asynchronously without blocking processes, and prevent application crashes.
The Weave platform can be self-hosted, but Weights & Biases offers a "dedicated" deployment option where the platform runs on a customer's own AWS or GCP infrastructure.
The pricing model for the Weave platform is based on data ingest volume, with user seats being free.
The system prompts for major AI applications, such as those from Replit, have been leaked and are available in public code repositories.
The CTO of Weights & Biases, Sean, was at one point ranked number one on the public Sweebench benchmark for code generation.
The Weave platform supports logging and visualization of multimodal data types, including images, video, and HTML, by specifying the return type of a traced function.
To manage the high cost of running the Sweebench benchmark, Weights & Biases' CTO Sean iterates on small subsets of 3-5 examples and runs the full evaluation only periodically.