Keep pulling the thread on Ankur Goyal.
Ankur Goyal observes that many incumbent companies now face an existential choice to either rebuild their products around AI or risk failure.
Ankur Goyal asserts that the most recent generation of AI models are now capable of evaluating and improving their own work.
Ankur Goyal believes that for AI product development, the primary and most critical activity is focusing on evaluations ('evals').
Ankur Goyal asserts that there is a general consensus among practitioners that the internal workings of Large Language Models are not fully understood.
Ankur Goyal predicts that the AI model a developer chooses today is highly unlikely to be the sole model they use in the future.
Stripe, Instacart, and Airtable are customers of Braintrust.
Braintrust's go-to-market strategy involved being highly selective about initial customers and focusing intensely on making that small group successful.
Braintrust's engineering culture prioritizes immediately fixing customer-reported issues over adhering to pre-planned sprint commitments.
Braintrust's initial open-source-based logging infrastructure failed to scale under the exponential growth of its early AI-native customers.
Braintrust's custom data system, Brainstore, is purpose-built to handle large volumes of text and complex JSON data characteristic of LLM workloads.
Ankur Goyal differentiates Braintrust from Datadog by stating that customers use Braintrust to achieve product 'quality', whereas they use Datadog to achieve 'uptime'.
A key feature of the Braintrust platform is the ability to connect production logs to evaluation datasets that are linked to code and prompts in GitHub.