Keep pulling the thread on Peter Dang.
In modern AI product development, the "eval" (evaluation) has become the new form of a product specification, defining the desired behaviors of the model.
A significant opportunity for startups exists because a large amount of valuable data is not on the open web and has not been used to train current foundation models.
Foundation model companies have a competitive advantage in product development due to early access to new models and the ability to directly tweak model behavior with post-training teams.
Startups hold a competitive advantage over large model companies by being able to dedicate 100% of their focus to a specific use case, allowing them to move faster and build more tailored products.
Peter Dang believes a key future frontier for AI is making models proactive, where the AI initiates conversation or action, rather than being purely reactive to user prompts.
Peter Dang believes the term "agent" is currently overhyped, as today's AI systems can think and communicate but lack the tools to reliably perform actions in the real world.
Peter Dang identifies three key components for defensibility in AI startups: unique proprietary data, a data flywheel from user interaction, and rapid market penetration.
Peter Dang predicts that user interfaces for many applications will become less important and may largely be replaced by voice-based interactions with AI agents.
Peter Dang believes "evals" (model evaluations) are the most underhyped aspect of AI development today, calling them the "unlock" for shaping model behavior.
Peter Dang predicts that subscription-based pricing models for AI products will evolve dramatically.
While at OpenAI, Peter Dang felt that pricing ChatGPT for Enterprise per-seat was flawed because it did not align with the value of the work being provided.
Peter Dang asserts that pricing AI products based on token consumption does not accurately reflect the value created for the customer.