Keep pulling the thread on Simon Syrupsen.
Simon, CEO of TurboPuffer, predicts it will become a baseline expectation for every SaaS company to offer an AI-powered research feature that can query all of a customer's data.
The development of databases built on object storage was enabled by three key technological advancements: the availability of high-bandwidth NVMe SSDs (c. 2017), S3's introduction of strong read-after-write consistency (Dec 2020), and S3's addition of a compare-and-swap primitive (late 2023).
The primary architectural trade-off of databases built on object storage is a high write latency, with a P90 of 100-200 milliseconds per commit to a service like Amazon S3.
TurboPuffer has developed an algorithm for incrementally maintaining its Approximate Nearest Neighbor (ANN) index that scales to hundreds of millions or billions of vectors per shard.
AI features are becoming "table stakes" for SaaS applications, similar to how mobile apps became a mandatory offering for software companies in the previous decade.
The new standard AI features for SaaS applications include semantic search, similarity detection for recommendations, question-answering over user data, and agentic workflows.
The AI-native code editor Cursor uses TurboPuffer to power semantic search over user codebases for its AI agent and other features.
Notion uses TurboPuffer to power its Q&A feature, enabling semantic search over documents such as internal wikis.
Linear uses TurboPuffer for its primary search functionality and for similarity-based features like identifying duplicate issues and suggesting issue assignees.
Relying solely on large context windows for data retrieval is limited by issues of scale, cost of VRAM, poor recall, lack of trusted access controls (ACLs), and high latency.
Modern NVMe SSDs offer bandwidth (e.g., 10 GB/s) that is only one order of magnitude lower than DRAM (e.g., 100 GB/s), but at a cost that is two orders of magnitude less ($0.05-$0.08/GB vs. ~$5/GB).
Using a vector search extension in a relational database like Postgres results in an effective storage cost of approximately $0.60 per gigabyte due to data replication, disk costs, and utilization overhead.