Superhuman Mail trusts 5x more emails to turbopuffer

I will always pick a product with great reliability over a product with great features. turbopuffer has both.

Rafael Melo Cardoso, Engineering Manager

Rafael Melo Cardoso is the Engineering Manager of the Platform pod at Superhuman Mail. His team is responsible for building and maintaining the AI infrastructure used by all the other product pods to build AI features, agents, and chat interfaces.

The Superhuman Mail team first attempted some basic AI chat and search features several years ago, but they were narrow and prompt-based, limited to the context of a specific email thread. When they set out to build Ask AI in early 2024, Rafael and his team designed a retrieval layer using a traditional vector database to handle semantic search over inboxes.

The vector database had three main problems:

1. Write throughput: Most emails are sent in the morning. During peak hours (~9AM PT), ingestion latency would spike. Users could see an email in their inbox but search couldn't find it. In the worst cases, indexing lag exceeded 24 hours. Rafael had to self-impose write throttles (~400 req/s per node) to avoid overwhelming their vector database provider.
2. Active tenant limits: The provider capped active tenants at 10k, but Superhuman Mail manages hundreds of thousands of inboxes. Reads and writes shared the same path, so any inbox receiving email counted as active, even if nobody was searching it. Rafael's team had to build processes that constantly shuffled inbox activation to balance write throughput and read consistency.
3. Degradation at scale: Based on Rafael's heuristics, indexing more than 1 year of email history per user degraded query performance. Regardless, the write throughput bottleneck meant they couldn't even test more data without risking further instability.

These problems directly shaped what the Superhuman Mail team felt they could build. They were rationing their own product, limiting what they indexed and which features they shipped based on what their search infra could tolerate.

why turbopuffer?

With turbopuffer, Superhuman Mail has shed the artificial limits that constrained their product ambition.

They've increased indexed email history from 1 year to 5 years per user, and they're already planning to index more. turbopuffer's write path separation means indexing load doesn't affect query performance, regardless of how much data they add. Unlimited namespaces remove the need to shuffle indexes into cache.

Superhuman Mail also consolidated email classification and auto-archive features onto turbopuffer. These had been running on Postgres, not because Postgres was the best tool for the job, but because the previous vector database was too unreliable to take on additional load. These run alongside Ask AI with no impact on search performance and minimal operational overhead.

The migration to turbopuffer took less than a day, and Superhuman Mail was able to migrate over 3 billion emails without a re-embedding (which would have cost over $300k).

The concern is never whether turbopuffer can handle it — it's whether we can build fast enough.

Rafael Melo Cardoso, Engineering Manager

Results

• 1 → 5 years of indexed email history per user, with capacity for more
• 3B+ embeddings migrated in a single day — "it just worked"
• 20%+ lower apples-to-apples cost versus other providers
• 60ms p90 latency across 9B+ emails and hundreds of thousands of inboxes
• 97%+ recall@10

turbopuffer in Superhuman Mail

Superhuman Mail uses a "namespace-per-inbox" approach, indexing 5+ years of emails for each inbox so users get fast, comprehensive search over their email history.

Superhuman Mail builds both vector and full-text search indexes, with filterable attributes such as timestamp, hasAttachments, sender, etc. They use turbopuffer's multi-query API to execute hybrid search queries with client-side LLM re-ranking for fast search and high recall.

New emails are immediately returnable in search with turbopuffer's strong consistency guarantees.

turbopuffer's indexing path separation ensures query performance and consistency aren't impacted by indexing operations. Only actively searched inboxes are loaded into cache, while the indexes for inactive inboxes remain on low-cost object storage.

When a user opens the Ask AI window, Superhuman Mail pre-warms the index, loading it into cache for fast queries without needing to evict another inbox.

Namespace routing is derived from the user's auth token, not defined within the application logic. Cross-tenant data access is impossible by design - important for emails, which contain private and sensitive info. All data in turbopuffer is encrypted at rest with AES-256.

What's next

The confidence that spurred Superhuman Mail to 5x their indexed email history is now driving an expanding roadmap:

• Even more email history: Expanding beyond 5 years of indexed emails per user.
• Attachment content: Currently only attachment filenames are indexed. Embedding full attachment content and internal metadata (e.g. comments) is next
• Calendar: Superhuman Mail plans to use turbopuffer as the search engine for their calendar product
• Team namespaces: Adding shared namespaces for team-level data alongside per-inbox namespaces, enabling features like cross-inbox search

We will continue to update this log as Superhuman Mail adds more features on turbopuffer.