TELUS indexes 25,000+ AI copilots on turbopuffer

With turbopuffer, there’s no scaling wall. It’s infinite.

Justin Watts, Distinguished Engineer

TELUS is one of Canada’s largest tech companies, with services for consumer telecom, enterprise networking, agriculture, and healthcare. TELUS thinks forward; their enterprise scale belies their progressive engineering culture.

In April 2024, TELUS announced Fuel iX™, a first-of-its-kind generative AI engine for Canadian enterprises. Fuel iX promises to help companies “upgrade their GenAI capabilities from pilot to production” with customized AI chatbots tailored to specific employees and roles.

Why turbopuffer? Unlimited namespaces

Before turbopuffer, Justin Watts, a distinguished engineer at TELUS, tried five different vector databases for chatbot RAG. All of them failed to satisfy the primary need: massive multi-tenancy.

Traditional vector DBs build the index in memory, so most providers max out indexes in the 10s or 100s to keep costs reasonable. Justin needed the capacity to build hundreds of thousands of personalized chatbots each with its own RAG index.

With other providers, they couldn’t scale indexes without scaling everything. To get breadth, they had to buy depth: more boxes, more compute, more RAM. The cost to do so was, in Justin’s words, “silly.”

Justin considered a move to Postgres with RAM + disk caching. It would have worked for their initial scale, but they would have hit limits within a few months and needed to aggressively shard or rebuild, again.

What they needed: a serverless search engine with unlimited indexes, fast queries on warm data, cheap storage for everything else.

turbopuffer’s architecture builds indexes on object storage and caches only warm namespaces. turbopuffer does not enforce namespace limits, so TELUS can create a unique namespace for every personalized chatbot, each with its own vector index. When the namespace is warm (it can be pre-warmed), query latency competes with RAM-based peers. When cold, storage is cheap and infinitely scalable.

Fuel iX doesn’t exist without turbopuffer. If we used a traditional vector db provider, our bill would be in the millions each year.

Justin Watts, Distinguished Engineer

Results

TELUS migrated to turbopuffer in less than 30 minutes, with zero downtime. With turbopuffer they’ve indexed millions of vectors across tens of thousands of namespaces, and counting.

1. <100 ms p99 latency across 25,000 unique namespaces
2. ~3 orders of magnitude less expensive than other vector database providers
3. TELUS reports over $90M in benefits from 57K+ internal employees using Fuel iX

turbopuffer in Fuel iX

Over 57,000 TELUS employees use personalized chatbots on Fuel iX with “drag-and-drop RAG” that allows them to upload their context (documents, images) into the copilot UI. For each copilot, Fuel iX vectorizes uploaded documents and builds a RAG index on a unique turbopuffer namespace, so each employee’s chat context is fine-tuned to their function.

Because each namespace has its own prefix in GCS, they don’t have to worry about perfecting security policies. User data is inherently isolated, and cross-contamination simply isn’t possible.

Next up for TELUS

TELUS is expanding Fuel iX beyond just internal employees, offering it as a SaaS to enterprises across Canada. turbopuffer gives Justin and the TELUS team confidence to infinitely scale namespaces at the lowest possible cost.