Introduction
╔═ turbopuffer ═════════════════════════════╗
╔════════════╗ ║ ║░
║ ║░ ║ ┏━━━━━━━━━━━━━━━┓ ┏━━━━━━━━━━━━━━┓ ║░
║ client ║░───API──▶║ ┃ Memory/ ┃────▶┃ Object ┃ ║░
║ ║░ ║ ┃ SSD Cache ┃ ┃ Storage (S3) ┃ ║░
╚════════════╝░ ║ ┗━━━━━━━━━━━━━━━┛ ┗━━━━━━━━━━━━━━┛ ║░
░░░░░░░░░░░░░░ ║ ║░
╚══════════════════════════════════════════╝░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
╔════════════╗
║ ║░
┌──║ client ║░
│ ║ ║░
API ╚════════════╝░
│ ░░░░░░░░░░░░░░
└─────────┐
│
▼
╔═ turbopuffer ════════════════╗
║ ║░
║ ┏━━━━━━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃ Memory/ ┃ ║░
║ ┃ SSD Cache ┃ ║░
║ ┗━━━━━━━━━━━━━━━━━━━━━━━━┛ ║░
║ │ ║░
║ ▼ ║░
║ ┏━━━━━━━━━━━━━━━━━━━┓ ║░
║ ┃Object storage (S3)┃ ║░
║ ┗━━━━━━━━━━━━━━━━━━━┛ ║░
║ ║░
╚══════════════════════════════╝░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░Diagram of turbopuffer's architecture
turbopuffer is a search engine that lets you store and search over both high-dimensional vectors and text.
Separating storage and compute allows it to scale in response to load, without complex data migration or replication techniques. turbopuffer commits ingested data directly to highly reliable, low-cost object storage.
turbopuffer uses a write-ahead log to ensure consistency. After data is committed to the log, it is asynchronously indexed to enable efficient retrieval. Any data that has not yet been indexed is still available to search, with a slower exhaustive search of recent data in the log. Read more about turbopuffer's architecture.
To start hacking with turbopuffer on the command line, see the quickstart guide.
To get a feel for what you can do with turbopuffer in a Python environment, see the demo notebook.
