# Embedding

Native embeddings are in private beta. Please [reach out to us](/contact/sales)
if you're interested. We're actively seeking new users.

Native embeddings convert your documents and queries into vectors as you read
from and write to turbopuffer. You don't need to write any code to integrate
with an embedding provider. Instead, turbopuffer handles embedding your data on
writes and queries.

Native embeddings are stored as a vector alongside the data that was embedded.
You can read and export the source data and the native embedding vectors like
you can any other field. You always own the data you store in turbopuffer.

To enable native embedding, use the `embed` property on an attribute's schema to
specify an embedding model. Writes and queries to that attribute are turned into
vectors on the fly.

<!-- multilang -->
```typescript
import { Turbopuffer } from "@turbopuffer/turbopuffer";

const tpuf = new Turbopuffer({
  region: "gcp-us-central1", // pick a region: https://turbopuffer.com/docs/regions
});

// create a new namespace with native embeddings
const ns = tpuf.namespace(`embedding-intro-ts`);
await ns.write({
  upsert_rows: [
    {
      id: 1,
      text: "A cat sleeping on a windowsill",
      category: "animal",
    },
    {
      id: 2,
      text: "A playful kitten chasing a toy",
      category: "animal",
    },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: {
      type: "string",
      // use native embeddings by setting the `embed` property on a text
      // attribute to your favorite model
      embed: "example/random",
    },
  },
});
```
<!-- /multilang -->

## Pricing

Embedding is billed based on the number of tokens embedded by a model. How your
document translates into tokens is model dependent. A rough rule of thumb is
that 4 bytes of English text is about 1 token. The number of tokens embedded is
included in both the [query](/docs/query#responsefield-performance) and
[write](/docs/write#responsefield-performance) responses.

See the [list of available models](/docs/embedding#models) for pricing.

## Limitations

Native embeddings have a few limitations. Like our [other limits](/docs/limits),
there is nothing here that we can't improve if prioritized. If your native
embeddings use case is blocked by one of these limits, [contact us](/contact).

- Native embeddings always store the source document in turbopuffer, along with
  the embedding vector.

- You're constrained by our rate limits. We work tirelessly with our partners to
  make sure turbopuffer can embed as fast as it can index, but there may be
  situations where we have to rate-limit and return 429s. We keep a close eye on
  our capacity and scale up within and across providers to make sure you can
  keep puffin'.

- Setting `embed` on an attribute does not yet automatically re-embed it with
  the specified model. Only subsequent writes are affected, so if you decide to
  change your embedding model you have to re-ingest the data in your namespaces.

- Multi-modal support is a work-in-progress. There is no dedicated
  [type](/docs/write#param-type) for image or video data, so to use multi-modal
  embeddings your source data must be stored as a base64 encoded `string`
  attribute.

- We don't yet support contextual models that chunk their inputs and produce a
  variable number of vectors. We want to get the API right before we make them
  available.

## Models

The following models are currently available as managed embedding models. If you
see a model you'd like to use that isn't listed here, please [get in touch](/contact).

Prices are subject to change while embedding is still in private beta. A model
listed here without a price is still available, but pricing is TBD. We'll always
try to work with our partners to price match your existing embedding provider.

### Recommendations

Picking an embedding model can deeply impact the quality, cost, and latency of
your retrieval pipeline. Unfortunately, it's both a difficult decision and one
that's hard to change (though we're working on making it easy). The quality of
an embedding model is highly dependent on exactly what you're searching over.
The best way to choose an embedding model is to [run an
eval](https://georgianailab.substack.com/p/evaluating-retrieval-without-ground)
that measures how an embedding model affects your application. If you can't do
that, the [RTEB](https://huggingface.co/spaces/embedding-benchmark/RTEB)
scores in the table below are an ok guide.

When picking a model, we recommend thinking about the quality, cost, and latency
tradeoffs in your application. It may be tempting to always pick the "best"
model, but if you need interactive search your users may be more sensitive to
latency than they are to slightly worse recall. Conversely, if you're powering a
search agent it may be better to spend more up-front on an embedding model to
reduce the number of tokens the agent needs to answer questions. 

The models below are grouped into `S`, `M`, and `L` categories that roughly
represent the number of parameters in a model and how long it takes to embed a
document. `S` models prioritize latency over quality and are ideal for
interactive applications that prioritize speed. `L` models maximize quality at
the expense of latency, and are ideal for applications where you're happy to
wait for high quality results. `M` models are somewhere in between.

### Model list

Click on a model for more detailed information. 

- Context length is the maximum number of input tokens a model will consider
  in an individual embedding input.

- The dimension marked `(default)` is the model's default output dimension.

- The listed [RTEB](https://huggingface.co/spaces/embedding-benchmark/RTEB)
  score is the overall score. 

If you're interested in using a model not available in the list below [contact us](/contact).

## Examples

### Creating a new namespace

Create a new namespace with native embeddings on a `text` attribute.

<!-- multilang -->
```typescript
import { Turbopuffer } from "@turbopuffer/turbopuffer";

const tpuf = new Turbopuffer({
  region: "gcp-us-central1", // pick a region: https://turbopuffer.com/docs/regions
});

// create a new namespace with native embeddings
const ns = tpuf.namespace(`embedding-new-namespace-ts`);
await ns.write({
  upsert_rows: [
    {
      id: 1,
      text: "A cat sleeping on a windowsill",
      category: "animal",
    },
    {
      id: 2,
      text: "A playful kitten chasing a toy",
      category: "animal",
    },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: {
      type: "string",
      // use native embeddings to set an embedding model and the size
      // of the embedding vector
      embed: {
        model: "example/random",
        dims: 512,
      }
    },
  },
});

// future writes don't need to specify the schema or the `embed_text` attribute.
// the text attribute is turned into a vector on every write.
await ns.write({
  upsert_rows: [
    {
      id: 3,
      text: "An airplane flying through clouds",
      category: "vehicle",
    },
    {
      id: 4,
      text: "A shiny red sports car",
      category: "vehicle",
    },
  ]
})

// use the Embed function with ANN or kNN to turn text input into a vector
await ns.query({
  include_attributes: ["id", "text", "embed_text"],
  rank_by: ["text", "ANN", ["Embed", "something high energy"]],
  limit: 10,
});
```
<!-- /multilang -->

### Query-only native embeddings

If you've already invested in embedding a large batch of documents, you can
ingest them into turbopuffer and use native embeddings to query them.

<!-- multilang -->
```typescript
import { Turbopuffer } from "@turbopuffer/turbopuffer";

const tpuf = new Turbopuffer({
  region: "gcp-us-central1", // pick a region: https://turbopuffer.com/docs/regions
});

function embed(_text: string): number[] {
  return [Math.random(), Math.random()];
}

// creating a new namespace that uses your own embedding model to embed
// documents, and write a few documents into the namespace.
const ns = tpuf.namespace(`embedding-query-only-ts`);
await ns.write({
  upsert_rows: [
    {
      id: 1,
      vector: embed("walrus narwhal"),
      public: true,
      text: "walrus narwhal",
    },
    {
      id: 2,
      vector: embed("pufferfish clownfish swordfish"),
      public: false,
      text: "pufferfish clownfish swordfish",
    },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: { type: "string", full_text_search: true, regex: true },
  },
});

// use the Embed function with an explicit model to let native embeddings handle
// the query embeddings. this should match the model you're using to embed
// documents or you might get some strange results.
await ns.query({
  include_attributes: ["id", "text", "vector"],
  rank_by: ["vector", "ANN", ["Embed", "something high energy", {model: "example/random"}]],
  limit: 10,
});
```
<!-- /multilang -->

### Migrating to native embeddings 

To migrate an existing namespace to native embeddings, first switch your
application's reads to use native embeddings [on
query](/docs/embedding#query-only-native-embeddings) and then switch writes by
[updating the `embed` attribute](/docs/write#updating-attributes) on the
attribute to embed. Future writes to that namespace must not provide that
vector, and future ANN queries can use the `Embed` function to rank by the
source field directly. 

Embedding an existing attribute into an existing vector does check that the
embedding model supports both the shape and datatype of the existing vector, but
**does not** check that the vectors semantically make sense and **does not**
re-embed the namespace. Before enabling native embedding on a namespace,
validate that the model already in use is compatible with the native embedding
model. 

<!-- multilang -->
```typescript
import { Turbopuffer } from "@turbopuffer/turbopuffer";

const tpuf = new Turbopuffer({
  region: "gcp-us-central1", // pick a region: https://turbopuffer.com/docs/regions
});

function random_vector(_text: string): number[] {
  return [Math.random(), Math.random()];
}

// create a new namespace without native embeddings
const ns = tpuf.namespace(`embedding-enable-ts`);
await ns.write({
  upsert_rows: [
    {
      id: 1,
      vector: random_vector("walrus narwhal"),
      text: "walrus narwhal",
    },
    {
      id: 2,
      vector: random_vector("pufferfish clownfish swordfish"),
      text: "pufferfish clownfish swordfish",
    },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: { type: "string", full_text_search: true, regex: true },
  },
});

// switch your queries to use query-time embedding
await ns.query({
  include_attributes: ["id", "text", "vector"],
  rank_by: ["vector", "ANN", ["Embed", "a pointy fish", {model: "example/random"}]],
  limit: 10,
});

// update your schema in-place to enable native embeddings
await ns.write({
  schema: {
    text: {
      type: "string",
      embed: {
        model: "example/random",
        attribute: "vector",
      },
    },
  },
});

// queries no longer need to specify a model
await ns.query({
  include_attributes: ["id", "text", "vector"],
  rank_by: ["text", "ANN", ["Embed", "lots and lots of blubber"]],
  limit: 10,
});

// and writes no longer need to include a vector
await ns.write({
  upsert_rows: [
    {
      id: 3,
      text: "zebra horse antelope",
    },
  ]
});
```
<!-- /multilang -->

### Disabling native embeddings 

To disable embedding, set `embed` to `null` on an attribute. After the write
succeeds, any future writes will require passing an explicit vector and any
future ANN queries will require passing a vector to `rank_by`. Any embedded
columns, including automatically generated embedding columns, are untouched
and can be queried as normal.

<!-- multilang -->
```typescript
import { Turbopuffer } from "@turbopuffer/turbopuffer";

const tpuf = new Turbopuffer({
  region: "gcp-us-central1", // pick a region: https://turbopuffer.com/docs/regions
});

function random_vector(_text: string, n: number = 8): number[] {
  return Array.from({ length: n }, () => Math.random());
}

// create a new namespace with native embeddings
const ns = tpuf.namespace(`embedding-disable-ts`);
await ns.write({
  upsert_rows: [
    {
      id: 1,
      text: "A cat sleeping on a windowsill",
      category: "animal",
    },
    {
      id: 2,
      text: "A playful kitten chasing a toy",
      category: "animal",
    },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: {
      type: "string",
      embed: {
        model: "example/random",
        dims: 8,
      },
    },
  },
});

// disable native embeddings
await ns.write({
  schema: {
    text: {
      type: "string",
      embed: null,
    },
  },
});

// any future write to this namespace requires writing the embed_text vector
await ns.write({
  upsert_rows: [
    {
      id: 1,
      embed_text: random_vector("zebra horse antelope"),
      text: "zebra horse antelope",
    },
  ],
});
```
<!-- /multilang -->

## Security

We work with external partners to provide access to a wide range of embedding
models. When you use automatic embedding, your data is sent from our cloud
account to one of our embedding partners. The exact list of partners is model
dependent, and listed on our [subprocessors](/docs/security#subprocessors-for-customer-data)
page.


---

This page: [/docs/embedding.md](https://turbopuffer.com/docs/embedding.md)

All documentation pages: [/llms.txt](https://turbopuffer.com/llms.txt)

All documentation in one file: [/llms-full.txt](https://turbopuffer.com/llms-full.txt)