Query documents

POST /v1/vectors/:namespace/query

Searches documents.

Latency

1M docs

Percentile

Latency

p50
28ms
p90
37ms
p99
63ms
MAX
98ms

Parameters

vector array[float]

The vector to search for. Optional, if not set, this endpoint performs a filter-only search.

It must have the same dimensions as the vectors in the namespace.

Example: [0.1, 0.2, 0.3, ..., 76.8]


rank_by array

Fields to rank by for BM25 full text search.

Currently, you can only rank by either BM25 or vector search. For hybrid search, you must do two queries and combine the results client-side. We will expose primitives to combine results server-side in the future.

The attribute must be configured to be BM25 searchable.

Example: ["text", "BM25", "fox jumping"]

Example with multiple fields:

["Sum", [["text", "BM25", "fox jumping"], ["name", "BM25", "fox jumping"]]]


distance_metric stringrequired if vector is set

The function used to calculate vector similarity. Possible values are cosine_distance or euclidean_squared.


top_k numberdefault: 10

Number of results to return.


include_vectors booleandefault: false

Return vector values in the response.


include_attributes array[string] | booleanoptional

List of attribute names to return in the response. Can be set to true to return all attributes.


filters object | arrayoptional

Filters provide a way to refine search results by matching against attributes.

For the best performance, prefer to separate documents into namespaces instead of filtering where possible.

See Filtering Parameters below for details.

Examples

Simple Query

// Request payload
{
  "vector": [0.1, 0.1],
  "distance_metric": "euclidean_squared",
  "top_k": 10
}

// Response payload
[
  {
    "dist": 0.0,
    "id": 1
  },
  {
    "dist": 2.0,
    "id": 2
  },
  ...
]

With Attributes

By default, only the distance and ID are returned.

If include_vectors is true, the associated vector is included in the vector key.

If include_attributes contains any attribute names, those attributes are included in the attributes key.

// Request payload
{
  "vector": [0.1, 0.1],
  "distance_metric": "euclidean_squared",
  "top_k": 10,
  "include_vectors": true,
  "include_attributes": ["key1"]
}

// Response payload
[
  {
    "dist": 0.0,
    "id": 1,
    "vector": [0.1, 0.1],
    "attributes": {"key1": "one"}
  },
  {
    "dist": 2.0,
    "id": 2,
    "vector": [0.2, 0.2],
    "attributes": {"key1": "two"}
  },
  ...
]

Filtering Parameters

Filters allow you to narrow down results by applying conditions to attributes. Conditions are arrays with an attribute name, operation, and value, for example:

  • ["attr_name", "Eq", 42]
  • ["page_id", "In", ["page1", "page2"]]

Values must have the same type as the attribute's value, or an array of that type for operators like In.

Conditions can be combined using And/Or operations:

// basic And condition
"filters": ["And", [
  ["attr_name", "Eq", 42],
  ["page_id", "In", ["page1", "page2"]]
]]

// conditions can be nested
"filters": ["And", [
  ["page_id", "In", ["page1", "page2"]],
  ["Or", [
    ["public", "Eq", 1],
    ["permission_id", "In", ["3iQK2VC4", "wzw8zpnQ"]]
  ]]
]]

// legacy API: an object may provided instead (implicitly And)
"filters": {
  "attr_name": ["Eq", 42],
  "page_id": ["In", ["page1", "page2"]],
}

Filters can also be applied to the id field, which refers to the document ID. If filtering by id, the filter operation must be Eq with a number value, or In with an array of numbers.

A filter on the id field will be evaluated as a pre-filter; i.e. the documents will be fetched by ID, then other filters and the kNN search will be applied.

Full list of operators:

Eq id or value

Exact match for id or attributes values.


NotEq value

Inverse of Eq, for attributes values.


In array[id] or array[value]

Matches any id or attributes values contained in the provided list. If both the provided value and the target document field are arrays, then this checks if any elements of the two sets intersect.


NotIn array[value]

Inverse of In, matches any attributes values not contained in the provided list.


Lt value

For ints, this is a numeric less-than on attributes values. For strings, lexicographic less-than.

Lte value

For ints, this is a numeric less-than-or-equal on attributes values. For strings, lexicographic less-than-or-equal.

Gt value

For ints, this is a numeric greater-than on attributes values. For strings, lexicographic greater-than.

Gte value

For ints, this is a numeric greater-than-or-equal on attributes values. For strings, lexicographic greater-than-or-equal.


Glob globset

Unix-style glob match against attributes values. The full syntax is described in the globset documentation.

NotGlob globset

Inverse of Glob, Unix-style glob filters against attributes values. The full syntax is described in the globset documentation.

IGlob globset

Case insensitive version of Glob.

NotIGlob globset

Case insensitive version of NotGlob.


And array[filter]

Matches if all of the filters match.

Or array[filter]

Matches if at least one of the filters matches.

Filtering Example

{
  "vector": [0.1, 0.1],
  "distance_metric": "euclidean_squared",
  "top_k": 10,
  "include_attributes": ["key1"],
  "filters": [
    "And",
    [
      ["id", "In", [1, 2, 3]],
      ["key1", "Eq", "one"],
      ["filename", "NotGlob", "/vendor/**"],
      [
        "Or",
        [
          ["filename", "Glob", "**.tsx"],
          ["filename", "Glob", "**.js"]
        ]
      ]
    ]
  ]
}

Filter-Only Search

When querying without passing vector or rank_by fields, this endpoint performs a filter-only search. This mode of operation allows pagination over the contents of a namespace, and supports filtering.

This can be used to delete by filter, by subsequently deleting IDs returned from the search.

Results are sorted in ascending ID order, which allows pagination with an ID greater-than filter. For example,

  • First request: {"top_k": 10, "filters": ["foo", "Eq", "bar"]}
    • Returns first 10 IDs: [{"id": 3}, ..., {"id": 14}]
  • Second request: {"top_k": 10, "filters": ["And", [["foo", "Eq", "bar"], ["id", "Gt", 14]]]}
    • Returns next 10 IDs: [{"id": 15}, ...]

Caveats:

  • Supports a limited number of filters, specifically, only Eq, In, And and Or filters (for any non-ID field)
© 2024 turbopuffer Inc.
Privacy PolicyTerms of service