Common Operations

While turbopuffer strives to be as low interaction as possible, there are certain manual operations you will have to perform in your BYOC deployment.

Securely partitioning your data

turbopuffer BYOC allows you to configure multiple organizations, each with their own set of API keys which you can use to scope data access. Currently, we only support creating admin API keys, that will apply to all namespaces in their organization. For this reason, if you need to ensure data is isolated we recommend creating multiple organizations instead. If this is a limitation, we recommend you contact us on Slack.

Generating org IDs and API keys

You can generate valid org IDs and API keys using any tooling that produces cryptographically random values. The format requirements are:

Org ID

  • 24 character random string
  • Alphabet: [a-z0-9] (lowercase alphanumeric only)

API Key

  • Prefix: tpuf_
  • Followed by: 32 character random string from alphabet [a-zA-Z0-9]
  • Full format: tpuf_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Stored API Key Hash

  • Format: base64(sha256(full_api_key))
  • The hash is computed over the complete API key including the tpuf_ prefix

Your BYOC kit includes a generate_secrets.py script that generates these values for you.

Storing org configuration

authentication:
  allowed_api_keys_sha256:
    "your24charorgidhere1234":
      - "YourBase64EncodedSHA256HashHere+40CharactersTotal="
      - "AnotherBase64EncodedSHA256HashHere+40CharactersTot="

Adding additional organizations

To add more organizations, generate new org ID and API key pairs and add them to the configuration:

authentication:
  allowed_api_keys_sha256:
    "existingorgid123456789012":
      - "ExistingOrgKeyHash..."
      - "AnotherExistingOrgKeyHash..."
    "neworgid0987654321abcdef":
      - "NewOrgKeyHash..."
      - "AnotherNewOrgKeyHash..."

Adding a new API key to an existing organization

Each organization can have multiple API keys for key rotation, different services, or other access patterns. To add a new API key, use the apikey.py script from your BYOC kit and append the generated hash to the organization's key list:

authentication:
  allowed_api_keys_sha256:
    "existingorgid123456789012":
      - "ExistingOrgKeyHash..."
      - "NewlyAddedKeyHash..."

Bring your own bucket

You may want to store a customer's data in a bucket that the customer owns, often called bring your own bucket (BYOB). This is configured by mapping the customer's org to a dedicated bucket with blob.bucket_overrides_by_org_id.

The bucket can live in a different GCP project than the turbopuffer cluster. Grant the cluster's service account roles/storage.objectAdmin on the bucket.

blob:
  bucket_overrides_by_org_id:  # org ID -> bucket ID
    "m3vjrpgnvlcm5x8olkguh1l2": customer-b-bucket
  buckets:
    customer-b-bucket:
      bucket_url: "https://customer-b-turbopuffer.storage.googleapis.com"
      provider: gcp-workload-identity

Customer managed encryption keys (CMEK)

Namespaces can be encrypted with a customer managed key. In BYOC the cluster's own identity accesses the key, so there is no turbopuffer service account to grant.

  1. Create a symmetric Cloud KMS key in the same region as the bucket.

  2. Grant roles/cloudkms.cryptoKeyEncrypterDecrypter on the key to the Cloud Storage service agent of the project that owns the bucket. Cloud Storage encrypts and decrypts objects with this service agent, not with the cluster's service account.

    # Prints the service agent email, e.g. service-123456789012@gs-project-accounts.iam.gserviceaccount.com
SERVICE_AGENT=$(gcloud storage service-agent --project BUCKET_PROJECT_ID)

gcloud kms keys add-iam-policy-binding KEY --keyring KEYRING --location REGION \
  --member "serviceAccount:$SERVICE_AGENT" \
  --role roles/cloudkms.cryptoKeyEncrypterDecrypter

  3. Set blob.cmek_enabled to true and apply the configuration change. The flag is per bucket. If the org's data lives in a bucket under blob.buckets (see bring your own bucket), set it on that entry, since entries do not inherit the top-level value:

    blob:
  cmek_enabled: true  # default bucket
  buckets:
    customer-b-bucket:
      bucket_url: "https://customer-b-turbopuffer.storage.googleapis.com"
      provider: gcp-workload-identity
      cmek_enabled: true  # required for orgs mapped to this bucket

  4. Write with encryption set to the key resource name.

Copying namespaces across clusters

You can use copy_from_namespace to copy data between BYOC clusters. On the destination cluster:

  1. If the clusters use different org IDs, add an API key hash under the source's org ID in authentication.allowed_api_keys_sha256. The org ID must match the data in the source bucket. You can register an existing source key or generate a new one. The destination cluster authenticates this key locally.
  2. Define the source bucket under blob.buckets and map a source region name to it with blob.known_region_buckets. Match the source bucket's prefix, if configured. Grant the destination cluster's cloud identity read access to the source bucket and any required encryption keys.
  3. On the source bucket entry, set enable_maintenance: false to disable automatic maintenance, including garbage collection. Leave maintenance enabled on the source cluster. Set read_only: true to prevent writes to the source bucket from the destination cluster.

Apply the changes using the Helm upgrade command, then send a copy_from_namespace request to the destination cluster. The destination namespace must not already exist:

curl "$DESTINATION_URL/v2/namespaces/destination-namespace" \
  -H "Authorization: Bearer $DESTINATION_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "copy_from_namespace": {
      "source_namespace": "source-namespace",
      "source_region": "source-cluster",
      "source_api_key": "<source API key registered on the destination cluster>"
    }
  }'

DESTINATION_API_KEY identifies the destination org. source_region must match a key in blob.known_region_buckets and selects the source bucket for this copy. source_api_key identifies the source org. Omit it when both clusters use the same org ID. The region mapping does not change the bucket used for ordinary requests.

Do not configure a blob.bucket_overrides_by_org_id entry for the source org on the destination cluster. Combining a source org override with source_region is currently unsupported.

If the source namespace uses a customer-managed encryption key, specify destination encryption in the copy request.

Creating additional clusters

You can create additional clusters by reusing your existing BYOC kit and control plane API key. Follow the deployment guide again with a unique cluster_name in values.yaml, and generate new turbopuffer org IDs and API keys for the new cluster with scripts/generate-secrets.py.

Applying configuration changes

After updating the configuration, apply the changes using the Helm upgrade command.