# Sizing and Capacity Planning turbopuffer gets faster the more of your data it keeps in cache. Query nodes hold namespaces on NVMe SSD and in memory, in front of object storage. Add query nodes and more of your working set stays cached, so latency drops. Run fewer and you spend less, while queries to evicted data read object storage and take longer. How much to spend is your call. The turbopuffer team can recommend a starting size and help you tune it from telemetry, and you can scale up or down at any time. A BYOC cluster runs two kinds of turbopuffer nodes: - **Query nodes** serve queries and writes from an NVMe SSD and memory cache in front of object storage. - **Index nodes** run background jobs: indexing, WAL compaction, and other maintenance. Each query and index pod gets a dedicated node, so the pod count is the node count. Router pods run on query nodes, and their count follows `min_query_nodes`. Node pools autoscale with Karpenter on AWS and Azure and the cluster autoscaler on GCP, so nodes are created and removed as pods are scheduled. You choose the pod counts. Index nodes autoscale on their own, while query nodes stay at the count you set. ## Minimums and maximums Your BYOC kit sets the node floors in `turbopuffer.tfvars`, and `terraform apply` writes them to `values.yaml` as `turbopuffer.min_replicas`: ```hcl min_query_nodes = 6 min_index_nodes = 1 ``` There is no single right minimum. For most clusters, `min_index_nodes = 1` is enough and the autoscaler adds index nodes when needed. Raise it only if you want capacity guaranteed rather than waiting for the cloud provider to supply nodes. The maximum number of index nodes comes from the node pool limits in Terraform (10 per zone on AWS and Azure, 128 on GCP). Don't raise `max_replicas` in Helm alone, since pods beyond the node pool limit can't be scheduled. Check that your cloud quota covers the maximum node count for both pools, plus any [pinned](/docs/byoc/pinning) nodes. ## Query nodes There is no autoscaler for query nodes, since adding and removing query nodes moves namespaces between caches and causes cache thrashing. The query StatefulSet stays at the size you set, and upgrades keep its current size. Raising `min_query_nodes` scales up on the next deploy. The turbopuffer team can also scale query nodes through the [control plane](/docs/byoc/operations#supported-operations). Each namespace is routed to one query node so that it stays in that node's cache. Size query nodes so that: - **The working set fits in cache.** The combined NVMe cache of all query nodes should hold the namespaces you query regularly, with headroom for growth. If it doesn't, namespaces are evicted and the next query to them reads object storage and is slow. - **CPU has headroom at peak.** Plan for peak QPS, not average QPS. - **Hot namespaces have room.** A single namespace with very high QPS is limited by one node. Spread it across more nodes with [`ingress.read_replicas`](/docs/byoc/configuration#param-ingress.read_replicas), or give it dedicated nodes with [pinning](/docs/byoc/pinning). Add query nodes when the following widgets on the BYOC Datadog dashboard trend the wrong way: | Widget | What to look for | |---|---| | **NVMe cache age (query)** | Age dropping over time. Data is evicted sooner, so the cache is too small for the working set. | | **Memory cache max evicted age (query)** | Same signal for the memory cache. | | **Storage cache hit ratio (query pods)** | Ratio dropping, so more queries read object storage. | | **Total CPU usage by query host (%)** | Hosts close to saturation at peak. A single hot host points to a hot namespace rather than too few nodes. | | **Query latency over time** | Rising p90 and p99. | | **tpuf 408,429,499 responses (queries)** | 429s and 408s as query concurrency or timeouts are hit. | | **Query pods** | Ready and not ready replicas, to confirm a scale-up has finished. | ## Index nodes Index nodes autoscale with a Kubernetes HPA on the `turbopuffer-index-0` Deployment. The HPA scales on queue delay: the age of the oldest job waiting to be picked up, with a target of about 4 minutes. If the oldest job has waited longer than the target, the HPA adds nodes in proportion. When the queue is empty, it scales down to the number of nodes needed for jobs that are still running, and never below `min_index_nodes`. Index nodes don't scale to 0, since compaction, garbage collection, and recall jobs also run on them. Scale-up adds at most 1 node per minute. Scale-down waits 3 minutes and then removes at most 3 nodes every 2 minutes. The autoscaler handles bursts such as backfills, but because it adds about 1 node per minute a large backfill takes a while to reach full capacity. If desired index pods rarely rise above `min_index_nodes`, the floor is higher than your workload needs and you can lower it. Data that hasn't been indexed yet is still searched, more slowly. If a namespace falls more than 2 GB or 1 million rows behind, writes to it return HTTP 429 until indexing catches up (see [`disable_backpressure`](/docs/write#param-disable_backpressure)). Check: | Widget | What to look for | |---|---| | **Indexing Job Max Queue Time** | Jobs waiting well past ~4 minutes for long periods. | | **Unindexed bytes** | Unindexed data per namespace that grows instead of draining. | | **Indexing autoscaling threshold** | Values above 1 mean the HPA wants more index nodes. | | **Index pods** | Desired and ready replicas. Desired replicas flat at the maximum mean the limit is too low, and desired replicas that never rise above the minimum mean the floor is too high. | | **Insufficient Capacity Errors from Cloud** | The cloud provider can't supply nodes of the requested type in your zones. | | **Total CPU usage by index host (%)** | Index node utilization. | | **Jobs waiting** and **Jobs running** | Queue depth and running jobs by queue. | --- This page: [/docs/byoc/sizing.md](https://turbopuffer.com/docs/byoc/sizing.md) All documentation pages: [/llms.txt](https://turbopuffer.com/llms.txt) All documentation in one file: [/llms-full.txt](https://turbopuffer.com/llms-full.txt)