Published benchmark report
Operational throughput, vector recall, and full-text relevance—read each result with its workload, hardware, and measurement boundary.
Read the boundary first
An HTTPS result includes network time. An in-process result does not. Compare workloads within their stated conditions.
The reference engine used substrate compression and a bounded memory guard. The in-region runs use separate production configurations. These published results do not establish an unthrottled engine ceiling or a service-level guarantee.
01 / Operational workloads
Five reported workloads: A, B, C, D, and F. Zipfian access on a 20K-row table, with 50,000 operations across the workloads.
| Workload | Mix | ops / sec | p50 (ms) | p95 (ms) | p99 (ms) | errors |
|---|---|---|---|---|---|---|
| A | 50% read / 50% update | 292 | 104 | 143 | 202 | 0 |
| B | 95% read / 5% update | 321 | 92 | 120 | 158 | 0 |
| C | 100% read | 146 | 125 | 557 | 1220 | 0 |
| D | 95% read latest / 5% insert | 338 | 92 | 119 | 138 | 0 |
| F | read-modify-write | 215 | 165 | 225 | 252 | 0 |
All five reported workloads recorded zero errors and no rate-limit hits. Workload E is not included in this dataset.
The same YCSB driver ran from a machine in the engine’s region against production-configuration tenants. The notes below are retained from the published run.
| Setup | ops / sec | p99 (ms) | Note |
|---|---|---|---|
| 4-vCPU node | 1,046 | - | at its provisioned ~1,000 req/s cap |
| 8-vCPU node | 1,384 | 117 | single client; the driver is the limit, not the engine |
| 8-vCPU node, 4-client fan-out (read) | 4,940 | 173 | 0 errors; the wall at ~5,000 is the provisioned cap, not an engine ceiling |
The four-client row reports read traffic. A dash means p99 was not supplied for that row. Provisioned rate limits and client concurrency are part of this setup.
python benchmarks/ycsb/run.py --record-count 20000 --op-count 10000 --workloads C,B,A,D,F02 / Vector retrieval
SIFT 128-dimensional vectors, a 100K-vector subset, and 1,000 queries. HNSW with M=16 and ef_construction=200, using a single-threaded AVX2 kernel.
| ef_search | recall@10 | QPS | p50 (µs) | p99 (µs) |
|---|---|---|---|---|
| 10 | 0.286 | 187.9 | 5,208 | 8,170 |
| 20 | 0.115 | 154.8 | 6,279 | 9,685 |
| 50 | 0.232 | 110.7 | 8,966 | 11,313 |
| 100 | 0.378 | 73.9 | 13,183 | 19,852 |
| 200 | 0.565 | 47.6 | 20,718 | 26,153 |
| 400 | 0.765 | 28.4 | 34,952 | 43,987 |
| 800 | 0.918 | 16.2 | 61,570 | 76,478 |
The published rows are shown without smoothing or interpolation, including the non-monotonic recall at ef_search 10 and 20. Choose a recall target, then compare the query rate and tail latency at that operating point.
This report identifies a Rust integration test on a development machine but does not include its invocation. The SIFT-1M path uses SIFT_DATA_DIR and was described for a 32 GB machine; no SIFT-1M result is published here.
IVF and IVF-PQ are separate index choices. This HNSW dataset does not measure their memory use, latency, or recall at larger corpus sizes.
Review index choices03 / Full-text relevance
NDCG@10 measures graded relevance in the first ten results. The Lucene BM25 reference is the baseline reported in the BEIR paper, Table 2.
| Dataset | Docs | Queries | OC NDCG@10 | Lucene BM25 | Δ |
|---|---|---|---|---|---|
| SciFact | 5,183 | 300 | 0.662 | 0.665 | -0.003 |
The report used BM25 with default Anserini parameters and English tokenisation. The reported score difference is −0.003; the table does not compare every BEIR dataset.
python benchmarks/beir/run.py --dataset scifactMake a comparison you can use
Run the listed commands in a checkout containing the benchmark harness. Keep the setup and the raw output alongside every result.
Record corpus size, query mix, concurrency, hardware, engine version, and network location. Keep provisioned limits visible.
For vector retrieval, report recall with latency and QPS. For full-text, retain relevance judgments and the dataset split.
Include tail latency, error counts, configuration, and run logs. Do not extrapolate a result to unmeasured workloads or corpus sizes.
These are published run results, not current service telemetry. The tables cover only the configurations and datasets shown above.
Plan a workload evaluationBring your workload