From client to engine.
YCSB and BEIR ran from an India-based laptop against a Mumbai reference engine.
- Reference engine
- 2 vCPU · 2 GB
- Network round trip
- Approximately 100 ms
- Included
- HTTPS + network time
Benchmarks
Explore published reads, writes and search results with the setup that produced them. Then test the workload your application actually needs.
Speed, quality and the conditions behind both.
The published runs measure three different paths. An end-to-end HTTPS result and an in-process search timing answer different questions.
YCSB and BEIR ran from an India-based laptop against a Mumbai reference engine.
A separate YCSB sweep used production configurations and a driver in the engine’s region.
The ANN test measured HNSW search directly in a Rust release build on a development machine.
The reference engine used substrate compression and a bounded memory guard. These runs describe the configurations tested; they do not establish an unthrottled engine ceiling or a service-level guarantee.
Open a report to inspect its configuration and measurements. Keep latency, throughput and retrieval quality in view together.
A 20,000-row table with Zipfian access and 50,000 operations across workloads A, B, C, D and F.
| Workload | Operation mix | Ops/sec | p50 (ms) | p95 (ms) | p99 (ms) | Errors |
|---|---|---|---|---|---|---|
| A | 50% read / 50% update | 292 | 104 | 143 | 202 | 0 |
| B | 95% read / 5% update | 321 | 92 | 120 | 158 | 0 |
| C | 100% read | 146 | 125 | 557 | 1,220 | 0 |
| D | 95% read latest / 5% insert | 338 | 92 | 119 | 138 | 0 |
| F | Read-modify-write | 215 | 165 | 225 | 252 | 0 |
All five reported workloads recorded zero errors and no rate-limit hits. Workload E is not included in this dataset.
A separate run used the same YCSB driver in the engine’s region. Client concurrency and provisioned request limits are part of these results.
| Setup | Ops/sec | p99 (ms) | Run notes |
|---|---|---|---|
| 4-vCPU node | 1,046 | Not supplied | Provisioned limit of approximately 1,000 requests/sec. |
| 8-vCPU node · one client | 1,384 | 117 | The report identifies the client driver as the limiting factor. |
| 8-vCPU node · four clients | 4,940 | 173 | Read traffic; zero errors. Provisioned limit of approximately 5,000 requests/sec. |
The four-client result is read traffic. The report supplies no p99 measurement for the 4-vCPU run. Treat the stated limits as properties of these test setups.
100,000 SIFT vectors with 128 dimensions, 1,000 queries, M=16 and ef_construction=200. Single-threaded AVX2, measured in process.
| ef_search | Recall@10 | Queries/sec | p50 (µs) | p99 (µs) |
|---|---|---|---|---|
| 10 | 0.286 | 187.9 | 5,208 | 8,170 |
| 20 | 0.115 | 154.8 | 6,279 | 9,685 |
| 50 | 0.232 | 110.7 | 8,966 | 11,313 |
| 100 | 0.378 | 73.9 | 13,183 | 19,852 |
| 200 | 0.565 | 47.6 | 20,718 | 26,153 |
| 400 | 0.765 | 28.4 | 34,952 | 43,987 |
| 800 | 0.918 | 16.2 | 61,570 | 76,478 |
Rows are retained as published, including the non-monotonic recall at ef_search 10 and 20. Pick the quality target first, then assess query rate and tail latency at that setting.
The report identifies a Rust integration test, but does not supply its invocation or exact development-machine specification. It publishes no SIFT-1M or 100M-vector result. These HNSW measurements do not establish IVF or IVF-PQ performance.
Review the index choicesThe published SciFact run used English tokenisation and default Anserini BM25 parameters. NDCG@10 evaluates the ranking of relevant results.
| Dataset | Documents | Queries | OriginChainDB NDCG@10 | Published BM25 NDCG@10 | Difference |
|---|---|---|---|---|---|
| SciFact | 5,183 | 300 | 0.662 | 0.665 | −0.003 |
The 0.665 reference comes from the BEIR paper, Table 2, rather than a side-by-side run on this hardware. This result does not cover the full BEIR suite.
Source: OriginChainDB’s published benchmark report. These are reported runs, not current service telemetry. The measurements apply to the datasets and conditions shown.
A useful evaluation starts with a real workload and ends with enough detail to repeat the test.
Choose your query mix, dataset size and expected concurrency. Set a latency target and, for retrieval, the quality level your application needs.
Record the engine version, CPU, memory, region, network path and provisioned limits. Keep index parameters, warm-up and test duration consistent.
Retain p50, p95 and p99 latency, error counts and raw logs. Report vector recall or text relevance alongside speed, then repeat the run.
Run these in a checkout containing the benchmark harness, with the matching dataset and environment. The report lists these commands but does not link a standalone harness or raw-log download; request the matching materials when planning an evaluation.
python benchmarks/ycsb/run.py --record-count 20000 --op-count 10000 --workloads C,B,A,D,F python benchmarks/beir/run.py --dataset scifact The ANN report does not include an executable test invocation. Confirm the exact test, build and machine before attempting to reproduce its timings.
Connect the reported results to the configuration and query paths your team plans to use.
Build with OriginChainDB