PostgreSQL, Neo4j Community, TETRA, lance-csr and SQLite
Each system ran the same scenarios against the same generated data on the same host. A system keeps its colour on every chart: PostgreSQL, Neo4j Community, TETRA, lance-csr, SQLite.
The standard workload, envelope S, 2 cores and 8 GiB, blended 80 / 15 / 5
15 runs at SF-300K, SF-1M, SF-3M.
Results
| System | Scale | DP | DP-Read | DP-Write | E | T ops/s | k* | R | O | M GiB | Disk GiB | Bytes per element | CPU ms per op | Verdict |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PostgreSQL 16.15 | SF-300K | 98 | 105 | 130 | 0.30 | 1,152.3 | 4 | 2.082 | 1.70 | 0.63 | 0.64 | 2,290 | 1.59 | Non-standard |
| PostgreSQL 16.15 | SF-1M | 172 | 167 | 373 | 1.00 | 601.7 | 4 | 2.058 | 1.70 | 0.44 | 0.55 | 588 | 3.05 | Non-standard |
| PostgreSQL 16.15 | SF-3M | 198 | 140 | 683 | 3.00 | 230.1 | 4 | 2.056 | 1.70 | 0.41 | 0.91 | 325 | 7.54 | Non-standard |
| Neo4j Community 5.26 | SF-300K | 33 | 39 | 153 | 0.30 | 261.4 | 4 | 2.157 | 1.10 | 1.23 | 0.59 | 2,120 | 6.11 | Non-standard |
| Neo4j Community 5.26 | SF-1M | 55 | 58 | 450 | 1.00 | 130.6 | 4 | 2.171 | 1.10 | 1.32 | 1.32 | 1,416 | 10.96 | Non-standard |
| Neo4j Community 5.26 | SF-3M | 67 | 68 | 1,254 | 3.00 | 54.9 | 4 | 2.228 | 1.10 | 1.67 | 3.92 | 1,402 | 25.89 | Non-standard |
| TETRA v0 | SF-300K | 80 | 346 | 20 | 0.30 | 552.6 | 1 | 2.068 | 1.00 | 0.54 | 0.04 | 158 | 1.53 | Non-standard |
| TETRA v0 | SF-1M | 265 | 730 | 67 | 1.00 | 557.6 | 4 | 2.107 | 1.00 | 0.85 | 0.12 | 125 | 2.12 | Non-standard |
| TETRA v0 | SF-3M | 520 | 1,000 | 193 | 3.00 | 375.8 | 4 | 2.167 | 1.00 | 1.33 | 0.23 | 82 | 2.94 | Non-standard |
| lance-csr 0.5.1 | SF-300K | 6 | 13 | 3 | 0.30 | 58.8 | 1 | 2.071 | 1.35 | 0.56 | 0.02 | 69 | 16.52 | Non-standard |
| lance-csr 0.5.1 | SF-1M | 12 | 26 | 12 | 1.00 | 35.7 | 1 | 2.156 | 1.35 | 1.25 | 0.04 | 40 | 20.57 | Non-standard |
| lance-csr 0.5.1 | SF-3M | 26 | 36 | 5 | 3.00 | 27.7 | 1 | 2.414 | 1.35 | 3.31 | 0.11 | 39 | 27.89 | Non-standard |
| SQLite 3.53.4 | SF-300K | 74 | 173 | 55 | 0.30 | 742.8 | 1 | 2.011 | 1.50 | 0.09 | 0.07 | 250 | 0.99 | Non-standard |
| SQLite 3.53.4 | SF-1M | 109 | 205 | 105 | 1.00 | 329.6 | 1 | 2.012 | 1.50 | 0.09 | 0.09 | 93 | 2.58 | Non-standard |
| SQLite 3.53.4 | SF-3M | 112 | 178 | 176 | 3.00 | 113.1 | 4 | 2.015 | 1.50 | 0.11 | 0.19 | 66 | 9.09 | Non-standard |
Scaling
Concurrency
From the cold restart
Each line starts when the run restarts the SUT with an empty page cache (P3), and follows the warm-up (P4) and the warm single-client run (P5). The scored phases come later and start warm, so these charts carry the part of each system's behaviour the score leaves out today.
SF-300K
SF-1M
SF-3M
Latency against target
p99 at k* in the mixed ladder, in milliseconds. Blue cells beat the target, grey cells sit on it, and red cells miss it; the colour deepens with each factor of ten.
| Class | Target | PostgreSQL SF-300K | PostgreSQL SF-1M | PostgreSQL SF-3M | Neo4j Community SF-300K | Neo4j Community SF-1M | Neo4j Community SF-3M | TETRA SF-300K | TETRA SF-1M | TETRA SF-3M | lance-csr SF-300K | lance-csr SF-1M | lance-csr SF-3M | SQLite SF-300K | SQLite SF-1M | SQLite SF-3M |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| R1 point | 10 | 6.3 | 9.8 | 9.7 | 14.0 | 12.3 | 12.6 | 0.6 | 17.3 | 31.6 | 16.3 | 18.4 | 18.6 | 0.2 | 0.3 | 10.0 |
| R1w point, worst case | 10 | 7.2 | 8.2 | 8.9 | 8.1 | 10.4 | 9.3 | 0.6 | 11.1 | 20.4 | 16.9 | 24.8 | 15.0 | 0.2 | 0.3 | 5.9 |
| R2 filter | 200 | 6.6 | 12.1 | 4.6 | 19.7 | 11.7 | 11.5 | 2.4 | 17.0 | 31.9 | 13.6 | 17.8 | 26.8 | 0.4 | 0.5 | 10.7 |
| R2w filter, worst case | 200 | 8.1 | 10.9 | 11.3 | 33.7 | 37.2 | 97.6 | 5.5 | 32.6 | 62.8 | 8.9 | 14.0 | 34.0 | 2.0 | 4.3 | 10.4 |
| R3 neighbourhood | 50 | 6.6 | 7.6 | 9.3 | 12.6 | 15.5 | 9.9 | 0.7 | 14.1 | 19.8 | 29.5 | 29.2 | 33.7 | 0.3 | 0.4 | 10.2 |
| R3w neighbourhood, worst case | 50 | 5.3 | 17.3 | 110.0 | 12.3 | 24.3 | 52.3 | 1.6 | 15.5 | 29.9 | 35.2 | 36.6 | 62.2 | 1.7 | 3.7 | 24.4 |
| R4 deep | 500 | 6.5 | 9.0 | 9.2 | 13.9 | 18.2 | 18.2 | 0.9 | 18.3 | 15.9 | 22.8 | 27.6 | 24.6 | 0.7 | 0.8 | 8.4 |
| R4w deep, worst case | 500 | 3.2 | 4.5 | 6.0 | 10.0 | 15.0 | 14.2 | 1.6 | 20.9 | 23.9 | 31.8 | 30.0 | 51.0 | 1.4 | 1.3 | 15.5 |
| R5 path | 500 | 8.2 | 11.1 | 100.8 | 10.6 | 13.0 | 11.6 | 0.6 | 26.7 | 22.4 | 16.1 | 20.0 | 16.8 | 4.4 | 13.1 | 88.5 |
| R5w path, worst case | 500 | 4.9 | 12.9 | 15.7 | 11.8 | 1.8 | 11.3 | 0.6 | 8.0 | 51.1 | 6.1 | 7.1 | 11.6 | 3.0 | 7.4 | 65.2 |
| R6 pattern | 2,000 | 4.0 | 13.0 | 4.2 | 6.7 | 13.2 | 6.9 | 10.0 | 15.3 | 25.0 | 36.5 | 38.5 | 46.2 | 0.3 | 0.4 | 4.7 |
| R6w pattern, worst case | 2,000 | 1.5 | 0.4 | 0.4 | 15.4 | 12.1 | 16.9 | 4.8 | 12.4 | 37.3 | 35.6 | 38.3 | 48.4 | 0.2 | 0.3 | 1.2 |
| R7 analytic | 5,000 | 54.8 | 146.6 | 400.1 | 118.4 | 209.2 | 697.9 | 14.5 | 54.2 | 115.7 | 22.5 | 39.6 | 71.1 | 18.8 | 53.9 | 309.5 |
| R7w analytic, worst case | 5,000 | 41.2 | 138.2 | 440.3 | 107.8 | 250.4 | 574.5 | 15.4 | 36.2 | 92.1 | 18.6 | 32.6 | 70.5 | 17.9 | 50.7 | 266.0 |
| S1 amount band | 1,000 | 83.5 | 104.3 | 183.2 | 266.0 | 622.1 | 1,593.3 | 10.8 | 22.3 | 31.2 | 17.5 | 51.2 | 102.3 | 11.6 | 20.5 | 131.8 |
| S3 text search | 2,000 | 13.7 | 51.7 | 126.4 | 45.5 | 113.1 | 303.9 | 1.0 | 19.2 | 15.5 | 11.2 | 18.9 | 49.0 | 4.6 | 14.8 | 103.7 |
| J4 neighbours alike | 500 | 11.9 | 23.6 | 72.1 | 15.2 | 10.1 | 10.3 | 1.1 | 10.2 | 19.2 | 28.1 | 40.0 | 43.1 | 3.8 | 12.7 | 106.2 |
| O1 person by email | 20 | 12.3 | 25.3 | 69.1 | 28.0 | 88.3 | 148.6 | 0.7 | 19.1 | 15.8 | 15.6 | 21.0 | 26.3 | 1.4 | 4.4 | 36.6 |
| O2 accounts by opening date | 200 | 9.4 | 18.0 | 57.6 | 46.1 | 136.1 | 257.5 | 0.9 | 29.1 | 32.2 | 10.8 | 11.5 | 11.0 | 1.4 | 24.4 | 321.3 |
| O3 people by surname and city | 100 | 11.0 | 25.6 | 61.3 | 11.5 | 9.3 | 16.2 | 1.1 | 16.1 | 16.0 | 22.9 | 27.2 | 37.3 | 1.5 | 4.5 | 38.4 |
| W1 insert | 20 | 14.2 | 17.9 | 12.2 | 21.5 | 29.6 | 23.4 | 4.3 | 32.1 | 29.8 | 27.4 | 32.6 | 230.0 | 2.4 | 2.4 | 133.8 |
| W2 insert edge | 10 | 13.2 | 14.2 | 15.7 | 20.3 | 28.8 | 37.2 | 3.9 | 28.8 | 26.7 | 10.5 | 58.9 | 8.0 | 1.7 | 1.7 | 133.8 |
| W3 batch | 500 | 71.4 | 69.1 | 85.6 | 122.8 | 194.0 | 124.4 | 28.5 | 68.2 | 58.5 | 12.5 | 13.9 | 12.8 | 33.5 | 34.3 | 51.1 |
| U1 modify | 20 | 9.9 | 13.4 | 8.1 | 17.7 | 27.8 | 11.1 | 1.8 | 28.6 | 19.8 | 38.9 | 56.4 | 72.6 | 1.6 | 1.6 | 105.0 |
| U2 rewire | 30 | 12.7 | 19.3 | 21.3 | 41.0 | 22.1 | 31.4 | 8.7 | 30.5 | 23.8 | 30.8 | 16.0 | 15.6 | 1.9 | 2.4 | 39.2 |
| D1 delete edge | 20 | 8.9 | 10.9 | 11.8 | 23.8 | 58.4 | 16.1 | 6.6 | 15.8 | 15.8 | 45.9 | 27.2 | 49.5 | 1.5 | 1.9 | 18.9 |
| D2 delete cascade | 100 | 26.9 | 62.1 | 191.9 | 15.6 | 12.9 | 37.3 | 2.3 | 10.6 | 67.8 | 108.4 | 164.2 | 165.6 | 22.1 | 37.4 | 221.8 |
Deviations
Every run here carries these:
- every container runs --privileged (DPBENCH_PRIVILEGED): the host's kernel has no BPF syscall, so the runtime cannot attach a cgroup v2 device filter; cpu, cpuset, memory and pids limits apply as before
- profile: quick timings: ladders, duty cycle and warm-up are shorter than spec §8.5, §8.9 and §9.5 require
- repetitions: 1; spec §9.5 publishes the median of 3
- tiers: 15% off-index and 5% worst-case operations blended in; spec §9.3's mix is typical operations alone
- warm-up before P5 runs at 16 clients: spec §9.5 says k*, which only P7 finds
PostgreSQL at SF-300K also carries:
- dataset.scale: 300K is not a DP-Core scale factor (spec §6.6)
PostgreSQL at SF-3M also carries:
- dataset.scale: 3M is not a DP-Core scale factor (spec §6.6)
Neo4j Community at SF-300K also carries:
- dataset.scale: 300K is not a DP-Core scale factor (spec §6.6)
- the SUT runs at read-committed isolation; spec §9.2 asks for snapshot isolation or stronger
Neo4j Community at SF-1M also carries:
- the SUT runs at read-committed isolation; spec §9.2 asks for snapshot isolation or stronger
Neo4j Community at SF-3M also carries:
- dataset.scale: 3M is not a DP-Core scale factor (spec §6.6)
- the SUT runs at read-committed isolation; spec §9.2 asks for snapshot isolation or stronger
TETRA at SF-300K also carries:
- dataset.scale: 300K is not a DP-Core scale factor (spec §6.6)
TETRA at SF-3M also carries:
- dataset.scale: 3M is not a DP-Core scale factor (spec §6.6)
lance-csr at SF-300K also carries:
- dataset.scale: 300K is not a DP-Core scale factor (spec §6.6)
- the SUT runs at per-dataset snapshot isolation; spec §9.2 asks for snapshot isolation or stronger
- commits are not durable when they return (Lance 13.0.0 calls no fsync: a commit returns once its data files and manifest are in the page cache, so a power loss can lose acknowledged writes, or keep a manifest whose data files never reached the disk); spec §9.2 counts only durable writes
lance-csr at SF-1M also carries:
- the SUT runs at per-dataset snapshot isolation; spec §9.2 asks for snapshot isolation or stronger
- commits are not durable when they return (Lance 13.0.0 calls no fsync: a commit returns once its data files and manifest are in the page cache, so a power loss can lose acknowledged writes, or keep a manifest whose data files never reached the disk); spec §9.2 counts only durable writes
lance-csr at SF-3M also carries:
- dataset.scale: 3M is not a DP-Core scale factor (spec §6.6)
- the SUT runs at per-dataset snapshot isolation; spec §9.2 asks for snapshot isolation or stronger
- commits are not durable when they return (Lance 13.0.0 calls no fsync: a commit returns once its data files and manifest are in the page cache, so a power loss can lose acknowledged writes, or keep a manifest whose data files never reached the disk); spec §9.2 counts only durable writes
SQLite at SF-300K also carries:
- dataset.scale: 300K is not a DP-Core scale factor (spec §6.6)
SQLite at SF-3M also carries:
- dataset.scale: 3M is not a DP-Core scale factor (spec §6.6)