We don't need to use 4k for our transaction pages; we can use any
value. For the tools/speed benchmark, any value between about 4k and
64M makes no difference, but that's probably because the entire
database is touched in each transaction.
So instead, I looked at tdbtorture to try to find an optimum value, as
it uses smaller transactions. 4k and 64k were equivalent. 16M was
almost three times slower, 1M was 5-10% slower. 1024 was also 5-10%
slower.
There's a slight advantage of having larger pages, both for allowing
direct access to the database (if it's all in one page we can sometimes
grant direct access even inside a transaction) and for the compactness
of our recovery area (since our code is naive and won't combine one
run across pages).
Before:
$ time ./growtdb-bench 250000 10 > /dev/null && ls -l /tmp/growtdb.tdb && time ./tdbtorture -s 0 && ls -l torture.tdb && ./speed --transaction
2000000
real 0m47.127s
user 0m17.125s
sys 0m2.456s
-rw------- 1 rusty rusty
366680288 2011-04-27 21:34 /tmp/growtdb.tdb
testing with 3 processes, 5000 loops, seed=0
OK
real 1m16.049s
user 0m0.300s
sys 0m0.492s
-rw------- 1 rusty rusty 244472 2011-04-27 21:35 torture.tdb
Adding
2000000 records: 894 ns (
110551992 bytes)
Finding
2000000 records: 564 ns (
110551992 bytes)
Missing
2000000 records: 398 ns (
110551992 bytes)
Traversing
2000000 records: 399 ns (
110551992 bytes)
Deleting
2000000 records: 711 ns (
225633208 bytes)
Re-adding
2000000 records: 819 ns (
225633208 bytes)
Appending
2000000 records: 1252 ns (
248196544 bytes)
Churning
2000000 records: 2319 ns (
424005056 bytes)
After:
$ time ./growtdb-bench 250000 10 > /dev/null && ls -l /tmp/growtdb.tdb && time ./tdbtorture -s 0 && ls -l torture.tdb && ./speed --transaction
2000000
real 0m45.021s
user 0m16.261s
sys 0m2.432s
-rw------- 1 rusty rusty
364469344 2011-04-27 22:55 /tmp/growtdb.tdb
testing with 3 processes, 5000 loops, seed=0
OK
real 1m10.144s
user 0m0.480s
sys 0m0.460s
-rw------- 1 rusty rusty 391992 2011-04-27 22:56 torture.tdb
Adding
2000000 records: 863 ns (
110601144 bytes)
Finding
2000000 records: 565 ns (
110601144 bytes)
Missing
2000000 records: 383 ns (
110601144 bytes)
Traversing
2000000 records: 409 ns (
110601144 bytes)
Deleting
2000000 records: 676 ns (
225354680 bytes)
Re-adding
2000000 records: 784 ns (
225354680 bytes)
Appending
2000000 records: 1191 ns (
247890168 bytes)
Churning
2000000 records: 2166 ns (
423133432 bytes)