From: Rusty Russell <rusty@rustcorp.com.au>
Date: Wed, 27 Apr 2011 13:40:24 +0000 (+0930)
Subject: tdb2: enlarge transaction pagesize to 64k
X-Git-Url: http://git.ozlabs.org/?p=ccan;a=commitdiff_plain;h=0753972a623c7bf24f13578a5ceb3995ea307876

tdb2: enlarge transaction pagesize to 64k

We don't need to use 4k for our transaction pages; we can use any
value.  For the tools/speed benchmark, any value between about 4k and
64M makes no difference, but that's probably because the entire
database is touched in each transaction.

So instead, I looked at tdbtorture to try to find an optimum value, as
it uses smaller transactions.  4k and 64k were equivalent.  16M was
almost three times slower, 1M was 5-10% slower.  1024 was also 5-10%
slower.

There's a slight advantage of having larger pages, both for allowing
direct access to the database (if it's all in one page we can sometimes
grant direct access even inside a transaction) and for the compactness
of our recovery area (since our code is naive and won't combine one
run across pages).

Before:
$ time ./growtdb-bench 250000 10 > /dev/null && ls -l /tmp/growtdb.tdb && time ./tdbtorture -s 0 && ls -l torture.tdb && ./speed --transaction 2000000
real	0m47.127s
user	0m17.125s
sys	0m2.456s
-rw------- 1 rusty rusty 366680288 2011-04-27 21:34 /tmp/growtdb.tdb
testing with 3 processes, 5000 loops, seed=0
OK

real	1m16.049s
user	0m0.300s
sys	0m0.492s
-rw------- 1 rusty rusty 244472 2011-04-27 21:35 torture.tdb
Adding 2000000 records:  894 ns (110551992 bytes)
Finding 2000000 records:  564 ns (110551992 bytes)
Missing 2000000 records:  398 ns (110551992 bytes)
Traversing 2000000 records:  399 ns (110551992 bytes)
Deleting 2000000 records:  711 ns (225633208 bytes)
Re-adding 2000000 records:  819 ns (225633208 bytes)
Appending 2000000 records:  1252 ns (248196544 bytes)
Churning 2000000 records:  2319 ns (424005056 bytes)

After:
$ time ./growtdb-bench 250000 10 > /dev/null && ls -l /tmp/growtdb.tdb && time ./tdbtorture -s 0 && ls -l torture.tdb && ./speed --transaction 2000000
real	0m45.021s
user	0m16.261s
sys	0m2.432s
-rw------- 1 rusty rusty 364469344 2011-04-27 22:55 /tmp/growtdb.tdb
testing with 3 processes, 5000 loops, seed=0
OK

real	1m10.144s
user	0m0.480s
sys	0m0.460s
-rw------- 1 rusty rusty 391992 2011-04-27 22:56 torture.tdb
Adding 2000000 records:  863 ns (110601144 bytes)
Finding 2000000 records:  565 ns (110601144 bytes)
Missing 2000000 records:  383 ns (110601144 bytes)
Traversing 2000000 records:  409 ns (110601144 bytes)
Deleting 2000000 records:  676 ns (225354680 bytes)
Re-adding 2000000 records:  784 ns (225354680 bytes)
Appending 2000000 records:  1191 ns (247890168 bytes)
Churning 2000000 records:  2166 ns (423133432 bytes)
---

diff --git a/ccan/tdb2/transaction.c b/ccan/tdb2/transaction.c
index 09f932b8..d9a1bc05 100644
--- a/ccan/tdb2/transaction.c
+++ b/ccan/tdb2/transaction.c
@@ -120,7 +120,7 @@ struct tdb_transaction {
 };
 
 /* This doesn't really need to be pagesize, but we use it for similar reasons. */
-#define PAGESIZE 4096
+#define PAGESIZE 65536
 
 /*
   read while in a transaction. We need to check first if the data is in our list