tdb2: use counters to decide when to coalesce records.
This simply uses a 7 bit counter which gets incremented on each addition
to the list (but not decremented on removals). When it wraps, we walk the
entire list looking for things to coalesce.
This causes performance problems, especially when appending records, so
we limit it in the next patch:
Before:
$ time ./growtdb-bench 250000 10 > /dev/null && ls -l /tmp/growtdb.tdb && time ./tdbtorture -s 0 && ls -l torture.tdb && ./speed --transaction
2000000
real 0m59.687s
user 0m11.593s
sys 0m4.100s
-rw------- 1 rusty rusty
752004064 2011-04-27 21:14 /tmp/growtdb.tdb
testing with 3 processes, 5000 loops, seed=0
OK
real 1m17.738s
user 0m0.348s
sys 0m0.580s
-rw------- 1 rusty rusty 663360 2011-04-27 21:15 torture.tdb
Adding
2000000 records: 926 ns (
110556088 bytes)
Finding
2000000 records: 592 ns (
110556088 bytes)
Missing
2000000 records: 416 ns (
110556088 bytes)
Traversing
2000000 records: 422 ns (
110556088 bytes)
Deleting
2000000 records: 741 ns (
244003768 bytes)
Re-adding
2000000 records: 799 ns (
244003768 bytes)
Appending
2000000 records: 1147 ns (
295244592 bytes)
Churning
2000000 records: 1827 ns (
568411440 bytes)
After:
$ time ./growtdb-bench 250000 10 > /dev/null && ls -l /tmp/growtdb.tdb && time ./tdbtorture -s 0 && ls -l torture.tdb && ./speed --transaction
2000000
real 1m17.022s
user 0m27.206s
sys 0m3.920s
-rw------- 1 rusty rusty
570130576 2011-04-27 21:17 /tmp/growtdb.tdb
testing with 3 processes, 5000 loops, seed=0
OK
real 1m27.355s
user 0m0.296s
sys 0m0.516s
-rw------- 1 rusty rusty 617352 2011-04-27 21:18 torture.tdb
Adding
2000000 records: 890 ns (
110556088 bytes)
Finding
2000000 records: 565 ns (
110556088 bytes)
Missing
2000000 records: 390 ns (
110556088 bytes)
Traversing
2000000 records: 410 ns (
110556088 bytes)
Deleting
2000000 records: 8623 ns (
244003768 bytes)
Re-adding
2000000 records: 7089 ns (
244003768 bytes)
Appending
2000000 records: 33708 ns (
244003768 bytes)
Churning
2000000 records: 2029 ns (
268404160 bytes)