Rusty Russell [Fri, 25 Sep 2009 05:03:08 +0000 (14:33 +0930)]
ccanlint: compile and run tests.
This means we should skip building if there are no C files in module: running tests requires building the module, but not necessarily that it has any C files.
Rusty Russell [Fri, 25 Sep 2009 03:16:08 +0000 (12:46 +0930)]
tdb: allow reads after prepare commit
(Imported from SAMBA commit 46c99ec2a3781d53138245182345c6c2ddd9a258:
From: Andrew Tridgell <tridge@samba.org>
Date: Tue, 15 Sep 2009 14:04:22 -0700
We previously only allowed a commit to happen after a prepare
commit. It is in fact safe to allow reads between a prepare and a
commit, and the s4 replication code can make use of that, so allow it.
Rusty Russell [Sat, 12 Sep 2009 01:27:44 +0000 (10:57 +0930)]
Build tests for ccan.
More sophisticated skipping: skip dependencies when one fails as well.
Allow tests to change their total_score; only access it after running (other than to check it's non-zero).
Rusty Russell [Tue, 18 Aug 2009 04:07:47 +0000 (13:37 +0930)]
tdb: locking cleanup
Explicitly split lock and unlock, for future replacement with user-supplied functions.
Use explicit flags enum for marking, blocking and probing.
Indicate upon unlock whether we had a read or write lock (POSIX doesn't care, but
wrappers might, and it's nice to be explicit).
Rusty Russell [Tue, 4 Aug 2009 03:52:28 +0000 (13:22 +0930)]
Wean off TDB_ERRCODE.
It was a regrettable hack to reduce line count in tdb; in fact it caused confusion as can be seen in this patch.
In particular, ecode now needs to be set before TDB_LOG.
Also, we should never set errno, as io.c was doing.
This greatly reduces the fragmentation of databases where records
tend to grow slowly by a small amount each time. The case where this
is most seen is the ldb index records. Adding this overallocation
reduced the size of the resulting database by more than 20x when
running a test that adds 10k users.
The idea behind this is to recover from badly fragmented free
lists. Choosing the point where the file expands is fairly arbitrary,
but seems to work well.
The tdb_repack() function repacks a TDB so that it has a single
freelist entry. The file doesn't shrink, but it does remove all
freelist fragmentation. This code originated in the CTDB vacuuming
code, but will now be used in ldb to cope with fragmentation from
re-indexing
make TDB_NOSYNC affect all the fsync/msync calls in transactions
During a transaction commit tdb normally uses fsync/msync calls to
make it crash safe. This can be disabled using the TDB_NOSYNC flag,
but it wasn't disabling all the code paths that caused a fsync/msync.
Using tdb_transaction_prepare_commit() gives us 2-phase commits. This
allows us to safely commit across multiple tdb databases at once, with
reasonable transaction semantics
Rusty Russell [Fri, 17 Jul 2009 07:29:58 +0000 (16:59 +0930)]
Cleanup variable names.
struct op: serial -> seqnum (and drop use of term "serial" everywhere).
struct op: op -> type
struct op_desc: general replacement for file/op_num pair.
Change add_dependency to take two const op_desc *: means reshuffle since we can't adjust the args any more.
Rusty Russell [Thu, 16 Jul 2009 01:58:33 +0000 (11:28 +0930)]
Implement timeout for the deadlock of traverse & transactions.
This has proven to be intractible: various attempts to eliminate have failed, so detect at runtime and cease the traversal (and do the remaining ops outside a traverse).
Rusty Russell [Wed, 15 Jul 2009 05:27:49 +0000 (14:57 +0930)]
Handle transactions!
Note: we can still deadlock on traversal vs transaction corner cases.
We handle transactions as single operation, which it logically is.
Rusty Russell [Wed, 15 Jul 2009 03:49:35 +0000 (13:19 +0930)]
Insert (implied) transaction cancel on tdb_close/EOF.
Also changes first member of transaction to have valid start_group field,
and fix outdated comment.
Rusty Russell [Tue, 14 Jul 2009 12:34:36 +0000 (22:04 +0930)]
Fix early transaction unlock when traverse done inside transaction.
Generalizes traverse in traverse fix from rusty@rustcorp.com.au-20090629073630-3eduhyypx2tp6u80