Rusty Russell [Wed, 31 Aug 2011 06:01:07 +0000 (15:31 +0930)]
tdb2: catch errors in tdb1_needs_recovery()
The tdb1 backend simply returns "true" if there's an error determining
if a tdb needs recovery. But this leads failtest down a rabbit hole;
it's better to return the error at this case (and makes for better for
diagnostics, since they will come from the first fault, not later in
tdb1_transaction_recover().
Rusty Russell [Wed, 31 Aug 2011 06:01:06 +0000 (15:31 +0930)]
tdb2: add TDB_ATTRIBUTE_TDB1_HASHSIZE
This replaces the tdb1_open "hash_size" argument. It will only be
valid when you call tdb_open() with O_CREAT in open_flags, and
TDB_VERSION1 in tdb_flags.
Rusty Russell [Wed, 31 Aug 2011 05:59:16 +0000 (15:29 +0930)]
tdb2: merge tdb1_context into tdb_context.
Finally, we split out the tdb2-specific parts of tdb_context, and put
them into a "tdb2" sub-struct; the tdb1 parts go into a "tdb1"
sub-struct. We get rido of tdb1_context and use tdb_context
everywhere.
Rusty Russell [Wed, 31 Aug 2011 05:58:16 +0000 (15:28 +0930)]
tdb2: use tdb->flags & TDB_RDONLY instead of tdb->read_only for TDB1 code.
There's also a semantic change here: for tdb1, being read-only meant no
locking, and it was an error to try to lock a r/o database. For TDB2, you'd
need to specify TDB_NOLOCK, which suppresses locking silently.
Rusty Russell [Wed, 31 Aug 2011 05:57:16 +0000 (15:27 +0930)]
tdb2: Make TDB1 use the same tdb_hash() wrapper as TDB2
This means converting the tdb1 inbuilt hash functions to the
tdb2-style, so they return 64 bit. We truncate to 32 bit everywhere
but in tdb_check() which needs to do so explicitly.
Rusty Russell [Wed, 31 Aug 2011 05:55:16 +0000 (15:25 +0930)]
tdb2: suppress tdb1 backend logging when locking returns EINTR or EAGAIN
The TDB1 code logs multiple times on errors; we must prevent that in
the limited case where locking fails. With TDB2, this can happen due
to the lock function attribute, where the user supplies replacement
locking functions which are allowed to return with errno EAGAIN or
EINTR for various special-effects. Flooding the logs for this is
unfriendly.
Rusty Russell [Wed, 31 Aug 2011 05:51:16 +0000 (15:21 +0930)]
tdb2: Make TDB1 code use TDB2's open flags.
This means getting rid of TDB_VOLATILE (perhaps we should use an
attribute for that?), TDB_INCOMPATIBLE_HASH (use the
tdb_attribute_hash for that) and TDB_CLEAR_IF_FIRST (use the
tdb_attribute_openhook for that).
We also get rid of TDB_DISALLOW_NESTING: that's the default for TDB2.
Rusty Russell [Wed, 31 Aug 2011 05:50:16 +0000 (15:20 +0930)]
tdb2: approximate INCOMPATIBLE_HASH flag with tdb1_incompatible_hash()
Rather than leak TDB_INCOMPATIBLE_HASH through to the TDB2 API, we make
it that if they use the tdb1_incompatible_hash function as their hash,
then we treat it as if they had specified the TDB_INCOMPATIBLE_HASH
flag (ie. we mark the header so it's unusable by tdb < 1.2.6).
This precludes the possibility of using TDB_INCOMPATIBLE_HASH with a
custom hash function: that used to allow the user to ensure that old TDB
versions couldn't open the TDB file (and recent ones check the header to
ensure they're using the right hash). But that's a small loss.
Rusty Russell [Wed, 31 Aug 2011 05:46:16 +0000 (15:16 +0930)]
tdb2: rearrange tdb_context layout so tdb1_context can match
By moving all the parts of struct tdb_context which logging and
locking use to the beginning of the structure, we can make
tdb1_context match, and thus pass that directly to the tdb_logerr()
function, and later the locking functions.
This is only necessary until we unify tdb1_context and tdb_context.
Rusty Russell [Wed, 31 Aug 2011 04:25:15 +0000 (13:55 +0930)]
tdb2: Hand error code to logging function.
Since we've deprecated tdb_error() function (and it didn't work right
from inside the logging function anyway, since we didn't set
tdb->ecode yet) we need to hand it to the log function.
Rusty Russell [Wed, 31 Aug 2011 01:15:12 +0000 (10:45 +0930)]
tdb2: move transaction lock
Make it the same offset as TDB1. This isn't strictly necessary, but
it would allow for total unification later, since TDB1 and TDB2's
transaction code is otherwise completely compatible.
Rusty Russell [Wed, 31 Aug 2011 01:14:12 +0000 (10:44 +0930)]
tdb2: enforce TDB_CONVERT
If the caller actually specifies TDB_CONVERT, fail if the TDB does not
need conversion. It's unusual for the user to specify TDB_CONVERT
(it's auto-detected) but better to be strict.
Rusty Russell [Wed, 31 Aug 2011 01:13:12 +0000 (10:43 +0930)]
tdb2: cleanup oob handling.
The tdb_oob() function can fail due to errors, as well as because the length
asked for is greater than the size of the file. Clean that up:
(1) If probe is true, only fail if there's an error, not if the length is
too great.
(2) Exit tdb_open() if it tdb_oob() probe fails; this helps cut down
test time for failtest.
(3) Don't set probe to true in tdb_direct() fail; a minor issue, but it means
we log failure.
Rusty Russell [Wed, 31 Aug 2011 01:10:12 +0000 (10:40 +0930)]
tdb2: Fix to always use 64-bit offset definition
We use off_t in the tdb2 interface (for tdb_attribute_flock); we need
to make sure that all callers agree on the size.
This also causes a problem in the tests: it's not enough to include
config.h first, we need the _FILE_OFFSET_BITS define from private.h.
Otherwise, we can disagree about the definitions of F_SETLK, F_SETLKW
and off_t, causing strange problems.
Rusty Russell [Tue, 30 Aug 2011 04:31:26 +0000 (14:01 +0930)]
Makefile: speed fastcheck
Rather than timeout, explicitly disable tests_pass_valgrind and
tests_compile_coverage. This is important now we run under valgrind
the first time we run the tests, thus would always time out and not
run most of the tests at all!
"make check" takes 7m57s, old "make fastcheck" takes 3m28s, and this takes
only 1m57s, but much more is run (as shown by the larger score total).
Rusty Russell [Tue, 30 Aug 2011 04:31:25 +0000 (14:01 +0930)]
ccanlint: clean up reduced feature handling.
Putting the reduced config.h in the current directory means that it's
actually being picked up by other tests, such as the string checks.
So move it to a sub-directory where we need an explicit -I.
We also fix the dependencies, so that "--target
tests_pass_without_features" works.
Rusty Russell [Tue, 30 Aug 2011 04:31:25 +0000 (14:01 +0930)]
ccanlint: handle duplicate dependencies in _info
We eliminate dependencies as we recurse, but if a single _info file
lists a dependency twice, we add it to the list twice and this skip
over the middle ones.
Rusty Russell [Tue, 30 Aug 2011 04:27:08 +0000 (13:57 +0930)]
ccanlint: keep separate array of compiled versions.
Rather than mug the old ->compiled version when we reduce features,
keep both in the structure. This makes it clear that we are using the
right version (we weren't in all cases, in particular we weren't
recompiling the test helpers correctly.