Rusty Russell [Tue, 27 Sep 2011 06:28:43 +0000 (15:58 +0930)]
likely: use htable_type
Also general cleanups:
(1) Don't assume that strings are folded by the compiler.
(2) Implement likely_stats_reset().
(3) Return non-const string from likely_stats(), as caller must free it.
(4) Don't use struct info indirection (that was from when we used callbacks?)
(5) Close memory leak in run-debug.c
Rusty Russell [Tue, 27 Sep 2011 05:07:01 +0000 (14:37 +0930)]
htable: clean up interface, document htable_type better.
We change from htable_new()/htable_free() to htable_init/htable_clear.
We also change HTABLE_DEFINE_TYPE() to be the full name, without automatically
prepending htable_.
Rusty Russell [Tue, 20 Sep 2011 02:32:43 +0000 (12:02 +0930)]
tdb2: allow readonly changes even while holding locks.
This happens in SAMBA with the TDB_VERSION1, presumably due to a
read-only traverse nested inside a normal traverse (since it doesn't
occur without TDB_VERSION1).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 12 Sep 2011 04:51:40 +0000 (14:21 +0930)]
tdb2: add error conversion functions.
This clarifies the code a little, but also provides a more explicit
mechanism which can be used to debug error handling (by introducing
tdb_err_t and making it a pointer type).
Daniel Burke [Mon, 5 Sep 2011 09:18:04 +0000 (18:48 +0930)]
ttxml: ccanlint fixes finished
Meets all of the requirements of ccanlint.
Code coverage of the tests covers everything except for a malloc() failure.
Added some xml files for the tests to work on.
Rusty Russell [Wed, 31 Aug 2011 06:09:17 +0000 (15:39 +0930)]
tdb2: test: Speed up run-01-new_database.
Don't run tdb_check with failtest on, since it is very slow. Do the
tdb_check w/ failtest in a new test, but skip MMAP suppression which
doesn't add much and slows down valgrind a lot.
Before this change run-01-new_database took 40 seconds (under
valgrind), after it takes 8 seconds, and run-12-check takes 3
seconds).
Rusty Russell [Wed, 31 Aug 2011 06:09:16 +0000 (15:39 +0930)]
tdb2: test: convert (non-invasive) run tests to api tests.
This reduces compilation time, since these are merely linked with the
pre-built module, rather than recompiling it into the test (which
allows for fancy things like failtest).
This reduces the test compile time down from about 62 seconds to 45
seconds. Since ccanlint compiles tests three times (once normally,
once with coverage, and once with reduced config.h) this makes a
difference: we go from 780 seconds to 729 seconds.
Rusty Russell [Wed, 31 Aug 2011 06:09:16 +0000 (15:39 +0930)]
tdb2: test: fix run-57-die-during-transaction.c to be more efficient.
We track malloc and free, but we didn't catch the free() inside
external_agent, which means that our list of allocations keeps
growing. Particularly under valgrind, which re-uses memory less than
the glibc allocator.
Rusty Russell [Wed, 31 Aug 2011 06:08:53 +0000 (15:38 +0930)]
tdb2: don't continue if tdb1_find fails.
The TDB1 code's tdb1_find() returns 0 on error; the callers should
not assume that the error means that the entry wasn't found, but use
last_error to determine it.
This was found by looking at how long the failure path testing for
test/run-10-simple-store.c was taking under valgrind, ie:
valgrind -q ./run-10-simple-store --show-slowest
This change dropped the time for that test from 53 seconds to 19
seconds.
Rusty Russell [Wed, 31 Aug 2011 06:01:08 +0000 (15:31 +0930)]
tdb2: test: try (almost) all tests with TDB_VERSION1 flag.
There are some minor changes required, in particular:
1) Make sure lockcheck understands tdb1 allrecord lock upgrades.
2) Handle tdb1 sequence number jumps: various operations increment the
sequence number twice, especually tdb_append.
3) Don't test fail on unlock, since it gets triggered with traversal on the
tdb1 backend (we didn't actually ever test this case for tdb2).
4) Move clear_if_first to offset 4, to match tdb1.
Rusty Russell [Wed, 31 Aug 2011 06:01:07 +0000 (15:31 +0930)]
tdb2: catch errors in tdb1_needs_recovery()
The tdb1 backend simply returns "true" if there's an error determining
if a tdb needs recovery. But this leads failtest down a rabbit hole;
it's better to return the error at this case (and makes for better for
diagnostics, since they will come from the first fault, not later in
tdb1_transaction_recover().
Rusty Russell [Wed, 31 Aug 2011 06:01:06 +0000 (15:31 +0930)]
tdb2: add TDB_ATTRIBUTE_TDB1_HASHSIZE
This replaces the tdb1_open "hash_size" argument. It will only be
valid when you call tdb_open() with O_CREAT in open_flags, and
TDB_VERSION1 in tdb_flags.
Rusty Russell [Wed, 31 Aug 2011 05:59:16 +0000 (15:29 +0930)]
tdb2: merge tdb1_context into tdb_context.
Finally, we split out the tdb2-specific parts of tdb_context, and put
them into a "tdb2" sub-struct; the tdb1 parts go into a "tdb1"
sub-struct. We get rido of tdb1_context and use tdb_context
everywhere.
Rusty Russell [Wed, 31 Aug 2011 05:58:16 +0000 (15:28 +0930)]
tdb2: use tdb->flags & TDB_RDONLY instead of tdb->read_only for TDB1 code.
There's also a semantic change here: for tdb1, being read-only meant no
locking, and it was an error to try to lock a r/o database. For TDB2, you'd
need to specify TDB_NOLOCK, which suppresses locking silently.
Rusty Russell [Wed, 31 Aug 2011 05:57:16 +0000 (15:27 +0930)]
tdb2: Make TDB1 use the same tdb_hash() wrapper as TDB2
This means converting the tdb1 inbuilt hash functions to the
tdb2-style, so they return 64 bit. We truncate to 32 bit everywhere
but in tdb_check() which needs to do so explicitly.
Rusty Russell [Wed, 31 Aug 2011 05:55:16 +0000 (15:25 +0930)]
tdb2: suppress tdb1 backend logging when locking returns EINTR or EAGAIN
The TDB1 code logs multiple times on errors; we must prevent that in
the limited case where locking fails. With TDB2, this can happen due
to the lock function attribute, where the user supplies replacement
locking functions which are allowed to return with errno EAGAIN or
EINTR for various special-effects. Flooding the logs for this is
unfriendly.
Rusty Russell [Wed, 31 Aug 2011 05:51:16 +0000 (15:21 +0930)]
tdb2: Make TDB1 code use TDB2's open flags.
This means getting rid of TDB_VOLATILE (perhaps we should use an
attribute for that?), TDB_INCOMPATIBLE_HASH (use the
tdb_attribute_hash for that) and TDB_CLEAR_IF_FIRST (use the
tdb_attribute_openhook for that).
We also get rid of TDB_DISALLOW_NESTING: that's the default for TDB2.
Rusty Russell [Wed, 31 Aug 2011 05:50:16 +0000 (15:20 +0930)]
tdb2: approximate INCOMPATIBLE_HASH flag with tdb1_incompatible_hash()
Rather than leak TDB_INCOMPATIBLE_HASH through to the TDB2 API, we make
it that if they use the tdb1_incompatible_hash function as their hash,
then we treat it as if they had specified the TDB_INCOMPATIBLE_HASH
flag (ie. we mark the header so it's unusable by tdb < 1.2.6).
This precludes the possibility of using TDB_INCOMPATIBLE_HASH with a
custom hash function: that used to allow the user to ensure that old TDB
versions couldn't open the TDB file (and recent ones check the header to
ensure they're using the right hash). But that's a small loss.
Rusty Russell [Wed, 31 Aug 2011 05:46:16 +0000 (15:16 +0930)]
tdb2: rearrange tdb_context layout so tdb1_context can match
By moving all the parts of struct tdb_context which logging and
locking use to the beginning of the structure, we can make
tdb1_context match, and thus pass that directly to the tdb_logerr()
function, and later the locking functions.
This is only necessary until we unify tdb1_context and tdb_context.