Rusty Russell [Mon, 19 Mar 2012 05:14:13 +0000 (15:44 +1030)]
tools: determing ccan directory differently
Don't assume it's two levels above the module dir, instead look for
last "/ccan/" in path. This is an incremental step towards nested
module directories.
Rusty Russell [Thu, 8 Mar 2012 03:44:22 +0000 (14:14 +1030)]
failtest: don't assume FD_SETSIZE is maximum runtime fd.
This breaks when rlimit is less. Unfortunately, valgrind (32 bit x86,
3.7.0.SVN, Ubuntu) fails to set the file limit properly on the test:
reducing it to the obvious getrlimit/setrlimit/getrlimit works fine,
so leaving diagnostics for another day.
Rusty Russell [Tue, 14 Feb 2012 09:39:18 +0000 (20:09 +1030)]
tdb: delete from CCAN.
This was an early experiment in putting code into CCAN, but TDB is a public
library distributed as part of the Samba sources; there is no reason to
publish it here (especially now the unit tests are also in Samba).
Rusty Russell [Tue, 14 Feb 2012 09:34:00 +0000 (20:04 +1030)]
configurator: define HAVE_CCAN.
An interesting case came up with the tdb2 code in SAMBA recently. It's a
public library, and it doesn't want to be dependent on CCAN, but the header
uses cast and typesafe_cb for extra type safety.
A good solution to this is to put dummy versions under !HAVE_CCAN. Of course,
any CCAN config.h should define HAVE_CCAN.
Rusty Russell [Sat, 28 Jan 2012 04:14:43 +0000 (14:44 +1030)]
tdb2: tdb1: use same expansion factor logic when expanding for new recovery area.
If we're expanding because the current recovery area is too small, we
expand only the amount we need. This can quickly lead to exponential
growth when we have a slowly-expanding record (hence a
slowly-expanding transaction size).
Rusty Russell [Wed, 21 Dec 2011 05:44:51 +0000 (16:14 +1030)]
tdb2: be more careful on 4G files (tdb1).
I came across a tdb which had wrapped to 4G + 4K, and the contents had been
destroyed by processes which thought it only 4k long. Fix this by checking
on open, and making tdb_oob() check for wrap itself.
Rusty Russell [Tue, 20 Dec 2011 07:10:57 +0000 (17:40 +1030)]
tdb: don't free old recovery area when expanding if already at EOF.
We allocate a new recovery area by expanding the file. But if the
recovery area is already at the end of file (as shown in at least one
client case), we can simply expand the record, rather than freeing it
and creating a new one.
Rusty Russell [Tue, 20 Dec 2011 07:09:28 +0000 (17:39 +1030)]
tdb: use same expansion factor logic when expanding for new recovery area.
If we're expanding because the current recovery area is too small, we
expand only the amount we need. This can quickly lead to exponential
growth when we have a slowly-expanding record (hence a
slowly-expanding transaction size).
Rusty Russell [Mon, 12 Dec 2011 03:20:05 +0000 (13:50 +1030)]
container_of: add container_off_var macro
Based on patch by Andrey Smirnov <andrew.smirnov@gmail.com>:
There is are certain use-cases when it is necessary to know the offset
of the member in a structure's memory layout. One such use-case can be
seen in `ccan/list/list.h' in macros `list_for_each' and
`list_for_each_safe'. This commit implements said functionality with
`container_of_var_off' macro.
Renamed container_of_var_off -> container_off_var now we have container_off.
Andrey Smirnov [Mon, 12 Dec 2011 03:19:57 +0000 (13:49 +1030)]
Makefile, configurator: Add additional debug flags to CFLAGS
Just adding `-g' to list of CFLAGS doesn't make gcc to generate debug
information required for macro expansion during debugging. Replacing
it with `-g3 -ggdb' rectifies this.
Rusty Russell [Wed, 7 Dec 2011 02:57:14 +0000 (13:27 +1030)]
autodata: add example, clean up description a little.
Also add #include to AUTODATA_TYPE example, so ccanlint doesn't try to
put it inside a function (which works, for this, but gives a warning
about unused variables).
Rusty Russell [Thu, 1 Dec 2011 06:14:51 +0000 (16:44 +1030)]
opt: add OPT_EARLY and opt_early_parse.
Parsing options like --verbose and --debug can be a pain. You need to
have everything set up before invoking parse_args(), but that may be a
significant amount of work, for which you may want verbose or
debugging enabled.
Thus the concept of "early" args: you can nominate arguments to be
parse before anything else, using opt_early_parse().
Rusty Russell [Wed, 30 Nov 2011 01:39:18 +0000 (12:09 +1030)]
tdb2: add a capability list from the header.
This allows even more extensibility in future: in particular, the top
bits of each capability tell us what to do if we don't understand it:
fail the open, fail to open for write, or don't try to check the
format.
tdb_check needs to understand the capability list so it can know to
skip over it: each element in the list is prefixed with the type tag
and the length.
Rusty Russell [Wed, 30 Nov 2011 00:23:42 +0000 (10:53 +1030)]
failtest: save and restore file state inside child (on-demand)
We currently save all files in the parent, and restore them once
the child is gone. That doesn't work in a case where the child
manipulates a file the parent doesn't currently have open, so
switch to a model where the child cleans itself up, using the
already-existing cleanup callbacks.
This means that we need to undo much more, especially restoring
file offsets. We also need to handle the case where we've already
closed the file, and now we're cleaning up. As a bonus, we now
handle open() with O_TRUNC properly.
The cleanup function now has two modes: one simply frees (so valgrind
doesn't complain about failtest leaking so the user can see real leaks
in their programs), the other restores things so the parent sees no
changes.
Rusty Russell [Tue, 29 Nov 2011 22:41:11 +0000 (09:11 +1030)]
tdb2: simplify failtest helper.
failtest now culls duplicates for itself (and more efficiently), so
don't replicate the logic here. It changes things a bit, because
failtest uses backtraces rather than a simple call point to find
duplicates.
Also, fix one case (in run-11-simple-fetch.c) where we simply exited
rather than using failtest_exit(). We got away with it before, because
we never hit that particular failure pattern.
Rusty Russell [Tue, 29 Nov 2011 22:40:11 +0000 (09:10 +1030)]
failtest: fix --debugpath
Calling failpath_string() here leaves the final letter randomly upper or
lower-cased, since call->fail is uninitialized. This means we sometimes
don't match the debug string.
1) Initialize call->fail here so it will match the debug string.
2) If our calls don't match --debugpath, abort.
3) Don't match the final letter (which may be upper or lower case)
when checking we're still on the path. We could do better, but this is
only a sanity-check anyway.
Rusty Russell [Tue, 29 Nov 2011 22:39:11 +0000 (09:09 +1030)]
failtest: internally eliminate duplicate calls.
If we can get a backtrace, we can automatically eliminate identical
failures. Surprisingly backtrace() is quite fast, but the savings for
the (naively-written) rbtree tests are impressive. ccanlint -v time
drops from 43 seconds to 6 seconds.
Rusty Russell [Tue, 29 Nov 2011 22:37:11 +0000 (09:07 +1030)]
failtest: simplify FAIL_PROBE.
When a failtest_hook returns fail_probe(), we start a counter to
continue for a little way. Replace this hack with a simple flag,
which disables further failures.
Interestingly, this doesn't reduce coverage on the current modules
which use failtest.
Rusty Russell [Tue, 29 Nov 2011 22:35:11 +0000 (09:05 +1030)]
failtest: use high-numbers file descriptors to stay out of the way.
We use file descriptors inside failtest; use dup2 to try to avoid
changing the file descriptor numbers used in the program (particularly
when programs mess with invalid file descriptors, they're likely to
still be invalid).
Rusty Russell [Tue, 29 Nov 2011 22:30:11 +0000 (09:00 +1030)]
failtest: fix internal cut & paste bug
failtest_malloc should use p->u.malloc not p->u.calloc. The layouts
are identical, so it doesn't matter, but it's confusing and leaves us
open to weird bugs in future should one change.
Rusty Russell [Tue, 29 Nov 2011 22:29:11 +0000 (08:59 +1030)]
ccanlint: fix depends_build_without_features
This depends on reduce_features; currently it tends to run before
that, so it doesn't think there are any features to reduce and doesn't
build or run the tests with reduced features.
Rusty Russell [Tue, 15 Nov 2011 02:12:25 +0000 (12:42 +1030)]
ccanlint: make fewer tests compulsory.
Compulsory means "malformed", we might get rid of it altogether, since
any test can mark "fail" and make ccanlint exit with non-zero status.
Now we only have four compulsory tests:
info_exists Module has _info file
depends_exist Module's CCAN dependencies can be found
objects_build Module object files can be built
module_builds Module can be built from object files
Rusty Russell [Tue, 1 Nov 2011 23:38:32 +0000 (10:08 +1030)]
tdb2: don't be fascist when TDB_VERSION1 is specified.
We currently insist that a tdb file be a version1 file if tdb_open() is
passed the TDB_VERSION1 flag; we fail if it's actually a tdb2.
But that makes generic wrappers harder, and is unlikely to be what the
user wants: if they do, they can check tdb_get_flags() & TDB_VERSION1
after opening.
Rusty Russell [Tue, 1 Nov 2011 23:06:29 +0000 (09:36 +1030)]
strset: set errno to ENOENT even if we return NULL.
It seems redundant: strset_test() and strset_clear() can only return NULL
when the string is not a member. However, it became clear in writing
ccan/tsort that it's much more convenient for callers if we set errno
in this case too, so they can pass it up.
Rusty Russell [Tue, 1 Nov 2011 23:05:29 +0000 (09:35 +1030)]
strmap: set errno to ENOENT even if we return NULL.
It seems redundant: strmap_get() and strmap_del() can only return NULL
when the string is not a member. However, it became clear in writing
ccan/tsort that it's much more convenient for callers if we set errno
in this case too, so they can pass it up.