ccan
8 years agoconfig.h: don't include in git repository.
Rusty Russell [Thu, 17 May 2012 03:44:18 +0000 (13:14 +0930)]
config.h: don't include in git repository.

We create and upload an example for the web anyway, if people really need it.

8 years agotools: Add create-ccan-tree utility
Jeremy Kerr [Wed, 16 May 2012 03:02:56 +0000 (11:02 +0800)]
tools: Add create-ccan-tree utility

Add a script to create a ccan source tree for inclusion into projects'
distributed code.

create-ccan-tree takes a list of ccan modules and copies them, plus any
dependent modules, into a target directory. Enough build infrastructure
is added to build libccan.a from the ccan modules.

Typical usage for my project that uses talloc and read-write-all:

  lib/ccan.git/tools/create-ccan-tree --exclude-tests \
lib/ccan/ talloc read-write-all

- this uses a git submodule in $PWDlib/ccan.git to create a source tree
in $PWD/lib/ccan, containing code for talloc and read-write-all.

Signed-off-by: Jeremy Kerr <jeremy.kerr@canonical.com>
8 years agoasort: fix test when typesafe_cb not supported. origin/suncc
Rusty Russell [Thu, 29 Mar 2012 04:28:33 +0000 (14:58 +1030)]
asort: fix test when typesafe_cb not supported.

Solaris CC seems to support typeof, but not __builtin_types_compatible_p.

8 years agocast: make sure suncc sees a constant.
Rusty Russell [Thu, 29 Mar 2012 04:28:33 +0000 (14:58 +1030)]
cast: make sure suncc sees a constant.

cast_const() et. al. are supposed to be a constant expression, so you can do things like:
static char *p = cast_const(char *, (const char *)"hello");

Unfortunately, a cast to intptr_t and arithmetic makes suncc reject it as
a constant expression.  We need the cast, because (1) the expression could be
a void *, so we can't just add to it, and (2) gcc complains with -Wcast-qual
without it.

So instead of adding BUILD_BUG_OR_ZERO, we use a ? :, which keeps everyone happy.

8 years agoccan_tokenizer: don't assume __VA_ARGS__ can be empty.
Rusty Russell [Thu, 29 Mar 2012 04:28:23 +0000 (14:58 +1030)]
ccan_tokenizer: don't assume __VA_ARGS__ can be empty.

That's a gcc extension.

8 years agocast: test/compile_fail-cast_static.c should fail without COMPOUND_LITERALS.
Rusty Russell [Tue, 27 Mar 2012 05:10:45 +0000 (15:40 +1030)]
cast: test/compile_fail-cast_static.c should fail without COMPOUND_LITERALS.

It still gave a warning on gcc, because casting a char to a char* gives a warning.  Not so on sun CC.

8 years agoccanlint: module_links test must link with module.
Rusty Russell [Tue, 27 Mar 2012 05:01:27 +0000 (15:31 +1030)]
ccanlint: module_links test must link with module.

Gcc doesn't include unused inline functions, even without -O, but Solaris CC
does, so fails to link block_pool where one inline function refers to a
non-inline function.

8 years agoasprintf: #include <stdarg.h> in test/run.c
Rusty Russell [Tue, 27 Mar 2012 04:37:50 +0000 (15:07 +1030)]
asprintf: #include <stdarg.h> in test/run.c

We get away with it on Linux, not on Solaris.

8 years agoccanlint: tests_compile_coverage needs GCC.
Rusty Russell [Tue, 27 Mar 2012 04:36:46 +0000 (15:06 +1030)]
ccanlint: tests_compile_coverage needs GCC.

We only support gcc/gcov so far.

8 years agoccanlint: add ilog dependency.
Rusty Russell [Tue, 27 Mar 2012 04:22:13 +0000 (14:52 +1030)]
ccanlint: add ilog dependency.

This only matters on non-gcc, where ilog functions are not just mapped to builtins.

8 years agotap: assume variadic macros.
Rusty Russell [Tue, 27 Mar 2012 04:08:13 +0000 (14:38 +1030)]
tap: assume variadic macros.

The Sun C compiler has them, but fails this overzealous check.

8 years agoopt: fix Solaris compile.
Rusty Russell [Tue, 27 Mar 2012 04:00:29 +0000 (14:30 +1030)]
opt: fix Solaris compile.

Need sys/termios.h for struct winsize.

8 years agoiscsi: use <sys/filio.h> on Solaris.
Rusty Russell [Tue, 27 Mar 2012 03:54:00 +0000 (14:24 +1030)]
iscsi: use <sys/filio.h> on Solaris.

Needed for FIONREAD.

8 years agoalloc: fix return of void.
Rusty Russell [Tue, 27 Mar 2012 03:45:19 +0000 (14:15 +1030)]
alloc: fix return of void.

Sun CC complains.

8 years agotools: don't use __attribute__
Rusty Russell [Tue, 27 Mar 2012 03:45:18 +0000 (14:15 +1030)]
tools: don't use __attribute__

We have compiler.h for that.

8 years agotools: fix bogus unreachable return.
Rusty Russell [Tue, 27 Mar 2012 03:45:18 +0000 (14:15 +1030)]
tools: fix bogus unreachable return.

8 years agotools/configurator: fix test for /proc/self/maps.
Rusty Russell [Tue, 27 Mar 2012 03:45:18 +0000 (14:15 +1030)]
tools/configurator: fix test for /proc/self/maps.

Header file was wrong, suncc gives warning about missing open prototype, causing configurator to fail.

8 years agoMakefile: generate config.h in two stages
Rusty Russell [Tue, 27 Mar 2012 03:45:16 +0000 (14:15 +1030)]
Makefile: generate config.h in two stages

This means we don't create an empty file if configurator fails.

8 years agofailtest: compile fix for OpenBSD
Rusty Russell [Wed, 21 Mar 2012 00:48:18 +0000 (11:18 +1030)]
failtest: compile fix for OpenBSD

OpenBSD doesn't idempotent-wrap sys/mman.h, so when we #define mmap to
an alternative, it fails to compile when sys/mman.h is included again.

Workaround is not to #define mmap to add arguments on Open BSD.

8 years agoMerge branch 'ccantool'
Rusty Russell [Mon, 19 Mar 2012 05:51:00 +0000 (16:21 +1030)]
Merge branch 'ccantool'

8 years agotools: move config.h reading to its own file
Rusty Russell [Mon, 19 Mar 2012 05:36:27 +0000 (16:06 +1030)]
tools: move config.h reading to its own file

Move ccan_dir determination out to its own function, rather than implying it
by the first time we get the manifest of a module.

8 years agotools: explicit find_ccan_dir()
Rusty Russell [Mon, 19 Mar 2012 05:15:45 +0000 (15:45 +1030)]
tools: explicit find_ccan_dir()

Move ccan_dir determination out to its own function, rather than implying it
by the first time we get the manifest of a module.

8 years agoccanlint: handle -x correctly with multiple modules.
Rusty Russell [Mon, 19 Mar 2012 05:14:59 +0000 (15:44 +1030)]
ccanlint: handle -x correctly with multiple modules.

After the first module we were resetting even the excluded tests, which caused
a segfault as they weren't initialized.

8 years agotools: determing ccan directory differently
Rusty Russell [Mon, 19 Mar 2012 05:14:13 +0000 (15:44 +1030)]
tools: determing ccan directory differently

Don't assume it's two levels above the module dir, instead look for
last "/ccan/" in path.  This is an incremental step towards nested
module directories.

8 years agotools: split out manifest from ccanlint
Rusty Russell [Mon, 19 Mar 2012 05:13:13 +0000 (15:43 +1030)]
tools: split out manifest from ccanlint

Other tools will soon want to use this, so make it independent.
(eg. ccantool).

8 years agotdb2: remove: it's now in SAMBA where it belongs.
Rusty Russell [Fri, 9 Mar 2012 02:55:03 +0000 (13:25 +1030)]
tdb2: remove: it's now in SAMBA where it belongs.

8 years agohtable: fix bug where first entry has hash of 0 or 1.
Rusty Russell [Fri, 9 Mar 2012 02:50:35 +0000 (13:20 +1030)]
htable: fix bug where first entry has hash of 0 or 1.

Thanks to Zolt├ín Lajos Kis for the bug report and test case!

8 years agofailtest: don't assume FD_SETSIZE is maximum runtime fd.
Rusty Russell [Thu, 8 Mar 2012 03:44:22 +0000 (14:14 +1030)]
failtest: don't assume FD_SETSIZE is maximum runtime fd.

This breaks when rlimit is less.  Unfortunately, valgrind (32 bit x86,
3.7.0.SVN, Ubuntu) fails to set the file limit properly on the test:
reducing it to the obvious getrlimit/setrlimit/getrlimit works fine,
so leaving diagnostics for another day.

8 years agottxml: made alloc failure consistent - return NULL
Daniel Burke [Sun, 26 Feb 2012 17:03:59 +0000 (03:33 +1030)]
ttxml: made alloc failure consistent - return NULL

8 years agottxml: removed cruft from tests
Daniel Burke [Sun, 26 Feb 2012 17:03:05 +0000 (03:33 +1030)]
ttxml: removed cruft from tests

8 years agottxml: fixed license test
Daniel Burke [Sun, 26 Feb 2012 17:02:39 +0000 (03:32 +1030)]
ttxml: fixed license test

8 years agotdb: delete from CCAN.
Rusty Russell [Tue, 14 Feb 2012 09:39:18 +0000 (20:09 +1030)]
tdb: delete from CCAN.

This was an early experiment in putting code into CCAN, but TDB is a public
library distributed as part of the Samba sources; there is no reason to
publish it here (especially now the unit tests are also in Samba).

8 years agoconfigurator: define HAVE_CCAN.
Rusty Russell [Tue, 14 Feb 2012 09:34:00 +0000 (20:04 +1030)]
configurator: define HAVE_CCAN.

An interesting case came up with the tdb2 code in SAMBA recently.  It's a
public library, and it doesn't want to be dependent on CCAN, but the header
uses cast and typesafe_cb for extra type safety.

A good solution to this is to put dummy versions under !HAVE_CCAN.  Of course,
any CCAN config.h should define HAVE_CCAN.

8 years agotdb2: copy tdb1's changed expansion logic.
Rusty Russell [Sat, 28 Jan 2012 04:14:43 +0000 (14:44 +1030)]
tdb2: copy tdb1's changed expansion logic.

TDB2 uses the same expansion logic as TDB1, which got factored out
recently.  So update TDB2 to match.

8 years agotdb2: tdb1: use same expansion factor logic when expanding for new recovery area.
Rusty Russell [Sat, 28 Jan 2012 04:14:43 +0000 (14:44 +1030)]
tdb2: tdb1: use same expansion factor logic when expanding for new recovery area.

If we're expanding because the current recovery area is too small, we
expand only the amount we need.  This can quickly lead to exponential
growth when we have a slowly-expanding record (hence a
slowly-expanding transaction size).

(Syncs with tdb1 SAMBA patch 3a2a755e3380a8f81374009d463cd06161352507)

8 years agotdb2: Avoid a malloc/memcpy in _tdb1_store
Volker Lendecke [Sat, 28 Jan 2012 04:14:36 +0000 (14:44 +1030)]
tdb2: Avoid a malloc/memcpy in _tdb1_store
(Syncs tdb2 with SAMBA tdb1's 664add17757836c5ee98618aef11371f412b6e44)

8 years agotdb2: careful on wrap.
Rusty Russell [Wed, 21 Dec 2011 05:54:41 +0000 (16:24 +1030)]
tdb2: careful on wrap.

It's much harder to wrap a 64-bit tdb2 than a 32-bit tdb1, but we should still
take care against bugs.

Also, we should *not* cast the length to a size_t when comparing it to
the stat result, in case size_t is 32 bit.

8 years agotdb2: be more careful on 4G files (tdb1).
Rusty Russell [Wed, 21 Dec 2011 05:44:51 +0000 (16:14 +1030)]
tdb2: be more careful on 4G files (tdb1).

I came across a tdb which had wrapped to 4G + 4K, and the contents had been
destroyed by processes which thought it only 4k long.  Fix this by checking
on open, and making tdb_oob() check for wrap itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Ported from tdb1 SAMBA commit b64494535dc62f4073fc6302847593ed6e6ec38b)

8 years agotdb: don't free old recovery area when expanding if already at EOF.
Rusty Russell [Tue, 20 Dec 2011 07:10:57 +0000 (17:40 +1030)]
tdb: don't free old recovery area when expanding if already at EOF.

We allocate a new recovery area by expanding the file.  But if the
recovery area is already at the end of file (as shown in at least one
client case), we can simply expand the record, rather than freeing it
and creating a new one.

8 years agotdb: use same expansion factor logic when expanding for new recovery area.
Rusty Russell [Tue, 20 Dec 2011 07:09:28 +0000 (17:39 +1030)]
tdb: use same expansion factor logic when expanding for new recovery area.

If we're expanding because the current recovery area is too small, we
expand only the amount we need.  This can quickly lead to exponential
growth when we have a slowly-expanding record (hence a
slowly-expanding transaction size).

8 years agotdb: test exponential recovery expansion problem.
Rusty Russell [Tue, 20 Dec 2011 07:07:45 +0000 (17:37 +1030)]
tdb: test exponential recovery expansion problem.

8 years agoopt: much prettier usage (using terminal size)
Rusty Russell [Mon, 19 Dec 2011 01:21:45 +0000 (11:51 +1030)]
opt: much prettier usage (using terminal size)

8 years agoopt: handle -Wcast-qual warnings.
Rusty Russell [Fri, 16 Dec 2011 00:09:47 +0000 (10:39 +1030)]
opt: handle -Wcast-qual warnings.

8 years agoopt: don't leak on exit (valgrind complains).
Rusty Russell [Thu, 15 Dec 2011 03:45:47 +0000 (14:15 +1030)]
opt: don't leak on exit (valgrind complains).

Recent real usage case showed this leak when we call opt_usage_and_exit:
we don't bother freeing before exit.  With valgrind, it matters.

8 years agolist.h: opaque list iteration functionality
Andrey Smirnov [Mon, 12 Dec 2011 03:22:44 +0000 (13:52 +1030)]
list.h: opaque list iteration functionality

See https://github.com/rustyrussell/ccan/pull/1#issuecomment-2856561
for details.
(Reworked with minor cleanups -- Rusty)

8 years agocontainer_of: add container_off_var macro
Rusty Russell [Mon, 12 Dec 2011 03:20:05 +0000 (13:50 +1030)]
container_of: add container_off_var macro

Based on patch by Andrey Smirnov <andrew.smirnov@gmail.com>:

    There is are certain use-cases when it is necessary to know the offset
    of the member in a structure's memory layout. One such use-case can be
    seen in `ccan/list/list.h' in macros `list_for_each' and
    `list_for_each_safe'. This commit implements said functionality with
    `container_of_var_off' macro.

Renamed container_of_var_off -> container_off_var now we have container_off.

8 years agocheck_type: fix incorrect documentation.
Rusty Russell [Mon, 12 Dec 2011 03:20:05 +0000 (13:50 +1030)]
check_type: fix incorrect documentation.

8 years agocontainer_of: don't put member_ptr in container_off.
Rusty Russell [Mon, 12 Dec 2011 03:20:05 +0000 (13:50 +1030)]
container_of: don't put member_ptr in container_off.

It's convenient to check that the member is the given type, but we can leave
that to the callers.

8 years agoMakefile, configurator: Add additional debug flags to CFLAGS
Andrey Smirnov [Mon, 12 Dec 2011 03:19:57 +0000 (13:49 +1030)]
Makefile, configurator: Add additional debug flags to CFLAGS
Just adding `-g' to list of CFLAGS doesn't make gcc to generate debug
information required for macro expansion during debugging. Replacing
it with `-g3 -ggdb' rectifies this.

(configurator.c commit folded by Rusty)

8 years agoopt: suggest they look at ccan/autodata.
Rusty Russell [Wed, 7 Dec 2011 02:57:27 +0000 (13:27 +1030)]
opt: suggest they look at ccan/autodata.

Nice for distributed option declaration.

8 years agoautodata: add example, clean up description a little.
Rusty Russell [Wed, 7 Dec 2011 02:57:14 +0000 (13:27 +1030)]
autodata: add example, clean up description a little.

Also add #include to AUTODATA_TYPE example, so ccanlint doesn't try to
put it inside a function (which works, for this, but gives a warning
about unused variables).

8 years agotools/doc_extract: trim trailing blank lines on sections.
Rusty Russell [Wed, 7 Dec 2011 02:55:41 +0000 (13:25 +1030)]
tools/doc_extract: trim trailing blank lines on sections.

Makes things a bit neater.

8 years agoweb: publish See Also: for modules which supply it.
Rusty Russell [Wed, 7 Dec 2011 02:55:41 +0000 (13:25 +1030)]
web: publish See Also: for modules which supply it.

8 years agotools/doc_extract: simplify handling fields with spaces in them.
Rusty Russell [Wed, 7 Dec 2011 02:55:41 +0000 (13:25 +1030)]
tools/doc_extract: simplify handling fields with spaces in them.

eg. "See also" can match "see-also".

8 years agoccanlint: use ccan/autodata
Rusty Russell [Mon, 5 Dec 2011 12:48:50 +0000 (23:18 +1030)]
ccanlint: use ccan/autodata

Gets rid of generated file.

8 years agoautodata: stash pointers in a binary.
Rusty Russell [Mon, 5 Dec 2011 12:48:21 +0000 (23:18 +1030)]
autodata: stash pointers in a binary.

This is a more portable variant of the ELF section trick.

8 years agoconfigurator: HAVE_SECTION_START_STOP
Rusty Russell [Mon, 5 Dec 2011 12:17:46 +0000 (22:47 +1030)]
configurator: HAVE_SECTION_START_STOP

8 years agoptr_valid: test whether a ptr is valid.
Rusty Russell [Mon, 5 Dec 2011 12:16:46 +0000 (22:46 +1030)]
ptr_valid: test whether a ptr is valid.

Very slow, but sometimes you need to know without crashing.

8 years agoconfigurator: HAVE_PROC_SELF_MAPS
Rusty Russell [Mon, 5 Dec 2011 03:46:33 +0000 (14:16 +1030)]
configurator: HAVE_PROC_SELF_MAPS

8 years agoobjset: new module.
Rusty Russell [Sun, 4 Dec 2011 22:50:34 +0000 (09:20 +1030)]
objset: new module.

8 years agoccanlint.1: fix description of per-test options for tests_pass_valgrind.
Rusty Russell [Sun, 4 Dec 2011 22:49:34 +0000 (09:19 +1030)]
ccanlint.1: fix description of per-test options for tests_pass_valgrind.

8 years agodoc: man page for ccanlint.
Rusty Russell [Fri, 2 Dec 2011 03:15:08 +0000 (13:45 +1030)]
doc: man page for ccanlint.

8 years agoccanlint: remove argument to -k/--keep
Rusty Russell [Fri, 2 Dec 2011 03:37:24 +0000 (14:07 +1030)]
ccanlint: remove argument to -k/--keep

It's much easier to simply say "keep all", and it simplifies the code
quite a bit.

8 years agoccanlint: use -t for --target not --timeout
Rusty Russell [Fri, 2 Dec 2011 03:37:23 +0000 (14:07 +1030)]
ccanlint: use -t for --target not --timeout

I use --target far more than --timeout.

8 years agoccanlint: use traversal to trim tests we want to skip.
Rusty Russell [Fri, 2 Dec 2011 03:37:22 +0000 (14:07 +1030)]
ccanlint: use traversal to trim tests we want to skip.

Rather than using a separate index of tetst to skip, make sure tests
we don't want to run are removed from the graph of tests to start
with.

This means that the "skip" field is only set when we fail a test (and
thus must be reset when we test the next module).

8 years agoccanlint: use node traverse for --target.
Rusty Russell [Fri, 2 Dec 2011 03:37:21 +0000 (14:07 +1030)]
ccanlint: use node traverse for --target.

Rather than walking the list of tests manually, use
dgraph_traverse_to() to run the tests.

8 years agoccanlint: use dgraph module.
Rusty Russell [Fri, 2 Dec 2011 03:37:21 +0000 (14:07 +1030)]
ccanlint: use dgraph module.

Instead of a linked list of tests, we use dgraph and strmap.

8 years agoccanlint: remove wart from info options.
Rusty Russell [Fri, 2 Dec 2011 03:37:20 +0000 (14:07 +1030)]
ccanlint: remove wart from info options.

Make info_exists() call add_info_options() directly, rather than detecting
that we've found _info inside the loop of tests.

It means targeting is now a global, which is a bit sucky.

8 years agodgraph: new module for directed graphs.
Rusty Russell [Fri, 2 Dec 2011 03:36:58 +0000 (14:06 +1030)]
dgraph: new module for directed graphs.

8 years agoccanlint: parse --verbose before anything else.
Rusty Russell [Fri, 2 Dec 2011 03:10:27 +0000 (13:40 +1030)]
ccanlint: parse --verbose before anything else.

This way, we can debug really early failures (eg. during option
parsing).

8 years agoccanlint: get rid of separate class of compulsory tests.
Rusty Russell [Thu, 1 Dec 2011 06:15:51 +0000 (16:45 +1030)]
ccanlint: get rid of separate class of compulsory tests.

It's reduced to a flag which means we stop processing the module.  We have
to enhance our dependencies a bit, but they've always been quite good.

This simplifies things quite a bit.

8 years agoopt: add OPT_EARLY and opt_early_parse.
Rusty Russell [Thu, 1 Dec 2011 06:14:51 +0000 (16:44 +1030)]
opt: add OPT_EARLY and opt_early_parse.

Parsing options like --verbose and --debug can be a pain.  You need to
have everything set up before invoking parse_args(), but that may be a
significant amount of work, for which you may want verbose or
debugging enabled.

Thus the concept of "early" args: you can nominate arguments to be
parse before anything else, using opt_early_parse().

8 years agostrset, strmap: invert iterator function meaning.
Rusty Russell [Thu, 1 Dec 2011 06:13:51 +0000 (16:43 +1030)]
strset, strmap: invert iterator function meaning.

Make a false return abort the iteration, not true.

The old way makes sense for search functions (true == I found it), but
other kinds of iteration are more common (brute force search is
probably dumb).

8 years agosetset: fix API to match strmap and common sense.
Rusty Russell [Thu, 1 Dec 2011 06:12:51 +0000 (16:42 +1030)]
setset: fix API to match strmap and common sense.

strset_set -> strset_add
strset_clear -> strset_del
strset_test -> strset_get
strset_destroy -> strset_clear

8 years agostrmap: allow const arguments to strset_iterate().
Rusty Russell [Thu, 1 Dec 2011 06:11:51 +0000 (16:41 +1030)]
strmap: allow const arguments to strset_iterate().

8 years agostrset: allow const arguments to strset_iterate().
Rusty Russell [Thu, 1 Dec 2011 06:10:51 +0000 (16:40 +1030)]
strset: allow const arguments to strset_iterate().

8 years agoopt: fix up outdated comments in documentation.
Rusty Russell [Thu, 1 Dec 2011 06:09:51 +0000 (16:39 +1030)]
opt: fix up outdated comments in documentation.

8 years agoopt: get rid of last remnant of getopt.
Rusty Russell [Thu, 1 Dec 2011 06:09:50 +0000 (16:39 +1030)]
opt: get rid of last remnant of getopt.

Testing code still using it.

8 years agotlist: remove type arg from tlist_top(), tlist_tail()
Rusty Russell [Thu, 1 Dec 2011 06:09:47 +0000 (16:39 +1030)]
tlist: remove type arg from tlist_top(), tlist_tail()

With the type canary, it's unnecessary.  Though the implementation is
a bit more awkward since they longer map directly through to list_top/tail.

8 years agolist: don't multiple-evaluate arguments to tlist_top and tlist_tail
Rusty Russell [Wed, 30 Nov 2011 03:02:18 +0000 (13:32 +1030)]
list: don't multiple-evaluate arguments to tlist_top and tlist_tail

8 years agocontainer_of: add container_off() helper.
Rusty Russell [Wed, 30 Nov 2011 03:02:06 +0000 (13:32 +1030)]
container_of: add container_off() helper.

Just like container_of() (with typechecking, etc), but return the offset
to the enclosing structure.

8 years agottxml: exclude license_comment check so it doesn't fail ccanlint.
Rusty Russell [Wed, 30 Nov 2011 02:41:00 +0000 (13:11 +1030)]
ttxml: exclude license_comment check so it doesn't fail ccanlint.

8 years agotdb2: display capability information in tdb_summary()
Rusty Russell [Wed, 30 Nov 2011 02:39:07 +0000 (13:09 +1030)]
tdb2: display capability information in tdb_summary()

This means we know they're there in future, and what restrictions they
carry.

8 years agotdb2: add a capability list from the header.
Rusty Russell [Wed, 30 Nov 2011 01:39:18 +0000 (12:09 +1030)]
tdb2: add a capability list from the header.

This allows even more extensibility in future: in particular, the top
bits of each capability tell us what to do if we don't understand it:
fail the open, fail to open for write, or don't try to check the
format.

tdb_check needs to understand the capability list so it can know to
skip over it: each element in the list is prefixed with the type tag
and the length.

8 years agotdb2: provide tdb_layout_write() rather than implying it by new_tdb_layout arg.
Rusty Russell [Wed, 30 Nov 2011 00:42:22 +0000 (11:12 +1030)]
tdb2: provide tdb_layout_write() rather than implying it by new_tdb_layout arg.

Neater API.

8 years agotdb2: add an internal TDB_CANT_CHECK flag.
Rusty Russell [Wed, 30 Nov 2011 00:42:22 +0000 (11:12 +1030)]
tdb2: add an internal TDB_CANT_CHECK flag.

This will be used shortly to indicate that a TDB2 file indicates it
cannot be checked.

8 years agofailtest: do trace via a FILE, not an fd.
Rusty Russell [Wed, 30 Nov 2011 00:42:03 +0000 (11:12 +1030)]
failtest: do trace via a FILE, not an fd.

The current mix of writing to an fd doesn't mix as well with
stderr/stdout.  And writing to a FILE * is more portable.

8 years agofailtest: add --trace to replace --tracepath
Rusty Russell [Wed, 30 Nov 2011 00:40:30 +0000 (11:10 +1030)]
failtest: add --trace to replace --tracepath

This gives a much deeper insight into what failtest is doing; good for
debugging failtest itself.

8 years agofailtest: save and restore file state inside child (on-demand)
Rusty Russell [Wed, 30 Nov 2011 00:23:42 +0000 (10:53 +1030)]
failtest: save and restore file state inside child (on-demand)

We currently save all files in the parent, and restore them once
the child is gone.  That doesn't work in a case where the child
manipulates a file the parent doesn't currently have open, so
switch to a model where the child cleans itself up, using the
already-existing cleanup callbacks.

This means that we need to undo much more, especially restoring
file offsets.  We also need to handle the case where we've already
closed the file, and now we're cleaning up.  As a bonus, we now
handle open() with O_TRUNC properly.

The cleanup function now has two modes: one simply frees (so valgrind
doesn't complain about failtest leaking so the user can see real leaks
in their programs), the other restores things so the parent sees no
changes.

8 years agofailtest: add comment about limitations of untracked pointers.
Rusty Russell [Tue, 29 Nov 2011 22:48:11 +0000 (09:18 +1030)]
failtest: add comment about limitations of untracked pointers.

8 years agofailtest: fix open in tests.
Rusty Russell [Tue, 29 Nov 2011 22:47:11 +0000 (09:17 +1030)]
failtest: fix open in tests.

The argument is wrong, so the return was -1.  That currently works,
but it will beak with the next patch.  Fix the typo.

8 years agofailtest: fix failpath on open.
Rusty Russell [Tue, 29 Nov 2011 22:46:11 +0000 (09:16 +1030)]
failtest: fix failpath on open.

And separate out the code which follows --failpath so failtest_close()
can use it too.

8 years agofailtest: report failpath problems correctly.
Rusty Russell [Tue, 29 Nov 2011 22:45:11 +0000 (09:15 +1030)]
failtest: report failpath problems correctly.

It was the wrong way around, and also it's better to show the whole
remaining failpath rather than the current letter.

8 years agofailtest: stop when leak detected.
Rusty Russell [Tue, 29 Nov 2011 22:44:11 +0000 (09:14 +1030)]
failtest: stop when leak detected.

Don't continue when we report a leak: tell the parent it's a failure.

8 years agotdb2: suppress failtest more than once on mmap.
Rusty Russell [Tue, 29 Nov 2011 22:43:11 +0000 (09:13 +1030)]
tdb2: suppress failtest more than once on mmap.

Now we test failing mmap, ccanlint -v time has increased from 200 to
330 seconds.  Worse, tests no time out on my laptop.

Fix this, by preventing us from going down that particular rabbit hole.
ccanlint -v now takes 201 seconds again.

8 years agofailtest: catch mmap.
Rusty Russell [Tue, 29 Nov 2011 22:42:11 +0000 (09:12 +1030)]
failtest: catch mmap.

mmap can also fail on out-of-memory, and for the coming change to the
way we save and restore files we want to know about them anyway.

8 years agotdb2: simplify failtest helper.
Rusty Russell [Tue, 29 Nov 2011 22:41:11 +0000 (09:11 +1030)]
tdb2: simplify failtest helper.

failtest now culls duplicates for itself (and more efficiently), so
don't replicate the logic here.  It changes things a bit, because
failtest uses backtraces rather than a simple call point to find
duplicates.

Also, fix one case (in run-11-simple-fetch.c) where we simply exited
rather than using failtest_exit().  We got away with it before, because
we never hit that particular failure pattern.

8 years agofailtest: fix --debugpath
Rusty Russell [Tue, 29 Nov 2011 22:40:11 +0000 (09:10 +1030)]
failtest: fix --debugpath

Calling failpath_string() here leaves the final letter randomly upper or
lower-cased, since call->fail is uninitialized.  This means we sometimes
don't match the debug string.

1) Initialize call->fail here so it will match the debug string.
2) If our calls don't match --debugpath, abort.
3) Don't match the final letter (which may be upper or lower case)
   when checking we're still on the path.  We could do better, but this is
   only a sanity-check anyway.

8 years agofailtest: internally eliminate duplicate calls.
Rusty Russell [Tue, 29 Nov 2011 22:39:11 +0000 (09:09 +1030)]
failtest: internally eliminate duplicate calls.

If we can get a backtrace, we can automatically eliminate identical
failures.  Surprisingly backtrace() is quite fast, but the savings for
the (naively-written) rbtree tests are impressive.  ccanlint -v time
drops from 43 seconds to 6 seconds.

8 years agoconfigurator: HAVE_BACKTRACE
Rusty Russell [Tue, 29 Nov 2011 22:38:11 +0000 (09:08 +1030)]
configurator: HAVE_BACKTRACE