ccan
9 years agoccanlint: add is_excluded() helper.
Rusty Russell [Mon, 29 Aug 2011 23:56:39 +0000 (09:26 +0930)]
ccanlint: add is_excluded() helper.

We'll need this in a moment.

9 years agotools: simplify get_safe_ccan_deps
Rusty Russell [Mon, 29 Aug 2011 23:55:39 +0000 (09:25 +0930)]
tools: simplify get_safe_ccan_deps

There's no reason to hand in the name of the compiled _info file,
since we don't compile it up in safe mode.

9 years agoccanlint: fix warning about bogus return value.
Rusty Russell [Mon, 29 Aug 2011 23:54:39 +0000 (09:24 +0930)]
ccanlint: fix warning about bogus return value.

9 years agoccanlint: rename obj_list in examples_compile.c to example_obj_list.
Rusty Russell [Mon, 29 Aug 2011 23:53:39 +0000 (09:23 +0930)]
ccanlint: rename obj_list in examples_compile.c to example_obj_list.

Trivial cleanup, but differentiates from the general obj_list used
elsewhere.

9 years agofailtest: replay --failpath correctly on really failing opens.
Rusty Russell [Mon, 29 Aug 2011 23:52:39 +0000 (09:22 +0930)]
failtest: replay --failpath correctly on really failing opens.

When an open() call fails, we don't inject an error, but we didn't
apply this logic when replaying a failpath, with strange results.

9 years agofailtest: add --show-slowest
Rusty Russell [Mon, 29 Aug 2011 23:51:39 +0000 (09:21 +0930)]
failtest: add --show-slowest

This is useful for seeing where failures cause us to run slowly
(ie. we continue for a long time before terminating).

9 years agofailtest: use ccan/time
Rusty Russell [Mon, 29 Aug 2011 23:50:39 +0000 (09:20 +0930)]
failtest: use ccan/time

9 years agofailtest: use 64 bit offsets if available.
Rusty Russell [Mon, 29 Aug 2011 08:03:25 +0000 (17:33 +0930)]
failtest: use 64 bit offsets if available.

We have to be consistent with the caller, so turn on 64 bit offsets if
available for both the caller and us.

9 years agohtable: fix tools/speed.
Rusty Russell [Mon, 29 Aug 2011 08:02:41 +0000 (17:32 +0930)]
htable: fix tools/speed.

As pointed out by Christian Thaeter, it has bitrotted.

9 years agobuild_assert: Remove stale LGPL license comment.
Joey Adams [Tue, 23 Aug 2011 22:35:53 +0000 (18:35 -0400)]
build_assert: Remove stale LGPL license comment.

9 years agotdb2: suppress valgrind testing on one test.
Rusty Russell [Wed, 24 Aug 2011 03:23:51 +0000 (12:53 +0930)]
tdb2: suppress valgrind testing on one test.

Now valgrind children report failures, this one test fails.

9 years agotdb: don't leak memory in tests.
Rusty Russell [Wed, 24 Aug 2011 03:23:08 +0000 (12:53 +0930)]
tdb: don't leak memory in tests.

9 years agotdb: only mark two files as failing, don't turn off all valgrind tests.
Rusty Russell [Wed, 24 Aug 2011 03:23:00 +0000 (12:53 +0930)]
tdb: only mark two files as failing, don't turn off all valgrind tests.

9 years agotdb: don't define _XOPEN_SOURCE ourselves, let config.h do it.
Rusty Russell [Wed, 24 Aug 2011 03:22:59 +0000 (12:52 +0930)]
tdb: don't define _XOPEN_SOURCE ourselves, let config.h do it.

This is the CCAN Way.  Plus, eliminates a compile warning here.

9 years agotdb: don't use 'private' in headers.
Rusty Russell [Wed, 24 Aug 2011 03:22:57 +0000 (12:52 +0930)]
tdb: don't use 'private' in headers.

As per the SAMBA version.

9 years agoccanlint: tests_pass_valgrind_noleaks: handle FAIL option on tests.
Rusty Russell [Wed, 24 Aug 2011 03:22:54 +0000 (12:52 +0930)]
ccanlint: tests_pass_valgrind_noleaks: handle FAIL option on tests.

Now we can mark test files as FAIL and not be bothered by them.

9 years agoccanlint: allow per-file restrictions, so we don't have to remove all tests.
Rusty Russell [Wed, 24 Aug 2011 03:22:46 +0000 (12:52 +0930)]
ccanlint: allow per-file restrictions, so we don't have to remove all tests.

For example, valgrind can't handle some things, so we may need to
switch it off, but we don't want to do that for every test.

So, start a standard format for per-file restrictions, eg:

    Ccanlint: tests_pass_valgrind test/foo.c:FAIL

For the moment, only tests_pass_valgrind takes options, so it has to
change: we now have a helper which returns the array of options
applicable to a given file.

9 years agofailtest: plug memory leak.
Rusty Russell [Wed, 24 Aug 2011 03:22:28 +0000 (12:52 +0930)]
failtest: plug memory leak.

Valgrind notes that our child processes leak memory.

9 years agoccanlint: make valgrind understand child output.
Rusty Russell [Wed, 24 Aug 2011 03:22:24 +0000 (12:52 +0930)]
ccanlint: make valgrind understand child output.

We turned on --child-silent-after-fork=yes, because the output
confused our dumb parser.  Now we split output by prefix ("==<pid>==")
and then we can sort out the leaks from the errors.

This means we find more errors.... next patch!

9 years agoccanlint: use ccan/time
Rusty Russell [Wed, 24 Aug 2011 03:22:04 +0000 (12:52 +0930)]
ccanlint: use ccan/time

Our own dogfood, and it's yummy!

9 years agotools: reorder object files into alphabetical order.
Rusty Russell [Wed, 24 Aug 2011 03:22:00 +0000 (12:52 +0930)]
tools: reorder object files into alphabetical order.

Simplifies future patches.

9 years agotalloc_link: close memory leak in test.
Rusty Russell [Tue, 23 Aug 2011 02:51:10 +0000 (12:21 +0930)]
talloc_link: close memory leak in test.

9 years agoread_write_all: fix leak in tests.
Rusty Russell [Tue, 23 Aug 2011 02:51:10 +0000 (12:21 +0930)]
read_write_all: fix leak in tests.

9 years agoopt: neaten tests with helpers.
Rusty Russell [Tue, 23 Aug 2011 02:51:05 +0000 (12:21 +0930)]
opt: neaten tests with helpers.

9 years agonoerr: remove bogus LICENSE link.
Rusty Russell [Tue, 23 Aug 2011 02:41:11 +0000 (12:11 +0930)]
noerr: remove bogus LICENSE link.

9 years agojmap: fix jmap_free, tests.
Rusty Russell [Tue, 23 Aug 2011 02:30:13 +0000 (12:00 +0930)]
jmap: fix jmap_free, tests.

Memory leak in tests revealed bug in jmap_free.  Also, out-by-one
error.

9 years agojbitset: fix examples, remove test memory leak.
Rusty Russell [Tue, 23 Aug 2011 02:06:49 +0000 (11:36 +0930)]
jbitset: fix examples, remove test memory leak.

9 years agograb_file: close memory leak in test.
Rusty Russell [Tue, 23 Aug 2011 02:01:04 +0000 (11:31 +0930)]
grab_file: close memory leak in test.

9 years agoalloc: fix !HAVE_BUILTIN_POPCOUNTL code.
Rusty Russell [Tue, 23 Aug 2011 01:54:21 +0000 (11:24 +0930)]
alloc: fix !HAVE_BUILTIN_POPCOUNTL code.

9 years agoccanlint: fix dependencies on tests_pass_without_features.
Rusty Russell [Mon, 22 Aug 2011 07:50:06 +0000 (17:20 +0930)]
ccanlint: fix dependencies on tests_pass_without_features.

We can only test the featureless tests, once we've built them!

9 years agotdb2: fix intermittant failure in run-50-multiple-freelists-fail.c
Rusty Russell [Mon, 22 Aug 2011 03:17:08 +0000 (12:47 +0930)]
tdb2: fix intermittant failure in run-50-multiple-freelists-fail.c

layout.c's TDB creation functions were incorrect in case of a hash
collision, causing occasional failure.  Make it always use the
(previously-failing) seed value, and fix it.

9 years agobdelta: new module for binary diff/patch
Joey Adams [Sat, 20 Aug 2011 02:29:44 +0000 (22:29 -0400)]
bdelta: new module for binary diff/patch

9 years agoblock_pool, ccan_tokenizer, stringmap: add ccanlint license suppressions.
Rusty Russell [Tue, 16 Aug 2011 06:04:14 +0000 (15:34 +0930)]
block_pool, ccan_tokenizer, stringmap: add ccanlint license suppressions.

9 years agoarray_size: relicense under public domain.
Rusty Russell [Tue, 16 Aug 2011 05:47:54 +0000 (15:17 +0930)]
array_size: relicense under public domain.

It's just a header, I don't care what's done with it.

9 years agocheck_type: remove erroneous license line (it's now public domain)
Rusty Russell [Tue, 16 Aug 2011 05:43:37 +0000 (15:13 +0930)]
check_type: remove erroneous license line (it's now public domain)

9 years agocharset: Updated copyright year, and set to version 0.3
Joey Adams [Mon, 15 Aug 2011 06:45:22 +0000 (02:45 -0400)]
charset: Updated copyright year, and set to version 0.3

9 years agoccan_tokenizer: Corrected LICENSE link so it points to BSD-3CLAUSE.
Joey Adams [Mon, 15 Aug 2011 06:41:40 +0000 (02:41 -0400)]
ccan_tokenizer: Corrected LICENSE link so it points to BSD-3CLAUSE.

9 years agobtree: Changed license from BSD-3 to MIT, and set version to 0.2
Joey Adams [Mon, 15 Aug 2011 06:36:33 +0000 (02:36 -0400)]
btree: Changed license from BSD-3 to MIT, and set version to 0.2

NOTE: btree was originally copyright 2010, and has not been
touched by me since then.  I don't know if changing the license
to something more permissive requires updating the copyright year
or not.

9 years agostringmap: Corrected LICENSE link so it points to BSD-3CLAUSE.
Joey Adams [Mon, 15 Aug 2011 06:29:05 +0000 (02:29 -0400)]
stringmap: Corrected LICENSE link so it points to BSD-3CLAUSE.

9 years agoavl: Added LICENSE link, and set version to 0.1
Joey Adams [Mon, 15 Aug 2011 06:26:17 +0000 (02:26 -0400)]
avl: Added LICENSE link, and set version to 0.1

9 years agoblock_pool: Changed license from BSD-3 to MIT, and set version to 0.1
Joey Adams [Mon, 15 Aug 2011 06:06:17 +0000 (02:06 -0400)]
block_pool: Changed license from BSD-3 to MIT, and set version to 0.1

NOTE: block_pool was originally copyright 2009, and has not been
touched by me since then.  I don't know if changing the license
to something more permissive requires updating the copyright year
or not.

9 years agodarray: Changed license from BSD-3 to MIT, and updated copyright year.
Joey Adams [Mon, 15 Aug 2011 05:47:58 +0000 (01:47 -0400)]
darray: Changed license from BSD-3 to MIT, and updated copyright year.

9 years agoopt: complete coverage, enhance opt_free_table.
Rusty Russell [Sun, 14 Aug 2011 01:04:44 +0000 (10:34 +0930)]
opt: complete coverage, enhance opt_free_table.

No point checking malloc failure in usage(), since we don't elsewhere.
We get 100% coverage with -O (due to code elimination) or 64 bit.

9 years agoopt: fix warnings in test, fix endian assumptions.
Rusty Russell [Sun, 14 Aug 2011 00:35:41 +0000 (10:05 +0930)]
opt: fix warnings in test, fix endian assumptions.

In particular, handing an pointer to ULL where a pointer to UL is expected
won't work on big endian.

9 years agoopt: functions to show integer values with kMGTPE suffixes
Douglas Bagnall [Sat, 13 Aug 2011 12:31:42 +0000 (22:01 +0930)]
opt: functions to show integer values with kMGTPE suffixes

As with the set_ functions, there are twelve permutations of integer size,
base, and signedness.  The supported sizes are int, long, and long long.

For example, this:

 char buf1[OPT_SHOW_LEN];
 char buf2[OPT_SHOW_LEN];
 unsigned i = 1024000;
 opt_show_uintval_bi(buf1, &i);
 opt_show_uintval_si(buf2, &i);

will put "1000k" in buf1, and "1024k" in buf2.

Unlike the opt_set_ functions, these use unsigned arithmetic for unsigned values.

(32 bit bug using sizeof(suffixes) instead of strlen(suffixes) fixed by Rusty)

9 years agoopt: incidental comment and whitespace repair
Douglas Bagnall [Sat, 13 Aug 2011 12:19:59 +0000 (21:49 +0930)]
opt: incidental comment and whitespace repair

This comment occurred in a couple of places:

 /* Set an integer value, various forms.  Sets to 1 on arg == NULL. */

One instance was clearly spurious, while the other was misleading.

Another resolution to this mismatch would be to add
"if (arg == NULL){*l = 1; return NULL}" somewhere, but I suspect
it may have been left out/removed because someone thought better.

9 years agoopt: add integer helpers that accept k, M, G, T, P, E suffixes
Douglas Bagnall [Sat, 13 Aug 2011 12:19:59 +0000 (21:49 +0930)]
opt: add integer helpers that accept k, M, G, T, P, E suffixes

These functions come in two flavours: those ending with "_si", which
have 1000-based interpretations of the suffixes; and those ending with
"_bi", which use base 1024.  There are versions for signed and
unsigned int, long, and long long destinations, with tests for all 12
new functions.  The tests get a bit repetitive, I am afraid.

As an example, if the -x option were using the opt_set_intval_bi
function, then all of these would do the same thing:

$ foo -x 5M
$ foo -x $((5 * 1024 * 1024))
$ foo -x 5242880
$ foo -x 5120k

quite what that thing is depends on the size of your int -- people
with 16 bit ints would see an "out of range" error message.

The arithmetic for unsigned variations is actually done using signed
long long integers, so the maximum possible value is LLONG_MAX, not
ULLONG_MAX.  This follows the practice of existing functions, and
avoids tedious work.

9 years agofailtest: fix silent exit when top-level return FAIL_PROBE
Rusty Russell [Mon, 1 Aug 2011 08:29:09 +0000 (17:59 +0930)]
failtest: fix silent exit when top-level return FAIL_PROBE

We were missing failed tests: if the top-level returns FAIL_PROBE, we would
exit; this should only apply to children.

9 years agotdb2: fix line numbers for tests.
Rusty Russell [Mon, 1 Aug 2011 08:29:08 +0000 (17:59 +0930)]
tdb2: fix line numbers for tests.

9 years agocast: downgrade license from LGPL3+ to LGPLv2.1+
Rusty Russell [Fri, 22 Jul 2011 12:13:39 +0000 (21:43 +0930)]
cast: downgrade license from LGPL3+ to LGPLv2.1+

Kirill A. Shutemov asked for libgit.  I would say they should upgrade their
license, but libhx on which these are based is also LGPLv2.1 or later, so
I prefer to match that.

9 years agoisaac, crcsync: acknowledge licensing issues.
Rusty Russell [Thu, 21 Jul 2011 05:20:00 +0000 (14:50 +0930)]
isaac, crcsync: acknowledge licensing issues.

The recently added ccanlint licensing checks revealed several cases
where the published license of a module is misleading: a dependency of
that module has a stricter license (eg. a public domain module which
depends on a GPL one).

Where these are my modules, I've fixed them.  Otherwise I'm overriding
the checks for the moment, and asking the authors what they want to do.

9 years agoccan/noerr: fix compiler warning with const strings.
Rusty Russell [Thu, 21 Jul 2011 05:20:00 +0000 (14:50 +0930)]
ccan/noerr: fix compiler warning with const strings.

9 years agovarious: add LICENSE comments.
Rusty Russell [Thu, 21 Jul 2011 05:19:56 +0000 (14:49 +0930)]
various: add LICENSE comments.

9 years agoccanlint: handle DOS-style \r\n lines when parsing.
Rusty Russell [Thu, 21 Jul 2011 05:14:50 +0000 (14:44 +0930)]
ccanlint: handle DOS-style \r\n lines when parsing.

We don't correctly detect pure-comment lines in ccan/ttxml/ttxml.c
without this.

9 years agotdb2: add full LGPL headers
Rusty Russell [Thu, 21 Jul 2011 05:14:50 +0000 (14:44 +0930)]
tdb2: add full LGPL headers

This is for SAMBA, so we follow their rules and do full license
headers.  Two files were missing them.

9 years agocontainer_of: relicense to Public domain
Rusty Russell [Thu, 21 Jul 2011 05:14:50 +0000 (14:44 +0930)]
container_of: relicense to Public domain

Too trivial to deserve LGPL, and all my code.

9 years agocheck_type: relicense to Public domain
Rusty Russell [Thu, 21 Jul 2011 05:14:49 +0000 (14:44 +0930)]
check_type: relicense to Public domain

Too trivial to deserve LGPL, and all my code.

9 years agohtable: relicense under LGPL
Rusty Russell [Thu, 21 Jul 2011 05:14:49 +0000 (14:44 +0930)]
htable: relicense under LGPL

Various LGPL components depend on it, via ccan/likely.  ccan/likely
really only needs it when CCAN_LIKELY_DEBUG is set, but making it a
conditional dependency is a bit nasty if defining that changes the
license.

So this is the simplest fix.  I might relicense under PD or BSD later,
since the likely module should probably have an even more liberal
license.

9 years agoccanlint: license_depends_compat checks dependencies are compatible.
Rusty Russell [Thu, 21 Jul 2011 05:14:46 +0000 (14:44 +0930)]
ccanlint: license_depends_compat checks dependencies are compatible.

We don't check external dependencies, but internal ccan deps are
pretty easy.

9 years agoccanlint: move license tag matching into common code.
Rusty Russell [Thu, 21 Jul 2011 04:59:06 +0000 (14:29 +0930)]
ccanlint: move license tag matching into common code.

Refactoring helps the next patch.

9 years agowwviaudio: fix license in _info, symlink (LGPL -> GPL)
Rusty Russell [Thu, 21 Jul 2011 04:59:06 +0000 (14:29 +0930)]
wwviaudio: fix license in _info, symlink (LGPL -> GPL)

Comments in code indicate this is actually GPL version 2 or later.

9 years agoogg_to_pcm: fix license in _info, symlink (LGPL -> GPLv2)
Rusty Russell [Thu, 21 Jul 2011 04:59:06 +0000 (14:29 +0930)]
ogg_to_pcm: fix license in _info, symlink (LGPL -> GPLv2)

Comments in code indicate this is actually GPL version 2 only.

9 years agomd4: fix license
Rusty Russell [Thu, 21 Jul 2011 04:59:06 +0000 (14:29 +0930)]
md4: fix license

As ccanlint now says:
  Source files don't contain incompatible licenses (license_file_compat): FAIL
  /home/rusty/devel/cvs/ccan/ccan/md4/md4.c:Found boilerplate for license 'GPLv2+' which is incompatible with 'LGPLv2+'

This is actually GPL code!

Add LICENSE link, too.

9 years agoccanlint: check for incompatible license boilerplates within subfiles.
Rusty Russell [Thu, 21 Jul 2011 04:59:03 +0000 (14:29 +0930)]
ccanlint: check for incompatible license boilerplates within subfiles.

This checks to make sure you're not accidentally relicensing code;
eg. it's OK (though a bit impolite) to turn a BSD-licensed file into a
GPL module, but not the other way around.

9 years agoccanlint: add simple check for comment referring to LICENSE file.
Rusty Russell [Thu, 21 Jul 2011 03:32:27 +0000 (13:02 +0930)]
ccanlint: add simple check for comment referring to LICENSE file.

After discussion with various developers (particularly the Samba
team), there's a consensus that a reference to the license in each
source file is useful.  Since CCAN modules are designed to be cut and
paste, this helps avoid any confusion should the LICENSE file go
missing.

We also detect standard boilerplates, in which case a one-line summary
isn't necessary.

9 years agonoerr: relicense to public domain.
Rusty Russell [Thu, 21 Jul 2011 03:32:04 +0000 (13:02 +0930)]
noerr: relicense to public domain.

We really want everyone to be using these; establishing conventions
helps all code, so make it the most liberal license possible.  It's
all my code, so I can do this unilaterally.

9 years agoalignof: relicense to public domain.
Rusty Russell [Thu, 21 Jul 2011 03:31:54 +0000 (13:01 +0930)]
alignof: relicense to public domain.

Trivial code, all mine.

9 years agobuild_assert: relicense to public domain.
Rusty Russell [Thu, 21 Jul 2011 03:31:49 +0000 (13:01 +0930)]
build_assert: relicense to public domain.

Trivial code, all mine.

9 years agoshort_types: relicense to public domain.
Rusty Russell [Thu, 21 Jul 2011 03:31:45 +0000 (13:01 +0930)]
short_types: relicense to public domain.

We really want everyone to be using these; establishing conventions
helps all code, so make it the most liberal license possible.  It's
all my code, so I can do this unilaterally.

9 years agocompiler: relicense to public domain.
Rusty Russell [Thu, 21 Jul 2011 03:31:39 +0000 (13:01 +0930)]
compiler: relicense to public domain.

We really want everyone to be using these; establishing conventions
helps all code, so make it the most liberal license possible.  It's
all my code, so I can do this unilaterally.

9 years agoccanlint: tighten license check.
Rusty Russell [Thu, 21 Jul 2011 02:26:15 +0000 (11:56 +0930)]
ccanlint: tighten license check.

Now we've made GPL wording uniform, use it everywhere.  There's no
point allowing variants which might be unclear.

We still have some non-conformant licenses in the tree (eg. just "BSD"),
so we only warn on unknown license strings for now.

9 years agovarious: make the _info License: wording uniform for GPL variants.
Rusty Russell [Tue, 19 Jul 2011 08:02:40 +0000 (17:32 +0930)]
various: make the _info License: wording uniform for GPL variants.

GPL versions 2 and 3 both specifically mention "any later version" as
the phrase which allows the user to choose to upgrade the license.
Make sure we use that phrase, and make the format consistent across
modules.

9 years agoccanlint: make a license enum, and parse the license string to set it.
Rusty Russell [Tue, 19 Jul 2011 08:00:49 +0000 (17:30 +0930)]
ccanlint: make a license enum, and parse the license string to set it.

This improves on the current ad-hoc methods, and also fixes a bug where
we mapped "GPLv2" to the GPLv3 symlink.

9 years agoMerge branch 'master' of ozlabs.org:ccan
Rusty Russell [Wed, 6 Jul 2011 05:21:54 +0000 (14:51 +0930)]
Merge branch 'master' of ozlabs.org:ccan

9 years agotally: don't use SIZE_MAX.
Rusty Russell [Wed, 6 Jul 2011 05:11:17 +0000 (14:41 +0930)]
tally: don't use SIZE_MAX.

Turns out it's not standard (thanks Samba build farm!)
And the previous test had a hole in it anyway.  This one is more conservative.

9 years agotap: WANT_PTHREAD not HAVE_PTHREAD
Rusty Russell [Mon, 4 Jul 2011 07:27:03 +0000 (16:57 +0930)]
tap: WANT_PTHREAD not HAVE_PTHREAD

I'm not sure that a "pthread-safe" tap library is very useful; how many
people have multiple threads calling ok()?

Kirill Shutemov noted that it gives a warning with -Wundef; indeed, we
should ask in this case whether they want pthread support, not whether the
system has pthread support to offer.

9 years agojson: Deleted the "notes" file.
Joey Adams [Sat, 2 Jul 2011 16:33:00 +0000 (12:33 -0400)]
json: Deleted the "notes" file.

This file contains my private ramblings about the JSON module,
and was not meant to be included in the public release.

9 years agojson: new module for parsing and generating JSON
Joey Adams [Thu, 30 Jun 2011 06:39:16 +0000 (02:39 -0400)]
json: new module for parsing and generating JSON

9 years agotally: fix FreeBSD compile, memleak in tests.
Rusty Russell [Tue, 21 Jun 2011 01:13:31 +0000 (10:43 +0930)]
tally: fix FreeBSD compile, memleak in tests.

Posix says ssize_t is in sys/types.h; on Linux stdlib.h is enough.

9 years agoantithread: patch to antithread arabella example
Russell Steicke [Fri, 17 Jun 2011 07:42:13 +0000 (15:42 +0800)]
antithread: patch to antithread arabella example

I've been using the antithread arabella example to generate some
"arty" portraits for decoration.  I've made a few changes to it
(triangle sizes and number of generations before giving up), and may
send those as patches later.

Because some of the images I'm generating have taken quite a while
(many days) I've needed to restart the run after rebooting machines
for other reasons, and noticed that arabella restarted the generation
count from zero.  I wanted to continue the generation count, so here's
a patch to do just that.

9 years agotdb2: Add tools/tdb2dump, tools/tdb2restore, use "tdb2.h" includes.
Rusty Russell [Fri, 17 Jun 2011 05:13:25 +0000 (14:43 +0930)]
tdb2: Add tools/tdb2dump, tools/tdb2restore, use "tdb2.h" includes.

Simple port from the TDB1 versions.  Also, change to "tdb2.h" includes
so they can be built even in other directories in future.

9 years agotdb2: rename the tools to tdb2torture, tdb2tool and mktdb2
Rusty Russell [Fri, 17 Jun 2011 05:11:55 +0000 (14:41 +0930)]
tdb2: rename the tools to tdb2torture, tdb2tool and mktdb2

This means they can be installed in parallel with tdb1's tools.

9 years agotdb2: use ccan/endian
Rusty Russell [Fri, 17 Jun 2011 02:57:44 +0000 (12:27 +0930)]
tdb2: use ccan/endian

This is where we should be getting bswap_64 from.

9 years agotools: trim leading whitespace in documentation extract.
Rusty Russell [Thu, 16 Jun 2011 03:03:23 +0000 (12:33 +0930)]
tools: trim leading whitespace in documentation extract.

Take some care to preserve formatting, even with mixed tabs and spaces.

9 years agocharset: Added utf8_validate_char (factored out of utf8_validate).
Joey Adams [Wed, 15 Jun 2011 02:13:01 +0000 (22:13 -0400)]
charset: Added utf8_validate_char (factored out of utf8_validate).

9 years agocharset: Rewrote utf8_validate, and added four new functions:
Joey Adams [Sat, 11 Jun 2011 07:58:10 +0000 (03:58 -0400)]
charset: Rewrote utf8_validate, and added four new functions:

 * utf8_read_char
 * utf8_write_char
 * from_surrogate_pair
 * to_surrogate_pair

9 years agohash: remove VALGRIND #ifdef - always run clean.
Rusty Russell [Wed, 8 Jun 2011 07:44:36 +0000 (17:14 +0930)]
hash: remove VALGRIND #ifdef - always run clean.

My simple test program on my laptop showed that with modern 32 bit Intel
CPUs and modern GCC, there's no measurable penalty for the clean version.

Andrew Bartlett complained that the valgrind noise was grating.  Agreed.

9 years agolbalance: add examples.
Rusty Russell [Sun, 5 Jun 2011 00:42:41 +0000 (10:12 +0930)]
lbalance: add examples.

9 years agolbalance: new module for load balancing
Rusty Russell [Tue, 31 May 2011 04:14:48 +0000 (13:44 +0930)]
lbalance: new module for load balancing

9 years agotime: new module for dealing with time.
Rusty Russell [Tue, 31 May 2011 04:14:36 +0000 (13:44 +0930)]
time: new module for dealing with time.

9 years agocast, container_of, tlist: Fix warning with GCC 4.6: -Wunused-but-set-variable
Joey Adams [Sun, 29 May 2011 02:23:55 +0000 (22:23 -0400)]
cast, container_of, tlist: Fix warning with GCC 4.6: -Wunused-but-set-variable

9 years agottxml: new module.
Rusty Russell [Fri, 20 May 2011 07:12:45 +0000 (16:42 +0930)]
ttxml: new module.

9 years agotdb2: fix O_RDONLY opens.
Rusty Russell [Fri, 20 May 2011 06:23:12 +0000 (15:53 +0930)]
tdb2: fix O_RDONLY opens.

We tried to get a F_WRLCK on the open lock; we shouldn't do that for a
read-only tdb.  (TDB1 gets away with it because a read-only open skips
all locking).

We also avoid leaking the fd in two tdb_open() failure paths revealed
by this extra testing.

9 years agofailtest: failtest_has_failed()
Rusty Russell [Fri, 20 May 2011 06:21:33 +0000 (15:51 +0930)]
failtest: failtest_has_failed()

Allows tests to explicitly avoid continuing when a failure has been
injected.

9 years agofailtest: override getpid() as well.
Rusty Russell [Fri, 20 May 2011 06:20:58 +0000 (15:50 +0930)]
failtest: override getpid() as well.

TDB2 tracks locks using getpid(), and gets upset when we fork behind
its back.

9 years agotypesafe_cb: don't use HAVE_CAST_TO_UNION in tests.
Rusty Russell [Tue, 10 May 2011 04:38:59 +0000 (14:08 +0930)]
typesafe_cb: don't use HAVE_CAST_TO_UNION in tests.

This crept in, it should be the same as the tests in typesafe_cb.h.

9 years agotdb2: more stats
Rusty Russell [Tue, 10 May 2011 00:33:50 +0000 (10:03 +0930)]
tdb2: more stats

More recording of interesting events.  As we don't have an ABI yet, we
don't need to put these at the end.

9 years agotdb2: check pid before unlock.
Rusty Russell [Tue, 10 May 2011 01:37:21 +0000 (11:07 +0930)]
tdb2: check pid before unlock.

The original code assumed that unlocking would fail if we didn't have a lock;
this isn't true (at least, on my machine).  So we have to always check the
pid before unlocking.

9 years agotdb2: fix msync() arg
Rusty Russell [Wed, 27 Apr 2011 13:30:25 +0000 (23:00 +0930)]
tdb2: fix msync() arg

PAGESIZE used to be defined to getpagesize(); we changed it to a
constant in b556ef1f, which broke the msync() call.

9 years agotdb2: use direct access functions when creating recovery blob
Rusty Russell [Wed, 27 Apr 2011 13:41:02 +0000 (23:11 +0930)]
tdb2: use direct access functions when creating recovery blob

We don't need to copy into a buffer to examine the old data: in the
common case, it's mmaped already.  It's made a bit trickier because
the tdb_access_read() function uses the current I/O methods, so we
need to restore that temporarily.

The difference was in the noise, however (the sync no-doubt
dominates).

Before:
$ time ./growtdb-bench 250000 10 > /dev/null && ls -l /tmp/growtdb.tdb && time ./tdbtorture -s 0 && ls -l torture.tdb && ./speed --transaction 2000000
real 0m45.021s
user 0m16.261s
sys 0m2.432s
-rw------- 1 rusty rusty 364469344 2011-04-27 22:55 /tmp/growtdb.tdb
testing with 3 processes, 5000 loops, seed=0
OK

real 1m10.144s
user 0m0.480s
sys 0m0.460s
-rw------- 1 rusty rusty 391992 2011-04-27 22:56 torture.tdb
Adding 2000000 records:  863 ns (110601144 bytes)
Finding 2000000 records:  565 ns (110601144 bytes)
Missing 2000000 records:  383 ns (110601144 bytes)
Traversing 2000000 records:  409 ns (110601144 bytes)
Deleting 2000000 records:  676 ns (225354680 bytes)
Re-adding 2000000 records:  784 ns (225354680 bytes)
Appending 2000000 records:  1191 ns (247890168 bytes)
Churning 2000000 records:  2166 ns (423133432 bytes)

After:
real 0m47.141s
user 0m16.073s
sys 0m2.460s
-rw------- 1 rusty rusty 364469344 2011-04-27 22:58 /tmp/growtdb.tdb
testing with 3 processes, 5000 loops, seed=0
OK

real 1m4.207s
user 0m0.416s
sys 0m0.504s
-rw------- 1 rusty rusty 313576 2011-04-27 22:59 torture.tdb
Adding 2000000 records:  874 ns (110601144 bytes)
Finding 2000000 records:  565 ns (110601144 bytes)
Missing 2000000 records:  393 ns (110601144 bytes)
Traversing 2000000 records:  404 ns (110601144 bytes)
Deleting 2000000 records:  684 ns (225354680 bytes)
Re-adding 2000000 records:  792 ns (225354680 bytes)
Appending 2000000 records:  1212 ns (247890168 bytes)
Churning 2000000 records:  2191 ns (423133432 bytes)