ccan
2 years agoshachain: remove unnecessary shachain_index_t
Rusty Russell [Tue, 15 Aug 2017 04:24:07 +0000 (13:54 +0930)]
shachain: remove unnecessary shachain_index_t

You can use SHACHAIN_BITS to contrain the size.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoshachain: add shachain_next_index()
Rusty Russell [Tue, 15 Aug 2017 04:18:19 +0000 (13:48 +0930)]
shachain: add shachain_next_index()

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoobjset: Use TCON_WRAP instead of TCON
David Gibson [Sun, 23 Jul 2017 05:21:36 +0000 (15:21 +1000)]
objset: Use TCON_WRAP instead of TCON

TCON() uses flexible-array members which aren't allowed in the middle
of structures, except as a gcc extension.  TCON_WRAP() avoids this and so
is more portable.

This doesn't change the objset interface, only its internals.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agojmap: Use TCON_WRAP instead of TCON
David Gibson [Sun, 23 Jul 2017 05:11:33 +0000 (15:11 +1000)]
jmap: Use TCON_WRAP instead of TCON

TCON() uses flexible-array members which aren't allowed in the middle
of structures, except as a gcc extension.  TCON_WRAP() avoids this and so
is more portable.

This doesn't change the jmap interface, only its internals.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agojset: Use TCON_WRAP instead of TCON
David Gibson [Sun, 23 Jul 2017 04:54:13 +0000 (14:54 +1000)]
jset: Use TCON_WRAP instead of TCON

TCON() uses flexible-array members which aren't allowed in the middle
of structures, except as a gcc extension.  TCON_WRAP() avoids this and so
is more portable.

This doesn't change the jset interface, only its internals.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agotlist: Use TCON_WRAP instead of TCON
David Gibson [Thu, 20 Jul 2017 14:06:01 +0000 (00:06 +1000)]
tlist: Use TCON_WRAP instead of TCON

TCON() uses flexible-array members which aren't allowed in the middle
of structures, except as a gcc extension.  TCON_WRAP() avoids this and so
is more portable.

This doesn't change the tlist interface, only its internals.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agoio: add io_fd_block() helper.
Rusty Russell [Tue, 27 Jun 2017 05:17:32 +0000 (14:47 +0930)]
io: add io_fd_block() helper.

It's a common thing to want to do, so add helper here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoio: allow overriding poll function.
Rusty Russell [Fri, 16 Jun 2017 03:47:32 +0000 (13:17 +0930)]
io: allow overriding poll function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoio: fix nasty io_wake corner case.
Rusty Russell [Wed, 31 May 2017 03:05:45 +0000 (12:35 +0930)]
io: fix nasty io_wake corner case.

If we're duplex and one io_always callback makes the other io_always,
we screwed up and hit an assertion later when the conn was in the
always list but didn't actually want to be.

io_wake() uses io_always(), so this is how it happened.  Writing a
test case for this was a bit fun, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoagar: Remove unnecessary NULL check
David Gibson [Tue, 4 Apr 2017 13:46:44 +0000 (23:46 +1000)]
agar: Remove unnecessary NULL check

At this point the construction of the function above means that nn cannot
be NULL.

Found by Coverity Scan.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agonet: Add check for failure of setsockopt()
David Gibson [Sun, 2 Apr 2017 15:15:53 +0000 (01:15 +1000)]
net: Add check for failure of setsockopt()

make_listen_fd() didn't check for failure of setsockopt().  There's no
real reason not to, since we have an obvious way to report an error to the
caller.

Found with Coverity Scan.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agoccanlint: Fix leak in do_reduce_features()
David Gibson [Sun, 2 Apr 2017 11:29:24 +0000 (21:29 +1000)]
ccanlint: Fix leak in do_reduce_features()

options_avail and options_used get freed, but options does not.

Found by Coverity scan.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agocrypto/ripemd160: Correct badly sized union member
David Gibson [Sun, 2 Apr 2017 11:21:02 +0000 (21:21 +1000)]
crypto/ripemd160: Correct badly sized union member

struct ripemd160_ctx has a union for converting between u8[] and u32[]
data.  Unfortunately the u32 array has a miscalculated size, half the size
of the u8 array.  That means some accesses which are within the union can
technically overrun the u32 array.

Found by Coverity scan.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agotools: Remove fd leak
David Gibson [Sun, 2 Apr 2017 09:17:12 +0000 (19:17 +1000)]
tools: Remove fd leak

compile_info() can leak an open file descriptor write_all() fails.  This
corrects it.

Found by Coverity.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agofailtest: Remove memory leak
David Gibson [Sun, 2 Apr 2017 09:06:06 +0000 (19:06 +1000)]
failtest: Remove memory leak

Somewhat ironically, a path in failtest related to detecting leaks in the
tested program itself leaks memory.  This corrects it.

Detected by Coverity.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years ago.gitignore: ignore .fast-ok files, too.
Rusty Russell [Mon, 3 Apr 2017 00:08:51 +0000 (09:38 +0930)]
.gitignore: ignore .fast-ok files, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoccan/ntdb: demote to junkcode.
Rusty Russell [Mon, 3 Apr 2017 00:08:26 +0000 (09:38 +0930)]
ccan/ntdb: demote to junkcode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoFix missing va_end()s
David Gibson [Fri, 31 Mar 2017 13:48:22 +0000 (00:48 +1100)]
Fix missing va_end()s

This corrects several places in ccan where stdarg.h is used but there is a
missing va_end().  You can get away with this on many platforms, but not
all.

Caught by Coverity scan.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agolbalance: Switch to tlist2
David Gibson [Fri, 31 Mar 2017 12:51:22 +0000 (23:51 +1100)]
lbalance: Switch to tlist2

lbalance uses the tlist module. tlist causes compile warnings on clang if
you're not careful, because it can put 0 length arrays in the middle of
structures.  tlist2 doesn't have the problem, and also has a slightly
cleaner interface.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agotools/ccanlint: Add missing header file
David Gibson [Fri, 31 Mar 2017 11:39:10 +0000 (22:39 +1100)]
tools/ccanlint: Add missing header file

tools/ccanlint/async.c uses kill(2), but doesn't include the signal.h
header it comes from.  One some platforms we get away with this via
indirect includes, but not on all.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agotools: Add asort module
David Gibson [Fri, 31 Mar 2017 11:25:07 +0000 (22:25 +1100)]
tools: Add asort module

tools/manifest.c uses asort(), but the asort module is not in
TOOLS_CCAN_MODULES.  That causes compile failures on some platforms, so
correct it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2 years agotake, tal, tal/path, tal/str, tal/talloc: annotate APIs with TAKES.
Rusty Russell [Wed, 15 Mar 2017 03:55:07 +0000 (14:25 +1030)]
take, tal, tal/path, tal/str, tal/talloc: annotate APIs with TAKES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agotake: add labels when CCAN_TAKE_DEBUG set, return in taken_any().
Rusty Russell [Wed, 15 Mar 2017 03:40:08 +0000 (14:10 +1030)]
take: add labels when CCAN_TAKE_DEBUG set, return in taken_any().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoMakefile: fix fastcheck.
Rusty Russell [Wed, 15 Mar 2017 02:54:02 +0000 (13:24 +1030)]
Makefile: fix fastcheck.

commit 25b7406d11ea3703be864097af66ce95611dde72 tried to make the tests
depend on the info file, but that broke .fast.ok, which used the same
pattern:

%.ok: $(LINT) %info

This is what happens when you're too tricky!  Simply duplicate the rule,
and change .fast.ok to .fast-ok so it doesn't match both.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoio/fdpass: fix example.
Rusty Russell [Wed, 15 Mar 2017 02:38:13 +0000 (13:08 +1030)]
io/fdpass: fix example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoio/fdpass: add flag to close fd after sending.
Rusty Russell [Tue, 14 Mar 2017 02:47:15 +0000 (13:17 +1030)]
io/fdpass: add flag to close fd after sending.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agoio: add io_flush_sync().
Rusty Russell [Tue, 14 Mar 2017 01:45:19 +0000 (12:15 +1030)]
io: add io_flush_sync().

This is needed for emergency handling in lightningd: we want to output
a (fatal) error packet on the socket, but we don't want to do so in the middle
of another packet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2 years agohex: fix comment
Martin Milata [Sun, 5 Feb 2017 13:29:45 +0000 (14:29 +0100)]
hex: fix comment

Signed-off-by: Martin Milata <martin@martinmilata.cz>
3 years ago.travis.yml: Add clang builds to trusty
David Gibson [Tue, 24 Jan 2017 09:45:53 +0000 (20:45 +1100)]
.travis.yml: Add clang builds to trusty

This enables clang compiler builds for the trusty Travis environment.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agocoroutine: Stack allocation
David Gibson [Fri, 20 Jan 2017 12:49:43 +0000 (23:49 +1100)]
coroutine: Stack allocation

At present, coroutine stacks must be allocated explicitly by the user,
then initialized with coroutine_stack_init().  This adds a new
coroutine_stack_alloc() function which allocates a stack, making life
easier for users.  coroutine_stack_release() will automatically determine
if the given stack was set up with _init() or alloc() and act
accordingly.

The stacks are allocate with mmap() rather than a plain malloc(), and a
guard page is added, so an overflow of the stack should result in a
relatively debuggable SEGV instead of random data corruption.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agocoroutine: Enable valgrind
David Gibson [Sat, 24 Dec 2016 10:08:55 +0000 (21:08 +1100)]
coroutine: Enable valgrind

Currently valgrind checks are disabled on the coroutine module,
because switching stacks tends to confuse it.  We can work around this
by using the valgrind client interface to explicitly inform it about
the stacks we create.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agocoroutine: Remove on-stack buffers from testcases
David Gibson [Sat, 24 Dec 2016 10:40:00 +0000 (21:40 +1100)]
coroutine: Remove on-stack buffers from testcases

In preparation for enabling valgrind tests, remove instances where we
allocate a coroutine's stack from a buffer itself on the stack.  Not all
that surprisingly, valgrind gets very, very confused by having one
"thread"'s stack embedded within another's.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agocoroutine: Move total initialization outside coroutine
David Gibson [Tue, 24 Jan 2017 09:39:45 +0000 (20:39 +1100)]
coroutine: Move total initialization outside coroutine

The sample coroutine in api-3 initializes a total to 0, then adds up the
pseudo-random data it has placed into a stack buffer, to ensure that the
compiler won't elide the reading and writing of that buffer.  After the
coroutine has completed, we verify that total is non-zero so that we'll
detect if the coroutine failed to execute entirely.

Except that the initialization of total is within the coroutine itself,
so it could also be non-zero due to it simply being uninitialized.  This
moves the initialization outside the coroutine, to make the test a little
more robust.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agocoroutine: Remove problematic diagnostic from api-3 test
David Gibson [Tue, 24 Jan 2017 06:59:16 +0000 (17:59 +1100)]
coroutine: Remove problematic diagnostic from api-3 test

The api-3 testcase devotes most of its available stack space to a test
buffer, leaving only a small amount (COROUTINE_MIN_STKSZ) for the actual
stack usage of the coroutine.

It turns out that the ccan/tap diag() function can - depending on compiler
version and flags, and on whether diagnostics are enabled - exceed that
limited stack space.  That leads to a stack overrun, and in turn corruption
of the parent routine's stack, generating unpredictable and hard to debug
SEGVs.

At present, this bug seems to be tripped by clang-3.8 when diagnostic
messages are printed.

This removes the troublesome diag() call.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agotal: make tal_len/tal_count(NULL) return 0.
Rusty Russell [Tue, 24 Jan 2017 06:22:00 +0000 (16:52 +1030)]
tal: make tal_len/tal_count(NULL) return 0.

Previously it crashed, but if you're always dealing with tal arrays,
this is painful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years ago.travis.yml: Add compilation with gcc 5 and gcc 6
David Gibson [Thu, 19 Jan 2017 01:58:24 +0000 (12:58 +1100)]
.travis.yml: Add compilation with gcc 5 and gcc 6

Now that we have a way to correctly set a matching coverage tool, we can
add more recent compiler versions to the Travis build.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoccanlint: Correct default coverage tool for clang
David Gibson [Wed, 18 Jan 2017 21:56:48 +0000 (08:56 +1100)]
ccanlint: Correct default coverage tool for clang

Currently ccanlint defaults to using "gcov" as the coverage analysis tool
for any compiler defining __GNUC__.  That's generally correct for the
(system default) gcc.  However, clang also defines __GNUC__ because it
implements the GCC langauge extensions.  For clang, "gcov" is not the
correct coverage tool (clang does use roughly the gcov format, but unless
you're very lucky the system gcc and system clang won't use the same gcov
versions).

This changes the default coverage tool in the case of clang to the correct
"llvm-cov gcov".

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoccanlint: Allow path to gcov to be overriden
David Gibson [Wed, 18 Jan 2017 03:23:51 +0000 (14:23 +1100)]
ccanlint: Allow path to gcov to be overriden

Currently ccanlint always assumes that the coverage tool can be
invoked under the command "gcov".

However, the coverage tool generally needs to be closely matched to
the compiler version.  So, the current behaviour won't work with
compilers other than gcc, like clang.  It won't even work for a gcc
version which isn't the standard system one matching gcov.

To address this, allow the command for the coverage tool to be
overridden on the ccanlint command line with a new --gcov option.  We
also allow it to be overridden for make check with a GCOV make
variable.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agotools: Consolidate gcov handling
David Gibson [Wed, 18 Jan 2017 03:09:29 +0000 (14:09 +1100)]
tools: Consolidate gcov handling

At the moment, invocation of the 'gcov' tool for coverage analysis
from ccanlint is put directly into the tests_compile_coverage.c and
tests_coverage.c files.  This makes it awkard to extend.

So, this patch moves the invocation of gcov into a new tools/gcov.v
file, analagous to tools/compile.c.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years ago.travis.yml: Add valgrind testing
David Gibson [Mon, 16 Jan 2017 23:37:15 +0000 (10:37 +1100)]
.travis.yml: Add valgrind testing

Currently, our Travis builds don't have valgrind installed, meaning
that ccanlint's valgrind based tests will be skipped, which is
unfortunate.

This adds valgrind to some of the builds to give us better CI
coverage.  It's not added for Precise with gcc, because that causes
failures which appear to be due to something in the builtins of that
gcc version.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years ago.travis.yml: Add builds under Ubuntu Trusty
David Gibson [Mon, 16 Jan 2017 05:10:47 +0000 (16:10 +1100)]
.travis.yml: Add builds under Ubuntu Trusty

At the moment our Travis builds all use Travis's default Ubuntu
Precise base distro.  For wider testing, add a build using their
Ubuntu Trusty distro.  Only build with gcc there, for now, since clang
will cause ccanlint failures, due to the gcov version there not being
suitable for clang output.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years ago.travis.yml: Rework Travis matrix
David Gibson [Mon, 16 Jan 2017 04:39:48 +0000 (15:39 +1100)]
.travis.yml: Rework Travis matrix

At the moment the .travis.yml implicitly constructs a build matrix
with the two compiler options.  In future we want to add more build
options for wider testing: different base distro, more compiler
versions, etc.  However, a fair few of the possible combinations have
various problems meaning we don't want to test them routinely.

So, this reworks from implicitly constructing the matrix to using
matrix: include: options to explicitly build the options we want.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agointmap: clean up iterators.
Rusty Russell [Fri, 13 Jan 2017 02:55:14 +0000 (13:25 +1030)]
intmap: clean up iterators.

By returning the value, we have a nice sentinal and we save a second lookup
if they want it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agointmap: new module.
Rusty Russell [Wed, 11 Jan 2017 03:25:54 +0000 (13:55 +1030)]
intmap: new module.

Critbit tree to map ints/uints to pointers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoio: make io_close_taken_fd() unset nonblocking on the fd.
Rusty Russell [Mon, 9 Jan 2017 23:16:26 +0000 (09:46 +1030)]
io: make io_close_taken_fd() unset nonblocking on the fd.

This is what users want, and expect: as demonstrated by the test failure
when not under valgrind!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoio: handle errors on listening file descriptors.
Rusty Russell [Mon, 9 Jan 2017 02:46:32 +0000 (13:16 +1030)]
io: handle errors on listening file descriptors.

While investigating the previous patch, a bug caused poll to return
POLLHUP on the listening socket, which caused us to spin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoio: io_close_taken_fd to steal fd from conn.
Rusty Russell [Mon, 9 Jan 2017 02:08:06 +0000 (12:38 +1030)]
io: io_close_taken_fd to steal fd from conn.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoio: don't try to close() connection twice, remove shutdown logic.
Rusty Russell [Mon, 9 Jan 2017 02:03:44 +0000 (12:33 +1030)]
io: don't try to close() connection twice, remove shutdown logic.

We were closing before calling del_fd, which also closed.

The shutdown() logic applies when a child and parent are using the
*same* socket fd to communicate to each other.  That's really unusual
(who would you connect to?), and should probably be done by the user.

Generally, you'd use socketpair() for this child-parent case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoio: update documentation for io_close.
Rusty Russell [Mon, 9 Jan 2017 00:15:31 +0000 (10:45 +1030)]
io: update documentation for io_close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agostr: make sure STR_MAX_CHARS gives a constant expression.
Rusty Russell [Sat, 7 Jan 2017 04:19:53 +0000 (14:49 +1030)]
str: make sure STR_MAX_CHARS gives a constant expression.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agotal/autoptr: new module.
Rusty Russell [Thu, 29 Dec 2016 04:34:30 +0000 (15:04 +1030)]
tal/autoptr: new module.

Helps with the common case of wanting to NULL out a pointer when the object
freed.  We could also track it if resized, but that's TODO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agotal: support destructors with an extra argument.
Rusty Russell [Thu, 29 Dec 2016 04:33:19 +0000 (15:03 +1030)]
tal: support destructors with an extra argument.

There are several times I've wanted an extra arg to the destructor, and had
to embed it in the thing destroyed.  It's more efficient to put it into
tal itself (since it allocates space anyway), but we make it conditional
on a flag to avoid bloating every destructor.

The infrastructure makes it easier to add an extra arg to the general
notifiers later if we want.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoio: allow freeing of io_conn at any time.
Rusty Russell [Thu, 29 Dec 2016 04:31:32 +0000 (15:01 +1030)]
io: allow freeing of io_conn at any time.

io_close() currently marks the io_conn for freeing, but doesn't
actually do it.  This is a problem for tal() users, because we can't
just call it in the parent's constructor.

Make io_close() just tal_free() + return &io_conn_freed (a magic
io_plan pointer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoio: remove io_debug support.
Rusty Russell [Thu, 29 Dec 2016 04:29:29 +0000 (14:59 +1030)]
io: remove io_debug support.

It seemed like a good idea, but it complicates things and I never used
it (since I never really trusted that the alternate paths would be
equivalent).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agotal: save and restore errno across all notifiers.
Rusty Russell [Thu, 29 Dec 2016 04:28:42 +0000 (14:58 +1030)]
tal: save and restore errno across all notifiers.

So the errno when you call tal_free() is handed to all the notifiers,
independent of what the others do.

This makes sense, but also helps for the upcoming ccan/io change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoccanlint: Move ccanlint test options from _info comments to code
David Gibson [Sat, 24 Dec 2016 12:58:19 +0000 (23:58 +1100)]
ccanlint: Move ccanlint test options from _info comments to code

Currently, _info files can specify options, or note expected failures, for
ccanlint checks in the _info file with specially structured comments. That
differs from most other things ccanlint gets from _info, where it instead
executes the info file with certain parameters.

This changes ccanlint and existing _info files to use the normal method for
the ccanlint test options as well.  This also has the advantage that an
info file can alter its test options based on things from config.h - in
some cases whether a test can work or not might depend on various things.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoMakefile: Make module checks depend on info file
David Gibson [Sat, 24 Dec 2016 12:46:29 +0000 (23:46 +1100)]
Makefile: Make module checks depend on info file

Changing the _info file can change how ccanlint assesses the module.
Therefore, if the _info file changes, we should re-run ccanlint module
tests with make check.  We didn't previously have a dependency for that,
though, so this adds it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoio/fdpass: new module for async fd passing.
Rusty Russell [Fri, 23 Dec 2016 00:40:57 +0000 (11:10 +1030)]
io/fdpass: new module for async fd passing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agobase64: correct linkage of base64_maps_rfc4648
Peter Barker [Thu, 18 Feb 2016 05:38:54 +0000 (16:38 +1100)]
base64: correct linkage of base64_maps_rfc4648

Two copies of this map meant only the tests would ever work.

3 years agogenerator: Rewrite to use coroutine module
David Gibson [Mon, 12 Dec 2016 09:58:35 +0000 (20:58 +1100)]
generator: Rewrite to use coroutine module

Use the new coroutine module to abstract away our dependence on
ucontext.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agocoroutine: New module
David Gibson [Wed, 6 Jul 2016 11:31:03 +0000 (21:31 +1000)]
coroutine: New module

This is essentially a wrapper around ucontext.h, but the idea is that
alternative back end implementations could be used in future.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agofdpass: new module.
Rusty Russell [Wed, 7 Dec 2016 04:55:10 +0000 (15:25 +1030)]
fdpass: new module.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoshachain: allow overriding of number of bits, add cmdline tool.
Rusty Russell [Mon, 5 Dec 2016 05:48:20 +0000 (16:18 +1030)]
shachain: allow overriding of number of bits, add cmdline tool.

lightning uses 48 bits, so this provides a quick utility for our test
vectors, and provides a nice example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agohkdf_sha256: okm is a void *.
Rusty Russell [Mon, 5 Dec 2016 05:25:08 +0000 (15:55 +1030)]
hkdf_sha256: okm is a void *.

Making arbitrary bytes a uchar* is always a bit nasty for the caller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoMakefile: Remove testdepends from make check dependencies
David Gibson [Thu, 1 Dec 2016 11:43:20 +0000 (22:43 +1100)]
Makefile: Remove testdepends from make check dependencies

The new Makefile system, via the helper script in tools/gen_deps.sh, when
generating the targets to test a module, inserts dependencies meaning it
must first check modules this one depends on, whether via 'depends' or
'testdepends' in _info.

Although it seems logical, including 'testdepends' is actually incorrect.
If ccan/a testepends on ccan/b then ccan/b must be *built* in order to test
ccan/a, but it doesn't need to be tested.  testepends are explicitly
permitted to contain loops - it's quite common for two complementary
modules to be used to test each other.  This is one of the reasons
testdepends exists separate from depends.

So, remove testdepends from the generated check dependencies, removing the
circular dependency that Make complains about.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agocrypto/hkdf_sha256: new module.
Rusty Russell [Wed, 30 Nov 2016 03:54:37 +0000 (14:24 +1030)]
crypto/hkdf_sha256: new module.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agocrypto/hmac_sha256: add partial progress functions.
Rusty Russell [Wed, 30 Nov 2016 03:54:23 +0000 (14:24 +1030)]
crypto/hmac_sha256: add partial progress functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agocrypto/hmac_sha256: new module.
Rusty Russell [Wed, 30 Nov 2016 03:54:08 +0000 (14:24 +1030)]
crypto/hmac_sha256: new module.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoedit_distance: Rename ED_STACK_ELEMS ED_STACK_DIST_VALS
Kevin Locke [Sun, 27 Nov 2016 00:44:01 +0000 (17:44 -0700)]
edit_distance: Rename ED_STACK_ELEMS ED_STACK_DIST_VALS

The previous name was misleading, since it does not define the number of
elements (ed_elem) on the stack, but rather the number of distance
values (ed_dist).  Rename to make this more clear and add more
documentation about what it does and how best to define it.

Note:  This is an API change for custom-compiled versions, but since the
module has only been included for a couple days I don't think it's worth
a back-compat #ifdef at this point.

Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
3 years ago.travis.yml: Enable C99 mode for older gcc versions
David Gibson [Wed, 23 Nov 2016 12:12:59 +0000 (23:12 +1100)]
.travis.yml: Enable C99 mode for older gcc versions

Currently Travis builds fail on the edit_distance module, because that
module uses C99 extensions, which aren't enabled by default on the
Ubuntu Precise compiler version.  Force it to allow these, by adding the
-std=gnu99 option to the compiler.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoedit_distance: calculate edit distance between strings
Kevin Locke [Sat, 19 Nov 2016 04:19:25 +0000 (21:19 -0700)]
edit_distance: calculate edit distance between strings

This commit creates the edit_distance module for calculating various
edit distance measures.  The edit distance quantifies the similarity
between two strings based on the number of modifications necessary to
turn one string into the other.  There are several edit distance
measures which differ in the operations which are permitted and the
cost (aka weight) of the operations.  This module provides functions
for calculating the Longest Common Subsequence (LCS), Levenshtein, and
Damerau-Levenshtein (restricted and unrestricted) distances.  Weighted
versions of these functions can be created by defining cost functions
as preprocessor macros when compiling this module.  Distances over
other array types (e.g. wide strings, integers, structs) can be
accomplished by defining the element type and equality test macros.

Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoMakefile: Remove unused test targets
David Gibson [Tue, 22 Nov 2016 22:50:48 +0000 (09:50 +1100)]
Makefile: Remove unused test targets

The Makefile provides for "fast" and "full" variants of make check.
However, nothing seems to use or implement the "full" variant at all.
Further, the %.check and %.fastcheck targets are not actually used, with
other targets instead using the actual .ok tag file as the target.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoweb: fix for PHP7.
Rusty Russell [Tue, 22 Nov 2016 00:47:09 +0000 (11:17 +1030)]
web: fix for PHP7.

split() was deprecated, is now removed. explode() is the new hotness.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoMakefile-web: make it usable with "make -f Makefile-web".
Rusty Russell [Tue, 22 Nov 2016 00:20:46 +0000 (10:50 +1030)]
Makefile-web: make it usable with "make -f Makefile-web".

Particularly since it's used by a gitolite hook on ozlabs.org to update
the webpages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agocppmagic: Tweak testcase to work on older clang versions
David Gibson [Mon, 21 Nov 2016 14:05:37 +0000 (01:05 +1100)]
cppmagic: Tweak testcase to work on older clang versions

clang version 3.4, for whatever reason, expands a few of the testcases
in cppmagic with slightly different whitespace from what we expect.  Since
this was changed in later clang versions, it was probably a compiler bug,
but it turns out an extra space in the testcase can make it pass on clang
3.4 as well.  This means we finally have clang builds working in Travis.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agotime: Change TIME_HAVE_MONOTONIC to be 0/1 rather than defined/undefined
David Gibson [Mon, 21 Nov 2016 13:52:53 +0000 (00:52 +1100)]
time: Change TIME_HAVE_MONOTONIC to be 0/1 rather than defined/undefined

Generally, ccan config variables are always defined as either 0 or 1 and
are tested with #if.  That's instead of being being either defined or
undefined and tested with #ifdef.

TIME_HAVE_MONOTONIC breaks that convention.  This can cause warnings in
ccan/timer which uses it assuming the 0/1 convention.  Change it to remove
that warning.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoccanlint: Remove unused variable
David Gibson [Mon, 21 Nov 2016 13:47:37 +0000 (00:47 +1100)]
ccanlint: Remove unused variable

Remove a warning caused by the unused 'explain' variable in
headers_idempotent.c.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoMakefile: Remove generator modules from the exclude list
David Gibson [Mon, 21 Nov 2016 12:45:50 +0000 (23:45 +1100)]
Makefile: Remove generator modules from the exclude list

There's no real reason to exclude 'generator' from the build or check
targets.  Add it back in.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years ago.travis.yml: Remove external libraries
David Gibson [Mon, 21 Nov 2016 12:35:22 +0000 (23:35 +1100)]
.travis.yml: Remove external libraries

The new makefile excludes all modules with external library dependencies
from the usual make check, meaning the libraries are not needed for the
Travis build.

Although it would be nice to include these modules, there are a bunch of
problems with sorting out a correct environment for them.  For now, just
leave them out.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoMerge Makefile rewrite into master
David Gibson [Mon, 21 Nov 2016 12:24:36 +0000 (23:24 +1100)]
Merge Makefile rewrite into master

# Conflicts:
# .travis.yml
# Makefile
# Makefile-ccan

3 years agowwviaudio: Mark examples as not compiling
David Gibson [Thu, 17 Nov 2016 12:24:31 +0000 (23:24 +1100)]
wwviaudio: Mark examples as not compiling

The example code in wwviaudio doesn't compile as is.  Mark it as an
expected failure for ccanlint.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agowwviaudio: Use correct C99 format specifiers for uint64_t
David Gibson [Thu, 17 Nov 2016 12:23:34 +0000 (23:23 +1100)]
wwviaudio: Use correct C99 format specifiers for uint64_t

This fixes a warning.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoClearer output from make check in Travis
David Gibson [Wed, 2 Nov 2016 22:35:31 +0000 (09:35 +1100)]
Clearer output from make check in Travis

This removes the parallel make (-j) option when running make check under
Travis CI.  This will slow down the build somewhat, but makes it easier to
match errors to the modules which caused them.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoccanlint: Alter meaning of --summary option
David Gibson [Wed, 2 Nov 2016 22:34:23 +0000 (09:34 +1100)]
ccanlint: Alter meaning of --summary option

--summary (used by make check) causes ccanlint to only ever print a
one-line summary of the results.  When one of the tests fails, this isn't
very useful, because it doesn't say why it failed, or even which test
failed.  This is particularly frustrating for failures in Travis CI builds
where it's not always straightforward to reproduce a similar enough build
environment to check the failure manually for a particular module.

--summary does have one purpose which is vital for make check, though: it
suppresses any attempt to interactively correct / improve the module
printing only results.

This changes --summary to only have the second effect, not the first.  This
won't change the output in the case of passing modules, but will give more
useful information in the case of failing modules.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agopipecmd: fix -Wextra complaint about empty body.
Rusty Russell [Sun, 6 Nov 2016 06:44:53 +0000 (17:14 +1030)]
pipecmd: fix -Wextra complaint about empty body.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoccanlint: remove empty statement warnings.
Rusty Russell [Sun, 6 Nov 2016 06:43:53 +0000 (17:13 +1030)]
ccanlint: remove empty statement warnings.

Since doesnt_matter() expands to nothing, GCC -Wextra suggests { }.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoccanlint: check unused result.
Rusty Russell [Sun, 6 Nov 2016 06:42:53 +0000 (17:12 +1030)]
ccanlint: check unused result.

GCC with -O gives an error here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoccanlint: mark unused parameters.
Rusty Russell [Sun, 6 Nov 2016 06:41:53 +0000 (17:11 +1030)]
ccanlint: mark unused parameters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agohtable: another unused parameter in tests.
Rusty Russell [Sun, 6 Nov 2016 06:40:53 +0000 (17:10 +1030)]
htable: another unused parameter in tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoccanlint: fix usage of unused main parameters in other cases where we add main()
Rusty Russell [Sun, 6 Nov 2016 06:33:19 +0000 (17:03 +1030)]
ccanlint: fix usage of unused main parameters in other cases where we add main()

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agocontainer_of: Make example compile cleanly with -Wunused
Rusty Russell [Sun, 6 Nov 2016 04:38:17 +0000 (15:08 +1030)]
container_of: Make example compile cleanly with -Wunused

Make register_timer "use" my_timer_callback and its own args.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoasort, cdump, htable, list, noerr, strmap, tal/link: fix sign warnings in examples.
Rusty Russell [Sun, 6 Nov 2016 04:37:51 +0000 (15:07 +1030)]
asort, cdump, htable, list, noerr, strmap, tal/link: fix sign warnings in examples.

Many are because argc is 'int' not 'unsigned', others just laziness.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agotools/ccanlint: add args to main.
Rusty Russell [Sun, 6 Nov 2016 04:35:17 +0000 (15:05 +1030)]
tools/ccanlint: add args to main.

This allows examples to refer to them, but we cast them to void at the
end to avoid -Wunused warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoMark unused arguments in many modules.
Rusty Russell [Sun, 6 Nov 2016 04:34:17 +0000 (15:04 +1030)]
Mark unused arguments in many modules.

Either with UNNEEDED (if the module already used ccan/compiler) or
with (void) casting.

The only other change is in ccan/list/test/run-CCAN_LIST_DEBUG.c, because
the linenumbers change and thus it needs updating.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agocast, str, take, tal/grabfile, tal/str, typesafe_cb: use argc
Rusty Russell [Fri, 4 Nov 2016 20:56:47 +0000 (07:26 +1030)]
cast, str, take, tal/grabfile, tal/str, typesafe_cb: use argc

This avoids the warning about it being unused with -Wunused.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoRemove unused main() args in many modules.
Rusty Russell [Fri, 4 Nov 2016 20:55:47 +0000 (07:25 +1030)]
Remove unused main() args in many modules.

This makes us closer to compiling cleanly with -Wunused (as used by -Wextra).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agotal: remove ccan/talloc from example.
Rusty Russell [Fri, 4 Nov 2016 20:54:47 +0000 (07:24 +1030)]
tal: remove ccan/talloc from example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agotools/ccanlint: compile cleanly with -Wextra.
Rusty Russell [Fri, 4 Nov 2016 20:53:47 +0000 (07:23 +1030)]
tools/ccanlint: compile cleanly with -Wextra.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agocpuid: increment over size of cpu array, not just first one.
Rusty Russell [Fri, 4 Nov 2016 10:44:21 +0000 (21:14 +1030)]
cpuid: increment over size of cpu array, not just first one.

Found by PCS Suite.

Reported-by: Jon Griffiths
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Ahmed Samy <f.fallen45@gmail.com>
3 years agoaga,agar: Negative weight cycle testcase
David Gibson [Thu, 3 Nov 2016 10:49:55 +0000 (21:49 +1100)]
aga,agar: Negative weight cycle testcase

Adds a new test graph which includes a negative weight cycle.  This means
that shortest paths are not well defined, and both Dijkstra's algorithm and
the Bellman-Ford algorithm (which can handle some negative edge weights)
will fail.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>