Rusty Russell [Fri, 12 Oct 2018 00:25:31 +0000 (10:55 +1030)]
rbuf, tools: clean up rbuf usage.
We should use rbuf primitives not reach inside to membuf.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 12 Oct 2018 00:23:35 +0000 (10:53 +1030)]
membuf: add membuf_added and membuf_unadd APIs.
Clean up some whitespace while we're there too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 21 Sep 2018 07:31:09 +0000 (17:01 +0930)]
crypto/shachain/tools: update to new rbuf API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 26 Sep 2018 23:39:50 +0000 (09:09 +0930)]
structeq: fix case where we mark padding as unknown.
And change semantics: a negative number means "up to this much padding".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 26 Sep 2018 23:57:40 +0000 (09:27 +0930)]
tools: fix compile after rbuf rewrite.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 17 Sep 2018 01:24:45 +0000 (10:54 +0930)]
rbuf: adapt to work on ccan/membuf.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 17 Sep 2018 01:17:42 +0000 (10:47 +0930)]
membuf: new module for linear memory buffers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Thu, 23 Aug 2018 02:41:13 +0000 (12:11 +0930)]
tal: allow notifiers on NULL.
A destructor on NULL doesn't make sense, but notifiers (eg. new children)
do. We fix up a mistake in run-notifier (comparing ctx with itself) and
loose typing in tal.c's tal_add_notifier_ too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Kazuhiro Sera [Fri, 10 Aug 2018 06:54:10 +0000 (15:54 +0900)]
Fix typos detected by github.com/client9/misspell
Rusty Russell [Fri, 27 Jul 2018 06:25:50 +0000 (15:55 +0930)]
tal/str: always create strings which have tal_count() == strlen() + 1.
This is least-surprise, but also means callers can sometimes do faster
string handling by avoiding strstr().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 27 Jul 2018 04:29:18 +0000 (13:59 +0930)]
ccan/tal: always include a length field.
The current semantics of tal_count() / tal_bytelen() are to return 0
for anything not allocated using tal_arr*. This is because we tried
to save a native-length word in the header, but produces an awkward
API.
(To make it worse, defining CCAN_TAL_DEBUG turns length to always on,
and we enable that for c-lightning developer mode, which hides bugs!).
However, for c-lightning, just over half of allocations want a length:
these use 3 words each, so we're actually worse off overall.
The answer is to always have a length field in the header. This also
simplfies the tal code.
samba-allocs stats before:
Tal time:
1237102-
1305755(1.251e+06+/-2.1e+04)ns
Tal_free time:
1346871-
1514514(1.37844e+06+/-5.2e+04)ns
After:
Tal time:
1115180-
1180633(1.1351e+06+/-2.1e+04)ns
Tal_free time:
1334381-
1465933(1.39148e+06+/-4.7e+04)ns
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 27 Jul 2018 04:23:19 +0000 (13:53 +0930)]
tal/str and tal/stack: use _label interfaces.
In particular, tal/str now passes through the label from the caller,
so (in case of CCAN_TAL_DEBUG) you can actually see the file and line
where the caller was, not just inside ccan/str.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 27 Jul 2018 04:22:20 +0000 (13:52 +0930)]
tal: add _label interfaces.
There are a number of other utilities which use the tal_alloc_ and
tal_dup_arr_ internal interfaces directly, because they want to set
the label themselves. We're about to break them all by changing those
internal interfaces, so give them a mid-level interface to use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 27 Jul 2018 04:20:42 +0000 (13:50 +0930)]
tal: rename tal_len to tal_bytelen.
I had a bug caused by using tal_len instead of tal_count: let's make
it explicit. @jb55 commented "ha. I always forget which one does which... Ack"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 4 Jul 2018 04:07:28 +0000 (13:37 +0930)]
ccan/structeq: make it safe when there's padding.
ccan/cppmagic FTW!
The only issue is that we can't tell if there's padding or they've missed
a member, so we add a padding bytes count, so they'll get an error if it
(for example) the structure adds a new member later.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 18 Jun 2018 10:42:46 +0000 (20:12 +0930)]
ccan/utf8: new module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:37:51 +0000 (12:07 +0930)]
tools/configurator: add manual page.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:36:51 +0000 (12:06 +0930)]
tools/configurator: --extra-tests to read more test descriptions from stdin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:35:51 +0000 (12:05 +0930)]
tools/configurator: don't rely on size of tests_array.
Groundwork for adding tests dynamically.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:34:51 +0000 (12:04 +0930)]
tools/configurator: better descriptions for --autotools-style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:33:51 +0000 (12:03 +0930)]
tools/configurator: Better argument handling, particularly unknown arguments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:32:51 +0000 (12:02 +0930)]
tools/configurator: support --header-file if we don't want to write to stdout.
Works well with --autotools-style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:31:51 +0000 (12:01 +0930)]
tools/configurator: support --autotools-style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:30:51 +0000 (12:00 +0930)]
tools/configurator: support --var-file for outputting VAR=VAL format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 12 Jun 2018 02:29:51 +0000 (11:59 +0930)]
tools/configurator: convert flags to text in 'struct test'
This is the only non-text field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Eric Wong [Fri, 8 Jun 2018 02:45:42 +0000 (02:45 +0000)]
timer: remove unnecessary dependency on likely
I don't see likely/unlikely being used by ccan/timer or any of its
dependencies right now.
Running `tools/ccanlint/ccanlint ccan/timer' reveals no regressions
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rusty Russell [Thu, 10 May 2018 02:25:59 +0000 (11:55 +0930)]
endian: fix compilation with musl libc.
It defines __BYTE_ORDER to __BYTE_ORDER__; gcc complains when we define
it to something else. Let it be already defined, but check that the
value is what we expect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 9 May 2018 04:13:17 +0000 (13:43 +0930)]
Makefile-web: any PHP version will do.
php5 was finally removed from ozlabs.org, but php7.2 seems to work fine.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 9 May 2018 03:09:47 +0000 (12:39 +0930)]
opt: print usage correctly for early args.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Kamil [Fri, 6 Apr 2018 23:00:42 +0000 (01:00 +0200)]
Fix HAVE_ASPRINTF detection
gcc6 introduceed a new warning switched with -Wmisleading-identation. This caused to generate a compilation warning for if statement misleadingly indented, which caused HAVE_ASPRINTF to be defined as 0. Adding newline after an if statement fixes the problem.
Rusty Russell [Fri, 6 Apr 2018 07:29:39 +0000 (16:59 +0930)]
tal/grab_file: be robust against EINTR.
Exracted (and slightly modified) from a MacOS PR for lightning.
Based-on-patch-by: https://github.com/conanoc
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 6 Apr 2018 07:26:26 +0000 (16:56 +0930)]
ptr_valid: fix spurious SIGINT under lldb on MacOS
(Taken from PR for lightning)
Patch-from: https://github.com/conanoc
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Thu, 5 Apr 2018 02:31:51 +0000 (12:01 +0930)]
base32: add ability to substitute character set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Thu, 5 Apr 2018 02:23:11 +0000 (11:53 +0930)]
str/base32: new module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Mar 2018 10:42:39 +0000 (21:12 +1030)]
intmap: add iterator-by-callback.
It's significantly faster because it assumes no deletion:
10000000,critbit iteration (nsec),316
10000000,critbit callback iteration (nsec),90
...
10000000,critbit consecutive iteration (nsec),308
10000000,critbit consecutive callback iteration (nsec),78
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Mar 2018 10:38:34 +0000 (21:08 +1030)]
intmap: add exhaustive testcases for intmap_after
We can't do the full range, but we can for a handful of bits (8).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Mar 2018 10:37:34 +0000 (21:07 +1030)]
intmap: add test case which failed, extracted from real world usage.
Because intmap_after_() would simply examine the critbits to walk the
tree, it wouldn't realize that it might be in the completely wrong tree.
In this case:
Bit 4:
0 1
/ \
/ \
100000011 100001011
When we ask for intmap_after_(
011111111) we would check the critbit, it's
a 1, so we end up on the right leaf instead of the left.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Mar 2018 10:36:34 +0000 (21:06 +1030)]
intmap: reimplement so that intmap_after works.
A critbit tree is a binary tree which keeps branches for each bit
which differs in the leaves. It's a simple data structure, but not
entirely simple to implement the primitives people expect, as this bus
shows.
The bug: I added an open iterator, and intmap_after_ for a random
value would sometimes return the wrong node.
Cause: we don't know what the prefix is as we iterate, so by only
testing the critbits in the tree, we can end up in the wrong place.
This is only a problem if the value isn't in the (sub)tree, but this
can easily happen even with contiguous trees should deletion occur.
You can see an example in the next patch, which adds a test.
After finding a bug in my intmap_after() routine, I went searching for
other implementations to see how they handled it. Most didn't provide
an open-ended iterator like this, relying on callback iterators which
don't allow deletion. Gah!
The exception was https://github.com/blynn/blt/blob/master/blt.c#L179
which implements blt_ceil() which does this (if you add one to the
key, at least). However, it does it by effectively finding a node,
using that to derive the prefix, then walking down the tree again.
That's pretty suboptimal.
There are basically two choices if you want an efficient after()
operation: to reimplement this approach with some optimizations
(ie. keep branches as we descend, and when we get to the bottom and
know the prefix, we know which branch to go down), or keep the bits
which got to each node.
The latter is more optimal, but less generally useful: for bit
strings, for example, we could keep the bits in common on each node,
rather than storing the entire string at the bottom. But in practice
you'd be doing allocations to re-create the index if the caller wanted
it.
However, in this implementation our keys are 64 bits only, and we
already use a u8 for the bit number: using a 64-bit value there
consumes no more space (thanks to alignment). We can store the
critbit by using the prefix capped by a bit:
0b10000...0000 means
no prefix and highest bit is the critbit, and 0bxxxxx1000...000
means the prefix is xxxxxx and the critbit is the 6th highest bit.
The penalty is that iteration 70% slower. It's still pretty fast
though.
Before:
$ for i in `seq 5`; do ./speed
10000000; done | stats
10000000,random generation (nsec),3-4(3.2+/-0.4)
10000000,critbit insert (nsec),1530-1751(1633.2+/-80)
10000000,critbit successful lookup (nsec),1723-1993(1806.8+/-97)
10000000,critbit failed lookup (nsec),1763-2104(1933.6+/-1.3e+02)
10000000,critbit iteration (nsec),208-266(242.2+/-19)
10000000,critbit memory (bytes),48
10000000,critbit delete (nsec),1747-1861(1803.8+/-42)
10000000,critbit consecutive iteration (nsec),182-228(210+/-18)
After:
10000000,random generation (nsec),3-4(3.2+/-0.4)
10000000,critbit insert (nsec),1533-1699(1628+/-65)
10000000,critbit successful lookup (nsec),1831-2104(1972.4+/-1e+02)
10000000,critbit failed lookup (nsec),1850-2152(2008.2+/-1.1e+02)
10000000,critbit iteration (nsec),304-324(312.8+/-7.5)
10000000,critbit memory (bytes),48
10000000,critbit delete (nsec),1617-1872(1752+/-99)
10000000,critbit consecutive iteration (nsec),303-318(311+/-5.4)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Mar 2018 10:27:12 +0000 (20:57 +1030)]
intmap: add benchmarks.
I wrote these a while ago, dig them out.
On my laptop, min-max(avg+/-stdev) of 5 runs:
make && for i in `seq 5`; do ./speed
10000000; done | stats
make: Nothing to be done for 'all'.
10000000,random generation (nsec),3-4(3.2+/-0.4)
10000000,critbit insert (nsec),1530-1751(1633.2+/-80)
10000000,critbit successful lookup (nsec),1723-1993(1806.8+/-97)
10000000,critbit failed lookup (nsec),1763-2104(1933.6+/-1.3e+02)
10000000,critbit iteration (nsec),208-266(242.2+/-19)
10000000,critbit memory (bytes),48
10000000,critbit delete (nsec),1747-1861(1803.8+/-42)
10000000,critbit consecutive iteration (nsec),182-228(210+/-18)
10000000,hash insert (nsec),396-424(412+/-9.6)
10000000,hash successful lookup (nsec),150-164(157.4+/-5.5)
10000000,hash failed lookup (nsec),163-178(170+/-5.5)
10000000,hash iteration (nsec),21-26(23.2+/-1.7)
10000000,hash memory (bytes),45
10000000,hash delete (nsec),179-194(183.6+/-5.3)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Mar 2018 05:03:11 +0000 (15:33 +1030)]
bitops: new module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Mar 2018 05:02:55 +0000 (15:32 +1030)]
configurator: add tests for other popcount variants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Jan Sarenik [Fri, 23 Feb 2018 05:44:25 +0000 (06:44 +0100)]
Makefile: Fix asort multiple definition error
Error I experienced on Alpine Linux without this patch:
In file included from ccan/generator/generator.c:8:0:
./ccan/generator/generator.h:23:2: error: #error Generators require coroutines
#error Generators require coroutines
^~~~~
make: *** [Makefile:32: ccan/generator/generator.o] Error 1
Yubin Ruan [Wed, 14 Mar 2018 03:16:42 +0000 (11:16 +0800)]
fix misuse of typesafe_cb_cast() in example
From
32f86c701ab0e0ad0ad6981314a9bff2dc5ebb74 Mon Sep 17 00:00:00 2001
From: Yubin Ruan <ablacktshirt@gmail.com>
Date: Wed, 14 Mar 2018 11:14:54 +0800
Subject: [PATCH] fix misuse of typesafe_cb_cast() in example
Signed-off-by: Yubin Ruan <ablacktshirt@gmail.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Yubin Ruan [Mon, 12 Mar 2018 03:24:14 +0000 (11:24 +0800)]
fix misspelling in the example of container_of
From
47c92fe951545e780ca31c598bbcbe5347059b27 Mon Sep 17 00:00:00 2001
From: Yubin Ruan <ablacktshirt@gmail.com>
Date: Mon, 12 Mar 2018 11:22:35 +0800
Subject: [PATCH] fix misspelling in the example of container_of
Signed-off-by: Yubin Ruan <ablacktshirt@gmail.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rusty Russell [Thu, 1 Mar 2018 06:25:51 +0000 (16:55 +1030)]
tal: don't access freed memory on unusual case of redundant tal_free() in destructor.
We already handle normal free traversal loops, just not ones caused by a
direct tal_free() call, such a calling tal_free() on one's own parent.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Feb 2018 04:33:28 +0000 (15:03 +1030)]
intmap: implement uintmap_last/sintmap_last.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 26 Feb 2018 02:26:18 +0000 (12:56 +1030)]
tools/configurator: allow overriding of which cc we will run.
This is for cross-configuring, where we might want to run
`qemu-user-... gcc` or even more exotic things.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 16 Feb 2018 00:06:11 +0000 (10:36 +1030)]
path: expose separator constants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Sun, 4 Feb 2018 23:31:51 +0000 (10:01 +1030)]
io: don't fail if we get a signal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Thu, 21 Dec 2017 23:41:09 +0000 (10:11 +1030)]
tal/path: handle weird case of path_join("")
It seems most sensible to make it a noop, but it definitely shouldn't
access out of bounds as it does.
Reported-by: Russ Dill
Fixes: #61
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 22 Nov 2017 05:44:02 +0000 (16:14 +1030)]
opt: fix libmusl compile.
Fixes: #63
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 25 Oct 2017 05:39:47 +0000 (16:09 +1030)]
io: query whether io_plan in/out have started.
For lightning, we want to hand the socket off to another daemon, but we need
to be on a packet boundary. This lets us check if we've part-read or
part-written.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Thu, 12 Oct 2017 06:33:09 +0000 (17:03 +1030)]
io: fix corner case in EPIPE handling.
If io_read is always called, we don't know that it will actually read,
so it might not notice error. In that case, safest to fail immediately.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Thu, 12 Oct 2017 06:28:34 +0000 (16:58 +1030)]
io: fix duplex read on last bytes of closed pipe.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 4 Sep 2017 00:51:45 +0000 (10:21 +0930)]
ccan/io: example failure
So many bugs in one example program!
There was an unrelated but which strace revealed (trying to write -7
bytes), but I think your issue was more prosaic: failing to zero the
from buffer.
Reported-by: Ian Zimmerman <itz@very.loosely.org>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Akshay Adiga [Tue, 12 Sep 2017 05:23:13 +0000 (10:53 +0530)]
endian: Add Glibc like endianess check
An application built using glibc would expect __BYTE_ORDER to tell if
it should be compiled for BIG_ENDIAN or LITTLE_ENDIAN, whereas ccan uses
HAVE_LITTLE_ENDIAN and HAVE_BIG_ENDIAN for the same purpose.
Hence setting __BYTE_ORDER based on what CCAN provides will no longer
break the applications which check endianness the glibc way.
Signed-off-by: Akshay Adiga <akshay.adiga@linux.vnet.ibm.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 4 Sep 2017 06:57:03 +0000 (16:27 +0930)]
tal: add xor into child_parent pointer.
I had a case where I was handing a sub-object (not a tal object!) to
tal_steal() and it wasn't detected, because the pointers looked correct.
This should help.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 4 Sep 2017 04:36:46 +0000 (14:06 +0930)]
tal: fix up benchmarks for interface changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Damien Grassart [Tue, 29 Aug 2017 10:08:42 +0000 (12:08 +0200)]
darray: Fix bug in the darray_remove() macro
The memmove() call should be using the index argument to determine the
number of bytes to copy. To be consistent with the rest of the code,
we should also not evaluate the index parameter multiple
times. Calling this with rand() % arr.size would otherwise generally
segfault.
Finally, we want to avoid using "index" as an identifier so as to not
shadow index(3) in the C library.
Signed-off-by: Damien Grassart <damien@grassart.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Damien Grassart [Tue, 29 Aug 2017 10:08:41 +0000 (12:08 +0200)]
darray: Rename identifiers starting with an underscore
Identifiers starting with underscores are technically reserved for
system use, so rename all of them to end with one instead.
Signed-off-by: Damien Grassart <damien@grassart.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Damien Grassart [Tue, 29 Aug 2017 10:08:40 +0000 (12:08 +0200)]
darray: Add darray_insert() to insert a value at a specified index
This module currently supports removing but not inserting at a
specified index, so this adds that along with some tests. Inserting a
value moves all existing data beyond index over one element.
Signed-off-by: Damien Grassart <damien@grassart.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rusty Russell [Tue, 15 Aug 2017 04:24:07 +0000 (13:54 +0930)]
shachain: remove unnecessary shachain_index_t
You can use SHACHAIN_BITS to contrain the size.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 15 Aug 2017 04:18:19 +0000 (13:48 +0930)]
shachain: add shachain_next_index()
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
David Gibson [Sun, 23 Jul 2017 05:21:36 +0000 (15:21 +1000)]
objset: Use TCON_WRAP instead of TCON
TCON() uses flexible-array members which aren't allowed in the middle
of structures, except as a gcc extension. TCON_WRAP() avoids this and so
is more portable.
This doesn't change the objset interface, only its internals.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sun, 23 Jul 2017 05:11:33 +0000 (15:11 +1000)]
jmap: Use TCON_WRAP instead of TCON
TCON() uses flexible-array members which aren't allowed in the middle
of structures, except as a gcc extension. TCON_WRAP() avoids this and so
is more portable.
This doesn't change the jmap interface, only its internals.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sun, 23 Jul 2017 04:54:13 +0000 (14:54 +1000)]
jset: Use TCON_WRAP instead of TCON
TCON() uses flexible-array members which aren't allowed in the middle
of structures, except as a gcc extension. TCON_WRAP() avoids this and so
is more portable.
This doesn't change the jset interface, only its internals.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Thu, 20 Jul 2017 14:06:01 +0000 (00:06 +1000)]
tlist: Use TCON_WRAP instead of TCON
TCON() uses flexible-array members which aren't allowed in the middle
of structures, except as a gcc extension. TCON_WRAP() avoids this and so
is more portable.
This doesn't change the tlist interface, only its internals.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rusty Russell [Tue, 27 Jun 2017 05:17:32 +0000 (14:47 +0930)]
io: add io_fd_block() helper.
It's a common thing to want to do, so add helper here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Fri, 16 Jun 2017 03:47:32 +0000 (13:17 +0930)]
io: allow overriding poll function.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 31 May 2017 03:05:45 +0000 (12:35 +0930)]
io: fix nasty io_wake corner case.
If we're duplex and one io_always callback makes the other io_always,
we screwed up and hit an assertion later when the conn was in the
always list but didn't actually want to be.
io_wake() uses io_always(), so this is how it happened. Writing a
test case for this was a bit fun, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
David Gibson [Tue, 4 Apr 2017 13:46:44 +0000 (23:46 +1000)]
agar: Remove unnecessary NULL check
At this point the construction of the function above means that nn cannot
be NULL.
Found by Coverity Scan.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sun, 2 Apr 2017 15:15:53 +0000 (01:15 +1000)]
net: Add check for failure of setsockopt()
make_listen_fd() didn't check for failure of setsockopt(). There's no
real reason not to, since we have an obvious way to report an error to the
caller.
Found with Coverity Scan.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sun, 2 Apr 2017 11:29:24 +0000 (21:29 +1000)]
ccanlint: Fix leak in do_reduce_features()
options_avail and options_used get freed, but options does not.
Found by Coverity scan.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sun, 2 Apr 2017 11:21:02 +0000 (21:21 +1000)]
crypto/ripemd160: Correct badly sized union member
struct ripemd160_ctx has a union for converting between u8[] and u32[]
data. Unfortunately the u32 array has a miscalculated size, half the size
of the u8 array. That means some accesses which are within the union can
technically overrun the u32 array.
Found by Coverity scan.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sun, 2 Apr 2017 09:17:12 +0000 (19:17 +1000)]
tools: Remove fd leak
compile_info() can leak an open file descriptor write_all() fails. This
corrects it.
Found by Coverity.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sun, 2 Apr 2017 09:06:06 +0000 (19:06 +1000)]
failtest: Remove memory leak
Somewhat ironically, a path in failtest related to detecting leaks in the
tested program itself leaks memory. This corrects it.
Detected by Coverity.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rusty Russell [Mon, 3 Apr 2017 00:08:51 +0000 (09:38 +0930)]
.gitignore: ignore .fast-ok files, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Mon, 3 Apr 2017 00:08:26 +0000 (09:38 +0930)]
ccan/ntdb: demote to junkcode.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
David Gibson [Fri, 31 Mar 2017 13:48:22 +0000 (00:48 +1100)]
Fix missing va_end()s
This corrects several places in ccan where stdarg.h is used but there is a
missing va_end(). You can get away with this on many platforms, but not
all.
Caught by Coverity scan.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Fri, 31 Mar 2017 12:51:22 +0000 (23:51 +1100)]
lbalance: Switch to tlist2
lbalance uses the tlist module. tlist causes compile warnings on clang if
you're not careful, because it can put 0 length arrays in the middle of
structures. tlist2 doesn't have the problem, and also has a slightly
cleaner interface.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Fri, 31 Mar 2017 11:39:10 +0000 (22:39 +1100)]
tools/ccanlint: Add missing header file
tools/ccanlint/async.c uses kill(2), but doesn't include the signal.h
header it comes from. One some platforms we get away with this via
indirect includes, but not on all.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Fri, 31 Mar 2017 11:25:07 +0000 (22:25 +1100)]
tools: Add asort module
tools/manifest.c uses asort(), but the asort module is not in
TOOLS_CCAN_MODULES. That causes compile failures on some platforms, so
correct it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rusty Russell [Wed, 15 Mar 2017 03:55:07 +0000 (14:25 +1030)]
take, tal, tal/path, tal/str, tal/talloc: annotate APIs with TAKES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 15 Mar 2017 03:40:08 +0000 (14:10 +1030)]
take: add labels when CCAN_TAKE_DEBUG set, return in taken_any().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 15 Mar 2017 02:54:02 +0000 (13:24 +1030)]
Makefile: fix fastcheck.
commit
25b7406d11ea3703be864097af66ce95611dde72 tried to make the tests
depend on the info file, but that broke .fast.ok, which used the same
pattern:
%.ok: $(LINT) %info
This is what happens when you're too tricky! Simply duplicate the rule,
and change .fast.ok to .fast-ok so it doesn't match both.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 15 Mar 2017 02:38:13 +0000 (13:08 +1030)]
io/fdpass: fix example.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 14 Mar 2017 02:47:15 +0000 (13:17 +1030)]
io/fdpass: add flag to close fd after sending.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Tue, 14 Mar 2017 01:45:19 +0000 (12:15 +1030)]
io: add io_flush_sync().
This is needed for emergency handling in lightningd: we want to output
a (fatal) error packet on the socket, but we don't want to do so in the middle
of another packet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Martin Milata [Sun, 5 Feb 2017 13:29:45 +0000 (14:29 +0100)]
hex: fix comment
Signed-off-by: Martin Milata <martin@martinmilata.cz>
David Gibson [Tue, 24 Jan 2017 09:45:53 +0000 (20:45 +1100)]
.travis.yml: Add clang builds to trusty
This enables clang compiler builds for the trusty Travis environment.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Fri, 20 Jan 2017 12:49:43 +0000 (23:49 +1100)]
coroutine: Stack allocation
At present, coroutine stacks must be allocated explicitly by the user,
then initialized with coroutine_stack_init(). This adds a new
coroutine_stack_alloc() function which allocates a stack, making life
easier for users. coroutine_stack_release() will automatically determine
if the given stack was set up with _init() or alloc() and act
accordingly.
The stacks are allocate with mmap() rather than a plain malloc(), and a
guard page is added, so an overflow of the stack should result in a
relatively debuggable SEGV instead of random data corruption.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sat, 24 Dec 2016 10:08:55 +0000 (21:08 +1100)]
coroutine: Enable valgrind
Currently valgrind checks are disabled on the coroutine module,
because switching stacks tends to confuse it. We can work around this
by using the valgrind client interface to explicitly inform it about
the stacks we create.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sat, 24 Dec 2016 10:40:00 +0000 (21:40 +1100)]
coroutine: Remove on-stack buffers from testcases
In preparation for enabling valgrind tests, remove instances where we
allocate a coroutine's stack from a buffer itself on the stack. Not all
that surprisingly, valgrind gets very, very confused by having one
"thread"'s stack embedded within another's.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Tue, 24 Jan 2017 09:39:45 +0000 (20:39 +1100)]
coroutine: Move total initialization outside coroutine
The sample coroutine in api-3 initializes a total to 0, then adds up the
pseudo-random data it has placed into a stack buffer, to ensure that the
compiler won't elide the reading and writing of that buffer. After the
coroutine has completed, we verify that total is non-zero so that we'll
detect if the coroutine failed to execute entirely.
Except that the initialization of total is within the coroutine itself,
so it could also be non-zero due to it simply being uninitialized. This
moves the initialization outside the coroutine, to make the test a little
more robust.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Tue, 24 Jan 2017 06:59:16 +0000 (17:59 +1100)]
coroutine: Remove problematic diagnostic from api-3 test
The api-3 testcase devotes most of its available stack space to a test
buffer, leaving only a small amount (COROUTINE_MIN_STKSZ) for the actual
stack usage of the coroutine.
It turns out that the ccan/tap diag() function can - depending on compiler
version and flags, and on whether diagnostics are enabled - exceed that
limited stack space. That leads to a stack overrun, and in turn corruption
of the parent routine's stack, generating unpredictable and hard to debug
SEGVs.
At present, this bug seems to be tripped by clang-3.8 when diagnostic
messages are printed.
This removes the troublesome diag() call.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rusty Russell [Tue, 24 Jan 2017 06:22:00 +0000 (16:52 +1030)]
tal: make tal_len/tal_count(NULL) return 0.
Previously it crashed, but if you're always dealing with tal arrays,
this is painful.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
David Gibson [Thu, 19 Jan 2017 01:58:24 +0000 (12:58 +1100)]
.travis.yml: Add compilation with gcc 5 and gcc 6
Now that we have a way to correctly set a matching coverage tool, we can
add more recent compiler versions to the Travis build.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Wed, 18 Jan 2017 21:56:48 +0000 (08:56 +1100)]
ccanlint: Correct default coverage tool for clang
Currently ccanlint defaults to using "gcov" as the coverage analysis tool
for any compiler defining __GNUC__. That's generally correct for the
(system default) gcc. However, clang also defines __GNUC__ because it
implements the GCC langauge extensions. For clang, "gcov" is not the
correct coverage tool (clang does use roughly the gcov format, but unless
you're very lucky the system gcc and system clang won't use the same gcov
versions).
This changes the default coverage tool in the case of clang to the correct
"llvm-cov gcov".
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Wed, 18 Jan 2017 03:23:51 +0000 (14:23 +1100)]
ccanlint: Allow path to gcov to be overriden
Currently ccanlint always assumes that the coverage tool can be
invoked under the command "gcov".
However, the coverage tool generally needs to be closely matched to
the compiler version. So, the current behaviour won't work with
compilers other than gcc, like clang. It won't even work for a gcc
version which isn't the standard system one matching gcov.
To address this, allow the command for the coverage tool to be
overridden on the ccanlint command line with a new --gcov option. We
also allow it to be overridden for make check with a GCOV make
variable.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Wed, 18 Jan 2017 03:09:29 +0000 (14:09 +1100)]
tools: Consolidate gcov handling
At the moment, invocation of the 'gcov' tool for coverage analysis
from ccanlint is put directly into the tests_compile_coverage.c and
tests_coverage.c files. This makes it awkard to extend.
So, this patch moves the invocation of gcov into a new tools/gcov.v
file, analagous to tools/compile.c.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>