ccan
3 years agoEnsure config.h is created before the main travis builds
Jon Griffiths [Mon, 29 Aug 2016 01:10:29 +0000 (13:10 +1200)]
Ensure config.h is created before the main travis builds

The current makefiles are not parallel safe and reliably fail on the
clang build. The real fix is to rewrite the makefiles, but in the
meantime, allow the clang tests to pass.

3 years agoMerge remote-tracking branch 'origin/pr/47'
Rusty Russell [Tue, 23 Aug 2016 03:39:44 +0000 (13:09 +0930)]
Merge remote-tracking branch 'origin/pr/47'

Closes: 47

3 years agoMerge remote-tracking branch 'origin/pr/46'
Rusty Russell [Tue, 23 Aug 2016 03:37:07 +0000 (13:07 +0930)]
Merge remote-tracking branch 'origin/pr/46'

Closes: 46

3 years agoMerge remote-tracking branch 'origin/pr/45'
Rusty Russell [Tue, 23 Aug 2016 03:33:10 +0000 (13:03 +0930)]
Merge remote-tracking branch 'origin/pr/45'

Closes: 45

3 years agoripemd160: Remove unused static function Initialize()
Jon Griffiths [Mon, 22 Aug 2016 13:20:32 +0000 (01:20 +1200)]
ripemd160: Remove unused static function Initialize()

The header macro RIPEMD160_INIT is intended to be used instead I believe.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agoripemd160: Declare variable at the start of its function
Jon Griffiths [Mon, 22 Aug 2016 11:34:19 +0000 (23:34 +1200)]
ripemd160: Declare variable at the start of its function

The resulting code is just as clear to read, so lets not require
a C99 compiler to compile this.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agohex: Simplify hex_encode
Jon Griffiths [Mon, 30 May 2016 12:39:10 +0000 (00:39 +1200)]
hex: Simplify hex_encode

The documentation for hex_encode indicates that it returns simply true or
false. The old implementation was returning the written length on success,
cast to boolean. This will elicit a warning under MSVC.

On further examination, there is no need to check/modify the length inside
the loop, since we can check it once before starting. As a result the code
can be simplified a bit.

A side affect of this change is that nothing will be written at all if the
length is incorrect, vs the previous code writing characters until the length
available is exhausted. I prefer the new semantics but YMMV.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agosha256: Use fewer magic magic numbers
Jon Griffiths [Wed, 23 Mar 2016 04:25:29 +0000 (17:25 +1300)]
sha256: Use fewer magic magic numbers

Show the derivation of the constants to match the comment above them.
The compiler doesn't care, but it helps the code read better.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agosha256: Move 'bytes' to the end of sha256_ctx and make it a size_t
Jon Griffiths [Tue, 22 Mar 2016 22:54:50 +0000 (11:54 +1300)]
sha256: Move 'bytes' to the end of sha256_ctx and make it a size_t

The code already assigns to/from bytes as a size_t, so make it
official (and better on platforms with a 32 bit size_t).

Moving bytes makes it act as a canary in the event that there is a rogue
write/off by one somewhere - since it ends up in the hash we are
more likely to detect this should we corrupt it. This also makes the
working buffer better aligned which can't hurt.

Also, initialise the buffer to zero while we are changing the initialisation
macro anyway. It costs little compared to the hashing overhead, should be
optimised away if redundant in most cases, and it removes a warning from both
gcc and clang about unititialised struct members.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agosha256: Use the same union order as the ctx structure
Jon Griffiths [Tue, 22 Mar 2016 22:51:50 +0000 (11:51 +1300)]
sha256: Use the same union order as the ctx structure

This reads better. Also remove duplicated comments for the members.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agosha256: Make our u32 and u8 fields the same size
Jon Griffiths [Tue, 22 Mar 2016 22:49:48 +0000 (11:49 +1300)]
sha256: Make our u32 and u8 fields the same size

These are just aliases to a buffer: its customary for these to
have the same size, and makes sizeof() consistent in case anyone
decides to use the members instead of the containing union.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agosha256: Mark parameters as potentially unused
Jon Griffiths [Tue, 15 Mar 2016 06:48:29 +0000 (19:48 +1300)]
sha256: Mark parameters as potentially unused

Prevents warnings from gcc at higher warning levels.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agocrypto: Comment typo
Jon Griffiths [Tue, 22 Mar 2016 11:04:04 +0000 (00:04 +1300)]
crypto: Comment typo

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agocrypto: Remove C++ comments
Jon Griffiths [Tue, 22 Mar 2016 06:57:29 +0000 (19:57 +1300)]
crypto: Remove C++ comments

Prevents warnings from gcc at higher warning levels.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agoripemd160: Fix a name typo
Jon Griffiths [Mon, 28 Mar 2016 14:09:30 +0000 (03:09 +1300)]
ripemd160: Fix a name typo

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agoripemd160: Mark parameters as potentially unused
Jon Griffiths [Tue, 22 Mar 2016 06:48:39 +0000 (19:48 +1300)]
ripemd160: Mark parameters as potentially unused

Prevents warnings from gcc at higher warning levels.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
3 years agocast: fix indentation in example.
Rusty Russell [Tue, 28 Jun 2016 20:39:52 +0000 (06:09 +0930)]
cast: fix indentation in example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agoaltstack: Don't log internal calls in test cases
David Gibson [Fri, 3 Jun 2016 08:42:03 +0000 (18:42 +1000)]
altstack: Don't log internal calls in test cases

altstack/test/run.c uses some hairy macros to intercept the standard
library functions that altstack uses.  This has two purposes: 1) to
conditionally cause those functions to fail, and thereby test altstack's
error paths, and 2) log which of the library functions was called in each
testcase.

The second function isn't actually useful - for the purposes of testing the
module, we want to check the actual behaviour, not what calls it made in
what order to accomplish it.  Explicitly checking the calls makes it much
harder to change altstack's implementation without breaking the tests.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoaltstack: Don't use 0 pointer literals
David Gibson [Fri, 3 Jun 2016 08:42:02 +0000 (18:42 +1000)]
altstack: Don't use 0 pointer literals

In a number of places the altstack module uses a literal '0' for pointer
values.  That's correct C, but doesn't make it obvious on a quick read
whether values are integers or pointers.  This patch changes those cases
to use the NULL define instead.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoaltstack: Use ptrint instead of bare casts
David Gibson [Fri, 3 Jun 2016 08:42:01 +0000 (18:42 +1000)]
altstack: Use ptrint instead of bare casts

Functions invoked with altstack take a void * parameter.  However, the
test program wants to pass an integer, and so uses the trick of casting
the integer values to (void *) and back again.

The ptrint() module handles exactly this case in a more portable and
(somewhat) typesafe way, so use that instead.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoaltstack: Restore alternate signal stack state
David Gibson [Fri, 3 Jun 2016 08:42:00 +0000 (18:42 +1000)]
altstack: Restore alternate signal stack state

altstack relies on catching a SIGSEGV caused when overrunning the stack.
This means that the SEGV handler itself can't use the already overflowed
stack, and so we use sigaltstack() to assign the signal handler a different
stack.  On completion, altstack() clears the alternate signal stack.

However, it's possible that the calling program could be using
sigaltstack() for its own reasons, so it's more correct to restore the
sigaltstack() state to that from the beginning of the altstack() call.
This patch implements this behaviour.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agoaltstack: Consolidate thread-local variables
David Gibson [Fri, 3 Jun 2016 08:41:59 +0000 (18:41 +1000)]
altstack: Consolidate thread-local variables

altstack uses a number of __thread variables to track internal state.  This
allows altstack to be thread-safe, although it's still not re-entrant.
This patch gathers all these variables into a single per-thread state
structure.  This makes it easy to see at a glance what the whole of the
required state is, and thereby easier to reason about correctness of
changes to the implementation.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
3 years agohtable: allow htable_type keys to be non-pointers.
Rusty Russell [Mon, 6 Jun 2016 04:37:44 +0000 (14:07 +0930)]
htable: allow htable_type keys to be non-pointers.

Common case is mapping ints to structures.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agohtable: htable_type add htable_copy.
Rusty Russell [Mon, 6 Jun 2016 04:20:39 +0000 (13:50 +0930)]
htable: htable_type add htable_copy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
3 years agohtable: add htable_copy.
Rusty Russell [Mon, 6 Jun 2016 04:14:36 +0000 (13:44 +0930)]
htable: add htable_copy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agohtable: add a htable_prev method to oppose _next
Cody P Schafer [Fri, 3 Jun 2016 15:55:45 +0000 (11:55 -0400)]
htable: add a htable_prev method to oppose _next

Useful for unwinding actions taken while iterating over a htable.

Signed-off-by: Cody P Schafer <dev@codyps.com>
4 years agotlist: Add tlist_next() and tlist_prev() functions
David Gibson [Sat, 4 Jun 2016 10:21:33 +0000 (20:21 +1000)]
tlist: Add tlist_next() and tlist_prev() functions

An odd omission from the tlist module is basic tlist_next() and
tlist_prev() macros matching list_next() and list_prev() in the basic
list module.  This adds them.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoagar: Add static graph initializer
David Gibson [Fri, 3 Jun 2016 10:07:10 +0000 (20:07 +1000)]
agar: Add static graph initializer

Sometimes it's not convenient to initialize an agar graph at runtime with
agar_init_graph().  This adds an AGAR_INIT_GRAPH() macro to do the same
thing as a static initializer.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoaga: Remove unused state defines
David Gibson [Tue, 31 May 2016 10:09:30 +0000 (20:09 +1000)]
aga: Remove unused state defines

These were left over from a previous approach which didn't pan out.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agotcon: add testcase for const members in tcon_container_of()
Rusty Russell [Thu, 2 Jun 2016 02:00:53 +0000 (11:30 +0930)]
tcon: add testcase for const members in tcon_container_of()

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agotcon: add test case for const members.
Rusty Russell [Thu, 2 Jun 2016 01:58:54 +0000 (11:28 +0930)]
tcon: add test case for const members.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agotlist2: a alternate to tlist that encodes the member offset into the container type
Cody P Schafer [Tue, 31 May 2016 19:09:13 +0000 (15:09 -0400)]
tlist2: a alternate to tlist that encodes the member offset into the container type

Signed-off-by: Cody P Schafer <dev@codyps.com>
4 years agotcon: avoid error 'expected `void *` but argument is of type `const void *`' in tcon_...
Cody P Schafer [Tue, 31 May 2016 19:30:40 +0000 (15:30 -0400)]
tcon: avoid error 'expected `void *` but argument is of type `const void *`' in tcon_container_of()

I did not see any failures of other modules with this change, but I'm
not completely certain there are no cases that it breaks.

Signed-off-by: Cody P Schafer <dev@codyps.com>
4 years agolist: add parens to gaurd macro args in LIST_INIT
Cody P Schafer [Tue, 31 May 2016 18:24:46 +0000 (14:24 -0400)]
list: add parens to gaurd macro args in LIST_INIT

When other macros are emitting LIST_INIT expansions, `name` can get a
bit complicated.

Signed-off-by: Cody P Schafer <dev@codyps.com>
4 years agohtable/htable_type: allow keyof to be a simple macro
Cody P Schafer [Tue, 31 May 2016 16:06:16 +0000 (12:06 -0400)]
htable/htable_type: allow keyof to be a simple macro

Without this, one had to either use a function or a macro with casts to
get the expected behavior.

Signed-off-by: Cody P Schafer <dev@codyps.com>
4 years agohtable/htable_type: avoid warning about an unused argument
Cody P Schafer [Tue, 31 May 2016 16:05:33 +0000 (12:05 -0400)]
htable/htable_type: avoid warning about an unused argument

Signed-off-by: Cody P Schafer <dev@codyps.com>
4 years agocrypto/siphash: new module.
Rusty Russell [Tue, 31 May 2016 02:35:44 +0000 (12:05 +0930)]
crypto/siphash: new module.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoBuild info files
David Gibson [Mon, 9 May 2016 19:44:47 +0000 (20:44 +0100)]
Build info files

All modules have a _info file (a C file, despite the lack of extension)
giving metadata for the module.  The Makefiles have a rule to build these..
but it's broken (missing an include directive).

This patch fixes the rule, and builds the info binaries for all modules by
default.  This is a useful check and also useful for manually inspecting
a module's metadata.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoAutomatically determine which modules have source
David Gibson [Thu, 5 May 2016 15:40:48 +0000 (16:40 +0100)]
Automatically determine which modules have source

Currently, build of ccan is controlled by two Make variables: MODS_WITH_SRC
and MODS_NO_SRC which list modules containing .c files, and modules with
only .h files respectively.

When adding new modules this is fiddly to get right, and there are a number
of modules already listed in the wrong variable.  There's also some
redundant logic in the DIRS variable to again filter out modules without
source.

This simplifies things by having a single manually updated MODS variable
listing every module, and determining MODS_WITH_SOURCE programmatically.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoMakefile: exclude altstack so Jenkins works again.
Rusty Russell [Mon, 9 May 2016 01:34:33 +0000 (11:04 +0930)]
Makefile: exclude altstack so Jenkins works again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoccanlint: fix missing file.
Rusty Russell [Mon, 9 May 2016 01:16:53 +0000 (10:46 +0930)]
ccanlint: fix missing file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agotools/ccanlint: make sure _info compiles.
Rusty Russell [Fri, 6 May 2016 00:44:10 +0000 (10:14 +0930)]
tools/ccanlint: make sure _info compiles.

We used to crash, as reported by Stephen M. Cameron

Closes: #39
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoa_star: new module added to hacky Makefile-ccan list.
Rusty Russell [Tue, 3 May 2016 20:53:24 +0000 (06:23 +0930)]
a_star: new module added to hacky Makefile-ccan list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoAdd A-star module
Stephen M. Cameron [Tue, 3 May 2016 06:02:26 +0000 (23:02 -0700)]
Add A-star module

Signed-off-by: Stephen M. Cameron <stephenmcameron@gmail.com>
4 years agoCorrectly include dependencies for nested modules
David Gibson [Tue, 16 Feb 2016 12:35:41 +0000 (23:35 +1100)]
Correctly include dependencies for nested modules

Currently we pull auto-generated dependencies into the Makefile with
include ccan/*/*.d.  That will omit any .d files from nested modules,
meaning things might not be correctly rebuilt there.

Correct this by using the list of modules instead of a 1-level wildcard.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoExclude system headers from .d files
David Gibson [Tue, 16 Feb 2016 12:32:34 +0000 (23:32 +1100)]
Exclude system headers from .d files

We currently generated .d dependency files with the -MD option to cc.  That
includes system header files in the dependencies, which isn't often useful
and makes the .d more complicated than necessary.

This changes to -MMD which excludes system headers.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoClean up use of 'rm' in Makefiles
David Gibson [Tue, 16 Feb 2016 12:24:05 +0000 (23:24 +1100)]
Clean up use of 'rm' in Makefiles

Most of the ccan Makefiles use $(RM) to remove files.  However, 'rm' is
traditionally considered one of the few shell tools which can be used in
Makefiles without indirecting via a variable.

rm is also typically invoked with -f in Makefiles, so that it doesn't cause
errors if the files don't exist (because they haven't been built).  A
number of instances in ccan were missing this.

This corrects these warts.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoAdd missing files to make clean
David Gibson [Tue, 16 Feb 2016 11:53:50 +0000 (22:53 +1100)]
Add missing files to make clean

At present, "make clean" will not remove the module-Makefile files for
non-top-level modules.  It also won't remove the generated config.h.
Correct those errors.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoconfigurator: fix HAVE_UCONTEXT test on Ubuntu 16.04.
Rusty Russell [Tue, 26 Apr 2016 04:55:22 +0000 (14:25 +0930)]
configurator: fix HAVE_UCONTEXT test on Ubuntu 16.04.

Seems to want more stack.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agogenerator: don't even try to compile if !HAVE_UCONTEXT.
Rusty Russell [Tue, 26 Apr 2016 04:54:21 +0000 (14:24 +0930)]
generator: don't even try to compile if !HAVE_UCONTEXT.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoMakefile: add altstack and generator to build exclusions.
Rusty Russell [Tue, 26 Apr 2016 04:41:01 +0000 (14:11 +0930)]
Makefile: add altstack and generator to build exclusions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoMakefile: fix random pattern hack.
Rusty Russell [Tue, 26 Apr 2016 04:40:18 +0000 (14:10 +0930)]
Makefile: fix random pattern hack.

Turns out that patterns with / cause % to match /.  OK...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoMakefile-ccan: add cppmagic.
Rusty Russell [Tue, 26 Apr 2016 04:23:15 +0000 (13:53 +0930)]
Makefile-ccan: add cppmagic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agohtable: add iterators to htable_type.
Rusty Russell [Tue, 26 Apr 2016 04:18:40 +0000 (13:48 +0930)]
htable: add iterators to htable_type.

Useful if you have more than one object with same key.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agostrgrp: Add cosine fudge-curve to unify filter comparison spaces
Andrew Jeffery [Sat, 20 Feb 2016 10:52:41 +0000 (21:22 +1030)]
strgrp: Add cosine fudge-curve to unify filter comparison spaces

If we are to use should_grp_score_cos(x,y) as a filter the the following
relationship must hold (from least to most expensive):

        should_grp_score_len(x,y)
                >= should_grp_score_cos(x,y)
                >= grp_score(x)

should_grp_score_cos(x,y) wasn't holding up its part of the bargain, so
real data was used to generate a fudge curve to bring
should_grp_score_cos(x,y) results into the same space. Really this is a
terrible hack and the problem needs more thought. Evaluation of
should_grp_score_cos(x,y)'s performance benefit (given the relaxation of
the filter under the fudge curve) is sorely needed.

4 years agostrgrp: Use angular similarity for distance metric properties
Andrew Jeffery [Sat, 20 Feb 2016 10:49:53 +0000 (21:19 +1030)]
strgrp: Use angular similarity for distance metric properties

Distance metrics allow us to compare similarity results, however
applying the change leads to test suite breakage as we no longer satisfy
the requirement that each filter's score is at most as large as that of
the previous filter^. As such, also stop ccanlint from executing the
tests that are known to fail until we work around the problem.

^ This is a problem that has existed since the introduction of the
cosine similarity filter, it just wasn't detected by the test suite.

4 years agostrgrp: Use ratio of hypotenuse for consistent comparisons
Andrew Jeffery [Sat, 20 Feb 2016 11:03:04 +0000 (21:33 +1030)]
strgrp: Use ratio of hypotenuse for consistent comparisons

Ensure comparing filter results is sensible by using a consistent
calculation. Note that the cosine similarity measurement doesn't yet
conform and this can give spurious results that are not detected by the
test suite.

4 years agostrgrp: Shift constant out of loop
Andrew Jeffery [Sat, 20 Feb 2016 11:01:43 +0000 (21:31 +1030)]
strgrp: Shift constant out of loop

Likely this was optimised away, but the code now represents the intent.

4 years agoshachain: clarify design in terms of binary tree, reverse indexes.
Rusty Russell [Tue, 8 Mar 2016 05:38:36 +0000 (16:08 +1030)]
shachain: clarify design in terms of binary tree, reverse indexes.

Olaoluwa Osuntokun came up with an alternative which used binary trees;
that's a much better way to explain it, so do that in design.txt and
update the implementation to work the same way.

Anthony Towns pointed out that the numbering is the reverse of the normal
hash chaining descriptions, so fix that too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agogenerator: Allow generators to take arguments
David Gibson [Fri, 12 Feb 2016 12:49:49 +0000 (23:49 +1100)]
generator: Allow generators to take arguments

Using some serious macro magic, this patch extends generators to allow
them to take arbitrary arguments.  The arguments are marshalled into a
structure placed at the far end of the generator's stack when it is
created.  Then, they're unmarshalled back into C parameters when we first
context switch into the generator.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agogenerator: Generators for C
David Gibson [Thu, 25 Feb 2016 11:07:17 +0000 (22:07 +1100)]
generator: Generators for C

Generators are a limited form of co-routine, which people may be familiar
with from Python.  This module adds an implementation of generators for C.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoconfigurator: Add test for ucontext.h
David Gibson [Sat, 19 Jul 2014 07:00:15 +0000 (17:00 +1000)]
configurator: Add test for ucontext.h

This adds a new HAVE_UCONTEXT define, which indicates that ucontext.h
is present, and more-or-less works.

It also adds HAVE_POINTER_SAFE_MAKECONTEXT, which indicates whether
pointer valued arguments can be passed through the varargs parameters
to makecontext().

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoaltstack: Clarify checking macros
David Gibson [Mon, 15 Feb 2016 11:49:54 +0000 (22:49 +1100)]
altstack: Clarify checking macros

The chkfail() and chkok() macros in altstack's test program are pretty
difficult to read.  More importantly, though, they do all their tests with
one big ok1().  That means if the test fails, you get no indication which
of the checks was actually wrong, making debugging harder.

This reworks the macros into a more verbose form that's easier to read,
and splits them into multiple ok1() tests to make failures more explicit.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoaltstack: Declare memory clobbers
David Gibson [Mon, 15 Feb 2016 11:47:36 +0000 (22:47 +1100)]
altstack: Declare memory clobbers

altstack includes a couple of inline asm blocks with x86 push and pop
instructions.  These instructions will access memory (the stack), but
that's not declared in inline asm statement.  We seem to be getting away
with it, but in theory that could allow the compiler to re-order accesses
to local variables across the asm block.  Since those blocks change the
location of the stack, that could be very bad.

Adding a "memory" clobber should prevent this (effectively making the asm
blocks a compiler memory barrier).

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoaltstack: Include config.h in run.c
David Gibson [Mon, 15 Feb 2016 11:43:07 +0000 (22:43 +1100)]
altstack: Include config.h in run.c

ccan programs should always include config.h before anything else to make
sure everything is set up correctly.  Doing so in altstack's run.c means
it no longer needs an explicit _XOPEN_SOURCE 700, since _GNU_SOURCE is set
in config.h (for GNU libc, anyway).

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoaltstack: Increase signal stack size
David Gibson [Mon, 15 Feb 2016 11:01:18 +0000 (22:01 +1100)]
altstack: Increase signal stack size

At present the altstack module uses a stack of size MINSIGSTKSZ for its
SIGSEGV handler.  Although MINSIGSTKSZ is defined to be large enough to
execute a signal handler, it doesn't guarantee that you can do anything
very much within it.

With certain libc versions, MINSIGSTKSZ is not enough to execute the
longjmp() used in altstack.  Specfically, with Ubuntu 12.04 (the default
install for Travis containers), the first time longjmp() is executed the
symbol must be resolved by the dynamic linker in a process which overruns
the MINSIGSTKSZ sized stack.  That then corrupts local variables in
altstack() itself causing a number of subsequent failures.

This patch addresses the problem by changing from MINSIGSTKSZ to SIGSTKSZ
which is supposed to cover "the usual requirements for an alternate signal
stack".

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoaltstack: stack alignment and accounting tweaks
Dan Good [Mon, 8 Feb 2016 20:57:23 +0000 (20:57 +0000)]
altstack: stack alignment and accounting tweaks

* add altstack_remn, returns amount of stack remaining
* increase mapping by 1 page to handle abutment case
* capture rsp earlier
* align stack to 16 bytes

Signed-off-by: Dan Good <dan@dancancode.com>
4 years agoccanlint: make _info ported an empty string on success.
Rusty Russell [Fri, 5 Feb 2016 03:39:45 +0000 (14:09 +1030)]
ccanlint: make _info ported an empty string on success.

Otherwise it describes what we need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agocppmagic: Iteration
David Gibson [Tue, 26 Jan 2016 10:54:47 +0000 (21:54 +1100)]
cppmagic: Iteration

This implements macros which iterate across their arguments.  This is
implemented in terms of (kinda sorta) recursion.  In fact, they will stop
working with enough arguments, but the limit is large and can be easily
increased by changing the depth of the CPPMAGIC_EVAL() macro.

There are 3 iterators (for now):
  CPPMAGIC_MAP
    applies another macro to each of its remaining arguments - the results
    are comma separated, so they can be passed into another CPPMAGIC_MAP
    invocation.
  CPPMAGIC_2MAP
    does the same thing, but takes the arguments a pair at a time, using
    a supplied two-argument macro.
  CPPMAGIC_JOIN
    combines the arguments with a chosen delimiter (effectively replacing
the commas between the arguments with the delimiter)
same thing, but takes the arguments a pair at a time.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agocppmagic: Allow multiple and deferred evaluation
David Gibson [Tue, 26 Jan 2016 10:32:07 +0000 (21:32 +1100)]
cppmagic: Allow multiple and deferred evaluation

Recursion (and therefore iteration) in cpp is difficult, since the
preprocessor explicitly looks for and inhibits recursion.

But, it's possible to trick it, up to a point.  CPPMAGIC_DEFER1() and
CPPMAGIC_DEFER2() can "hide" a macro, preventing it from being expanded
and being noticed as recursion.

Along with that we need to cause extra expansion passes to be executed.
There has to be a finite limit here - true recursion is impossible - but
that number can be made very large pretty easily.  CPPMAGIC_EVAL() multiply
expands its argument(s) - up to 1024 times.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agocppmagic: Conditionals
David Gibson [Tue, 26 Jan 2016 10:00:01 +0000 (21:00 +1100)]
cppmagic: Conditionals

Implement CPPMAGIC_IFELSE which operates similar to the C ? : operator, but
is evaluated at preprocessing time.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agocppmagic: Logical operations
David Gibson [Tue, 26 Jan 2016 09:52:32 +0000 (20:52 +1100)]
cppmagic: Logical operations

In order to implement fancier things, we need to represent truth values in
cpp.  We use '0' and '1' strings, like in C, but we need ways to get these
values from other conditions.

CPPMAGIC_ISZERO() and CPPMAGIC_NONZERO() test if the argument is '0' or
anything else (ISZERO doubles as a logical not).

CPPMAGIC_ISEMPTY() and CPPMAGIC_NON_EMPTY() expand to 0 or 1 depending on
whether they have any arguments at all or not.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agocppmagic: New module
David Gibson [Tue, 26 Jan 2016 09:51:57 +0000 (20:51 +1100)]
cppmagic: New module

A module for some of the awesome / horrifying techniques described at:
    http://jhnet.co.uk/articles/cpp_magic
    https://github.com/pfultz2/Cloak/wiki/C-Preprocessor-tricks,-tips,-and-idioms

Start off with just some simple things.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years ago.travis.yml: Add -k to make check script
David Gibson [Mon, 1 Feb 2016 11:54:32 +0000 (22:54 +1100)]
.travis.yml: Add -k to make check script

At the moment when Travis runs make check it will stop on the first
failure.  That's not particularly useful, so add a -k so that all ccanlint
failures can be seen.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoccanlint: Report failures with --summary
David Gibson [Mon, 1 Feb 2016 11:50:45 +0000 (22:50 +1100)]
ccanlint: Report failures with --summary

When run in --summary mode ccanlint doesn't actually report whether it
passed or failed in the message .  In particular this means that when make
check is run with -j, it can be hard to tell which modules failed.

This adds a more obvious failure message.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoccanlint: add missing file for "info_ported" test.
Rusty Russell [Wed, 3 Feb 2016 06:04:05 +0000 (16:34 +1030)]
ccanlint: add missing file for "info_ported" test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agorszshm: disable valgrind for tests.
Rusty Russell [Wed, 3 Feb 2016 05:41:05 +0000 (16:11 +1030)]
rszshm: disable valgrind for tests.

It returns EINVAL instead of ENOMEM for test/run.c line 96, then
complains on line 137:

==29368== Invalid read of size 4
==29368==    at 0x4033BC: main (run.c:137)
==29368==  Address 0x400000000018 is not stack'd, malloc'd or (recently) free'd

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agomem: add memtaint().
Rusty Russell [Wed, 3 Feb 2016 03:41:38 +0000 (14:11 +1030)]
mem: add memtaint().

Useful if you're going to reuse a buffer later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoaltstack: fix _info test.
Rusty Russell [Wed, 3 Feb 2016 03:01:44 +0000 (13:31 +1030)]
altstack: fix _info test.

Since we use -Wundef by default, ccanlint gets upset if __X86_64__ isn't set.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agotools: commit missing support for _info ported flag.
Rusty Russell [Wed, 3 Feb 2016 02:49:57 +0000 (13:19 +1030)]
tools: commit missing support for _info ported flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoccanlint: add "_info ported" (for ccan/altstack).
Rusty Russell [Wed, 3 Feb 2016 02:27:33 +0000 (12:57 +1030)]
ccanlint: add "_info ported" (for ccan/altstack).

If _info handles the arg "ported" it should print out 1 or 0; 0 means
it can't be compiled/run/tested on this platform.  This lets ccanlint
easily skip such modules.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoidtree: Fix undefined behaviour (left shift of signed value)
David Gibson [Wed, 27 Jan 2016 13:17:47 +0000 (00:17 +1100)]
idtree: Fix undefined behaviour (left shift of signed value)

~0 will be signed and negative on any 2s complement system, and
left shifting a negative value has undefined behaviour in C.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoidtree: Fix misindented statement
David Gibson [Wed, 27 Jan 2016 13:14:43 +0000 (00:14 +1100)]
idtree: Fix misindented statement

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoidtree: Fix comparison is always false warning
David Gibson [Wed, 27 Jan 2016 13:14:19 +0000 (00:14 +1100)]
idtree: Fix comparison is always false warning

idtree.c:146 triggers a "comparison is always false" warning on some
compiler configurations, since the 'id' variable is unsigned.

Elsewhere in the module ids seem to be represented by (signed) ints, so
use the same convention here, suppressing the warning and also maybe being
more correct in other ways.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agohtable: Mark functions constructed by HTABLE_DEFINE_TYPE as UNNEEDED
David Gibson [Wed, 27 Jan 2016 13:06:02 +0000 (00:06 +1100)]
htable: Mark functions constructed by HTABLE_DEFINE_TYPE as UNNEEDED

The HTABLE_DEFINE_TYPE macro builds a type-specific hash table by
constructing a bunch of simple wrapper functions.  The user of the hash
table may not end up using all of these.  With gcc the fact that the
functions are inline stops an warnings about unused functions, but that's
not the case with clang.

Suppress these warnings by marking all the constructed functions except
for name##_add() as UNNEEDED (using the macro from ccan/compiler).  _add
is left alone on the grounds that a hash table you never add anything to
isn't much use, so this will help you to spot an entirely redundant
HTABLE_DEFINE_TYPE invocation.  *_init() would be a more obvious choice,
except that there is both *_init() and *_init_sized() and you only need
to use one of these.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agostrmap: Convert to using TCON_WRAP() instead of plain TCON()
David Gibson [Wed, 27 Jan 2016 12:52:12 +0000 (23:52 +1100)]
strmap: Convert to using TCON_WRAP() instead of plain TCON()

The usual way of construction strmap objects is to use the STRMAP_MEMBERS()
macro which expands to both a raw strmap structure and a tcon type canary.
However, the tcon type canary involves a flexible array member which means
that in standard C99 STRMAP_MEMBERS() must appear only at the end of a
structure definition.  But worse, that structure can then only appear at
the end of any other structure it is included in, which is pretty
inconvenient for the intended purpose of creating type specific strmaps.

gcc extensions allow this to work (somehow), but clang complains loudly
about it.

The tcon module already includes the TCON_WRAP() mechanism, which already
provides this same sort of type-specific definitions in a more general way.
So convert strmap (and its users) to that approach.

This removes STRMAP_MEMBERS() entirely, breaking compatibility.  I'm hoping
strmap is used in few enough places that we can get away with that.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoaltstack: disable valgrind.
Rusty Russell [Fri, 29 Jan 2016 05:56:12 +0000 (16:26 +1030)]
altstack: disable valgrind.

It does not seem to respect stack games!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoaltstack: New module
Dan Good [Tue, 26 Jan 2016 22:18:43 +0000 (22:18 +0000)]
altstack: New module

altstack - run a function with a dedicated stack, and then release the memory

Signed-off-by: Dan Good <dan@dancancode.com>
4 years agoaga: Annotate unused return values
David Gibson [Wed, 27 Jan 2016 12:11:31 +0000 (23:11 +1100)]
aga: Annotate unused return values

bfs_dequeue() and dfs_pop() discard the return values of lqueue_dequeue()
and lstack_pop() respectively.  This is correct, but causes warnings in
some compiler configurations (including the one currently used by
travis-ci.org).

Use the cast-to-void idiom to tell the compiler this is intentional.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoccanlint: Remove unused variable
David Gibson [Wed, 27 Jan 2016 12:06:05 +0000 (23:06 +1100)]
ccanlint: Remove unused variable

The 'rest' variable in examples_run.c:find_expect() was unused.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoconfigurator: Clarify empty if
David Gibson [Tue, 26 Jan 2016 11:28:41 +0000 (22:28 +1100)]
configurator: Clarify empty if

configurator.c contains an if with an empty statement on the same line as
the condition.  This is very easy to misread, and also causes a warning
from clang, so move the ; onto the next line.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years ago.travis.yml: Add clang builds to Travis
David Gibson [Tue, 26 Jan 2016 11:27:28 +0000 (22:27 +1100)]
.travis.yml: Add clang builds to Travis

Still not exactly a great coverage of different C compilers, but clearly
better than *just* gcc.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agorszshm: New module
Dan Good [Tue, 19 Jan 2016 06:04:02 +0000 (06:04 +0000)]
rszshm: New module

rszshm - resizable pointer-safe shared memory

Signed-off-by: Dan Good <dan@dancancode.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agotal/str: fix infinite loop of tal_fmt() with empty string.
Rusty Russell [Tue, 19 Jan 2016 06:36:55 +0000 (17:06 +1030)]
tal/str: fix infinite loop of tal_fmt() with empty string.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agopipecmd: add stderr fd arg.
Rusty Russell [Tue, 19 Jan 2016 03:45:25 +0000 (14:15 +1030)]
pipecmd: add stderr fd arg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agopipecmd: fix fd leak.
Rusty Russell [Tue, 19 Jan 2016 03:13:07 +0000 (13:43 +1030)]
pipecmd: fix fd leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agoio: io_time_override to insert fake times.
Rusty Russell [Mon, 18 Jan 2016 19:47:13 +0000 (06:17 +1030)]
io: io_time_override to insert fake times.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
4 years agodeque: check HAVE_STATEMENT_EXPR, tweaks from feedback
Dan Good [Tue, 5 Jan 2016 01:07:19 +0000 (01:07 +0000)]
deque: check HAVE_STATEMENT_EXPR, tweaks from feedback

Thanks to the detailed feedback from David Gibson, I made the
following improvements:

* add missing includes
* check for statement expression support, give an error if absent
* ccanlint directive to skip "without features" steps
* add license ref to top of source files
* rename run1.c test to api1.c

Signed-off-by: Dan Good <dan@dancancode.com>
4 years agodeque: New module
Dan Good [Wed, 23 Dec 2015 12:11:44 +0000 (12:11 +0000)]
deque: New module

deque - type-preserving resizing circular deque

Signed-off-by: Dan Good <dan@dancancode.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
4 years agoAdd file describing Signed-off-by and guiding new module authors.
Rusty Russell [Mon, 14 Dec 2015 03:37:42 +0000 (14:07 +1030)]
Add file describing Signed-off-by and guiding new module authors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>