Kevin Locke [Thu, 29 Sep 2016 00:44:43 +0000 (18:44 -0600)]
configurator: Inline err.h functions
On systems where err.h is not provided (e.g. MSVC) configurator must
provide its own definition. The err module can not be used by
configurator due to its dependency on config.h, so the relevant source
is copied into configurator with the minimum changes necessary.
Changes since v2:
- Use the CCAN err module sources instead of musl libc sources to avoid
introducing another implementation of these functions.
- Prefix err.h functions with "c12r_" to avoid name conflicts and
"static declaration follows non-static" errors.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Kevin Locke [Thu, 29 Sep 2016 00:44:42 +0000 (18:44 -0600)]
configurator: Reimplement run using popen
Rather than using fork+pipe+system+waitpid, most of which are only
available on POSIX-like systems, use popen which is also available on
Windows (under the name _popen).
Changes since v1:
- Create fread_noeintr to avoid EINTR in fread without reading any data.
- Handle short reads from fread. This can happen with non-conformant
libc or if EINTR occurs after reading some data.
- Define _POSIX_C_SOURCE for popen/pclose with strict implementations
which require it (e.g. gcc with -std=c11).
Changes since v2:
- Revert fread_noeintr and short read changes in v1 as unnecessary.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Kevin Locke [Fri, 23 Sep 2016 03:33:13 +0000 (21:33 -0600)]
configurator: Fix warning in HAVE_FOR_LOOP_DECLARATION
Visual C++ prints "warning C4702: unreachable code" due to the return
statement after the for loop which is never reached. Fix this by
setting a variable returned by a single return statement at the end.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Kevin Locke [Fri, 23 Sep 2016 03:33:10 +0000 (21:33 -0600)]
configurator: Fix compiler warning with compare
Visual Studio prints warning C4706 "assignment within conditional
expression" when there is an assignment without a comparison in a
conditional expression. Therefore, to silence the warning, add an
explicit comparison.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Kevin Locke [Fri, 23 Sep 2016 03:33:08 +0000 (21:33 -0600)]
configurator: Mark non-Windows tests MAY_NOT_COMPILE
Several of the EXECUTABLE tests depend on headers not available on
Windows. Mark these tests MAY_NOT_COMPILE to handle this.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Kevin Locke [Fri, 23 Sep 2016 03:33:07 +0000 (21:33 -0600)]
configurator: Use native directory separator
Although Windows APIs generally permit "/" or "\\" for directory
separators in paths, cmd.exe does not recognize "./" when invoking
executables using a relative path and prints the following error:
'.' is not recognized as an internal or external command,
operable program or batch file.
Therefore, use "\\" when invoking tests on Windows.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Kevin Locke [Fri, 23 Sep 2016 03:33:04 +0000 (21:33 -0600)]
configurator: Replace unlink with remove
Although Windows provides unlink, using it adds complication due to the
lack of unistd.h which must be included to define the function on POSIX
systems. Instead, use remove, which C89 requires to be in stdio.h.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
This is the same as list_empty but without the debug checks. This is
useful when wanting to check for an empty list without locks held,
potentially racing with addition/removal, which can be a valid thing
to do under some circumstances.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell [Wed, 7 Sep 2016 04:27:01 +0000 (13:57 +0930)]
tal: make tal_next() only return immediate children.
I tried to use it and got this wrong: moreover, I wanted to control
topology, which requires nested iteration, and skip children
of a node which I knew was changing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Jon Griffiths [Sun, 28 Aug 2016 23:43:39 +0000 (11:43 +1200)]
Add a SHA512 implementation
This largely follows the SHA256 style. I've named Rusty as the maintainer.
Currently the functions to add data of various sizes/endianness have not
been implemented: There are no public test vectors for these cases and
I believe most use cases are working on byte buffers. They can be added
later if desired.
The openssl implementation has been tested on x86-64, while the inbuilt
version has been tested on 32/64 bit, little/big endian boxes.
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Rusty Russell [Tue, 30 Aug 2016 00:35:06 +0000 (10:05 +0930)]
tools/ccanlint: test_depends_accurate: catch #include of other ccan C files.
Jon sent a test patch for sha256 which "#include <ccan/str/hex/hex.c>"
without adding str/hex to testdepends. It "worked" (it wouldn't have
linked with "hex.h"), but ccanlint didn't spot the dependency.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Jon Griffiths [Wed, 24 Aug 2016 06:44:12 +0000 (18:44 +1200)]
strmap: Set the license for strmap to CC0
This matches the license in _info and the source. It also means every
module now has a LICENSE file and so anyone happening to rewrite the
ccan makefiles can use wildcard to locate modules rather than listing
them by hand.
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Jon Griffiths [Mon, 29 Aug 2016 01:10:29 +0000 (13:10 +1200)]
Ensure config.h is created before the main travis builds
The current makefiles are not parallel safe and reliably fail on the
clang build. The real fix is to rewrite the makefiles, but in the
meantime, allow the clang tests to pass.
Jon Griffiths [Sun, 28 Aug 2016 23:52:44 +0000 (11:52 +1200)]
sha256: Simplify test
We can use the iteration count in the test case to determine
whether a single call is required. This simplifies the code and
also means that we don't overstate the actual number of tests
performed by a factor of 2. Simplify a couple of expressions
while we are changing this.
Jon Griffiths [Mon, 22 Aug 2016 14:04:06 +0000 (02:04 +1200)]
tlist: Place tlists last in structures
TCON suggests placing _tcon members last in structs. Placing variable
sized structs anywhere but last is apparently a gcc extension that
gives warnings under clang.
This applies to tlists because they use TCON internally. Update the
docs and examples to place tlists last and so compile without clang
warnings.
There are other places where this occurs; they will be dealt with
separately.
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Jon Griffiths [Mon, 30 May 2016 12:39:10 +0000 (00:39 +1200)]
hex: Simplify hex_encode
The documentation for hex_encode indicates that it returns simply true or
false. The old implementation was returning the written length on success,
cast to boolean. This will elicit a warning under MSVC.
On further examination, there is no need to check/modify the length inside
the loop, since we can check it once before starting. As a result the code
can be simplified a bit.
A side affect of this change is that nothing will be written at all if the
length is incorrect, vs the previous code writing characters until the length
available is exhausted. I prefer the new semantics but YMMV.
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Jon Griffiths [Tue, 22 Mar 2016 22:54:50 +0000 (11:54 +1300)]
sha256: Move 'bytes' to the end of sha256_ctx and make it a size_t
The code already assigns to/from bytes as a size_t, so make it
official (and better on platforms with a 32 bit size_t).
Moving bytes makes it act as a canary in the event that there is a rogue
write/off by one somewhere - since it ends up in the hash we are
more likely to detect this should we corrupt it. This also makes the
working buffer better aligned which can't hurt.
Also, initialise the buffer to zero while we are changing the initialisation
macro anyway. It costs little compared to the hashing overhead, should be
optimised away if redundant in most cases, and it removes a warning from both
gcc and clang about unititialised struct members.
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Jon Griffiths [Tue, 22 Mar 2016 22:49:48 +0000 (11:49 +1300)]
sha256: Make our u32 and u8 fields the same size
These are just aliases to a buffer: its customary for these to
have the same size, and makes sizeof() consistent in case anyone
decides to use the members instead of the containing union.
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
David Gibson [Fri, 3 Jun 2016 08:42:03 +0000 (18:42 +1000)]
altstack: Don't log internal calls in test cases
altstack/test/run.c uses some hairy macros to intercept the standard
library functions that altstack uses. This has two purposes: 1) to
conditionally cause those functions to fail, and thereby test altstack's
error paths, and 2) log which of the library functions was called in each
testcase.
The second function isn't actually useful - for the purposes of testing the
module, we want to check the actual behaviour, not what calls it made in
what order to accomplish it. Explicitly checking the calls makes it much
harder to change altstack's implementation without breaking the tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Fri, 3 Jun 2016 08:42:02 +0000 (18:42 +1000)]
altstack: Don't use 0 pointer literals
In a number of places the altstack module uses a literal '0' for pointer
values. That's correct C, but doesn't make it obvious on a quick read
whether values are integers or pointers. This patch changes those cases
to use the NULL define instead.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Fri, 3 Jun 2016 08:42:01 +0000 (18:42 +1000)]
altstack: Use ptrint instead of bare casts
Functions invoked with altstack take a void * parameter. However, the
test program wants to pass an integer, and so uses the trick of casting
the integer values to (void *) and back again.
The ptrint() module handles exactly this case in a more portable and
(somewhat) typesafe way, so use that instead.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Fri, 3 Jun 2016 08:42:00 +0000 (18:42 +1000)]
altstack: Restore alternate signal stack state
altstack relies on catching a SIGSEGV caused when overrunning the stack.
This means that the SEGV handler itself can't use the already overflowed
stack, and so we use sigaltstack() to assign the signal handler a different
stack. On completion, altstack() clears the alternate signal stack.
However, it's possible that the calling program could be using
sigaltstack() for its own reasons, so it's more correct to restore the
sigaltstack() state to that from the beginning of the altstack() call.
This patch implements this behaviour.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Fri, 3 Jun 2016 08:41:59 +0000 (18:41 +1000)]
altstack: Consolidate thread-local variables
altstack uses a number of __thread variables to track internal state. This
allows altstack to be thread-safe, although it's still not re-entrant.
This patch gathers all these variables into a single per-thread state
structure. This makes it easy to see at a glance what the whole of the
required state is, and thereby easier to reason about correctness of
changes to the implementation.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sat, 4 Jun 2016 10:21:33 +0000 (20:21 +1000)]
tlist: Add tlist_next() and tlist_prev() functions
An odd omission from the tlist module is basic tlist_next() and
tlist_prev() macros matching list_next() and list_prev() in the basic
list module. This adds them.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
David Gibson [Fri, 3 Jun 2016 10:07:10 +0000 (20:07 +1000)]
agar: Add static graph initializer
Sometimes it's not convenient to initialize an agar graph at runtime with
agar_init_graph(). This adds an AGAR_INIT_GRAPH() macro to do the same
thing as a static initializer.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Mon, 9 May 2016 19:44:47 +0000 (20:44 +0100)]
Build info files
All modules have a _info file (a C file, despite the lack of extension)
giving metadata for the module. The Makefiles have a rule to build these..
but it's broken (missing an include directive).
This patch fixes the rule, and builds the info binaries for all modules by
default. This is a useful check and also useful for manually inspecting
a module's metadata.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Thu, 5 May 2016 15:40:48 +0000 (16:40 +0100)]
Automatically determine which modules have source
Currently, build of ccan is controlled by two Make variables: MODS_WITH_SRC
and MODS_NO_SRC which list modules containing .c files, and modules with
only .h files respectively.
When adding new modules this is fiddly to get right, and there are a number
of modules already listed in the wrong variable. There's also some
redundant logic in the DIRS variable to again filter out modules without
source.
This simplifies things by having a single manually updated MODS variable
listing every module, and determining MODS_WITH_SOURCE programmatically.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Tue, 16 Feb 2016 12:35:41 +0000 (23:35 +1100)]
Correctly include dependencies for nested modules
Currently we pull auto-generated dependencies into the Makefile with
include ccan/*/*.d. That will omit any .d files from nested modules,
meaning things might not be correctly rebuilt there.
Correct this by using the list of modules instead of a 1-level wildcard.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Tue, 16 Feb 2016 12:32:34 +0000 (23:32 +1100)]
Exclude system headers from .d files
We currently generated .d dependency files with the -MD option to cc. That
includes system header files in the dependencies, which isn't often useful
and makes the .d more complicated than necessary.
This changes to -MMD which excludes system headers.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Tue, 16 Feb 2016 12:24:05 +0000 (23:24 +1100)]
Clean up use of 'rm' in Makefiles
Most of the ccan Makefiles use $(RM) to remove files. However, 'rm' is
traditionally considered one of the few shell tools which can be used in
Makefiles without indirecting via a variable.
rm is also typically invoked with -f in Makefiles, so that it doesn't cause
errors if the files don't exist (because they haven't been built). A
number of instances in ccan were missing this.
This corrects these warts.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Tue, 16 Feb 2016 11:53:50 +0000 (22:53 +1100)]
Add missing files to make clean
At present, "make clean" will not remove the module-Makefile files for
non-top-level modules. It also won't remove the generated config.h.
Correct those errors.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
should_grp_score_cos(x,y) wasn't holding up its part of the bargain, so
real data was used to generate a fudge curve to bring
should_grp_score_cos(x,y) results into the same space. Really this is a
terrible hack and the problem needs more thought. Evaluation of
should_grp_score_cos(x,y)'s performance benefit (given the relaxation of
the filter under the fudge curve) is sorely needed.
Andrew Jeffery [Sat, 20 Feb 2016 10:49:53 +0000 (21:19 +1030)]
strgrp: Use angular similarity for distance metric properties
Distance metrics allow us to compare similarity results, however
applying the change leads to test suite breakage as we no longer satisfy
the requirement that each filter's score is at most as large as that of
the previous filter^. As such, also stop ccanlint from executing the
tests that are known to fail until we work around the problem.
^ This is a problem that has existed since the introduction of the
cosine similarity filter, it just wasn't detected by the test suite.
Andrew Jeffery [Sat, 20 Feb 2016 11:03:04 +0000 (21:33 +1030)]
strgrp: Use ratio of hypotenuse for consistent comparisons
Ensure comparing filter results is sensible by using a consistent
calculation. Note that the cosine similarity measurement doesn't yet
conform and this can give spurious results that are not detected by the
test suite.
Rusty Russell [Tue, 8 Mar 2016 05:38:36 +0000 (16:08 +1030)]
shachain: clarify design in terms of binary tree, reverse indexes.
Olaoluwa Osuntokun came up with an alternative which used binary trees;
that's a much better way to explain it, so do that in design.txt and
update the implementation to work the same way.
Anthony Towns pointed out that the numbering is the reverse of the normal
hash chaining descriptions, so fix that too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
David Gibson [Fri, 12 Feb 2016 12:49:49 +0000 (23:49 +1100)]
generator: Allow generators to take arguments
Using some serious macro magic, this patch extends generators to allow
them to take arbitrary arguments. The arguments are marshalled into a
structure placed at the far end of the generator's stack when it is
created. Then, they're unmarshalled back into C parameters when we first
context switch into the generator.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Sat, 19 Jul 2014 07:00:15 +0000 (17:00 +1000)]
configurator: Add test for ucontext.h
This adds a new HAVE_UCONTEXT define, which indicates that ucontext.h
is present, and more-or-less works.
It also adds HAVE_POINTER_SAFE_MAKECONTEXT, which indicates whether
pointer valued arguments can be passed through the varargs parameters
to makecontext().
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Mon, 15 Feb 2016 11:49:54 +0000 (22:49 +1100)]
altstack: Clarify checking macros
The chkfail() and chkok() macros in altstack's test program are pretty
difficult to read. More importantly, though, they do all their tests with
one big ok1(). That means if the test fails, you get no indication which
of the checks was actually wrong, making debugging harder.
This reworks the macros into a more verbose form that's easier to read,
and splits them into multiple ok1() tests to make failures more explicit.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
David Gibson [Mon, 15 Feb 2016 11:47:36 +0000 (22:47 +1100)]
altstack: Declare memory clobbers
altstack includes a couple of inline asm blocks with x86 push and pop
instructions. These instructions will access memory (the stack), but
that's not declared in inline asm statement. We seem to be getting away
with it, but in theory that could allow the compiler to re-order accesses
to local variables across the asm block. Since those blocks change the
location of the stack, that could be very bad.
Adding a "memory" clobber should prevent this (effectively making the asm
blocks a compiler memory barrier).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
projects / ccan / log