discover/device-handler: Fix use-after-free when unmounting

We need to do the rmdir after clearing mount_path.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Don't call config_set_autoboot before config_init

Change f611bde3 shifted the config_init until after process_init, as we
need to run processes during config init. We also needed to move the
config_set_autoboot invocation too.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Fix uninitialised yylineno

Valgrind reports an invalid usage of an uninitialised var in yyerror,
causing testcase failures.

Initialise lineno before we start parsing.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement save_env command

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

test/parser: Add check_file_contents

We want to test grub2's save_env command, which requires a new function
to check the contents of a file.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement load_env

Use the new parser_request_file API to access the GRUB environment
block.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: strdup strings used in the environment

Use a copy of the name & value pairs that we pass to the environment, as
the data loaded from load_env will be talloc_free-ed.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add default prefix

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Add device_{request,release}_write

Add a pair of functions to the parser API to allow write access to the
underlying device. We'll use this in the GRUB2 parser to implement
environment persistence.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Add parser_request_file

Add a function to allow parsers to access files on a local device.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Add test_data member to struct discover_context

We have parsers passing discover_contexts around, which we want to hook
into the test framework. Add a void * member, which the test code can
use to reference the test.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Don't continue discovery if mount fails

Check the return value from mount_device and abort the discover on
failure.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Allow for already-mounted devices

When we start the discover server, we may find that devices are already
mounted. In this case, mount_device will fail, and we'll abort the
parse.

This change uses /proc/self/mounts to check if new devices are already
mounted, and uses the existing mount point.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Clean up error-handling for grub2 parser & lexer

Rather than printf() & exit(), use the pb logging functions and abort
the parse.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

test/parser: Enable logging to stdout

We may as well show the logs while testing.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

tests/urls: Run URL tests under valgrind

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/resource: create_url_resource should take ownership of url

The URL needs to share the lifetime of resource, so talloc_steal it in
create_url_resource.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

lib/url: talloc from new URL in pb_url_copy

We're ending up with members of new_url being allocated from the old
URL's context. We should be tallocing from the new_url instead.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/paths: Check local file URLs in load_url

Make the error case for local files the same as for remotes.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

lib/system: Don't modify dir parameter to pb_rmdir_recursive

We're modifying dir rather than cur (the local copy of dir), so the
caller-provided (const!) string is no longer const.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Log boot status updates

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Download resources in parallel

Now that we can download asynchronously, we can download boot resources
in parallel. This also means we don't need a state machine; we're either
not done (loads are still pending) or done.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Add struct load_url_result and load_task

Currently, load_url and load_url_async return a filename parameter and a
tempfile flag (indicating whether the file needs to be cleaned after
use).

Instead, encapsulate this data in a struct load_url_result, which the
caller (and async callbacks) can read the status, filename and clean
parameters.

For internal use in load_url and helpers, we add a struct load_task to
hold a pointer to the load_url_result and async data.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Don't depend on tftp failure for type detection

Rather than always trying both TFTP client types, do a runtime detection
on first invocation. This can be fixed at build-time with
--with-tftp=TYPE.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Store clean_* in the boot task

... otherwise we won't clean up local files from earlier boot stages.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Check for failed loads in boot()

Use the status argument to the URL load callback to check for load
failures.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: load_url_async callback should take an int status

We don't need a pointer here, just the status value.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Implement asynchronous boot() process

The current boot process handled by boot() function is implemented such
that it blocks on the boot resources being downloaded which blocks
the discover process until the download completes. This patch
re-implements the boot() function using callback mechanism and
asynchronous notifications so that boot() returns to continue the
discover process while the download is in progress.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>
Signed-off-by: Neelesh Gupta <neelegup@linux.vnet.ibm.com>

discover: Add an asynchronous version of load_url

A new load_url_async() function handles retrieving the resources by
forking an external process which currently blocks the parent waiting
for the process to complete. This patch modifies the load_*() functions
to handle it asynchronously by providing the exit callback and returning
without waiting for the child process to exit.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>
Signed-off-by: Neelesh Gupta <neelegup@linux.vnet.ibm.com>

discover/grub2: Reimplement default options

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Check for devices with duplicate serial properties

If we have multipath devices present in the system, we'll end up with
duplicate mounts, parse results and boot options. This change adds a
check to see if we've encountered a device with this serial number
previously.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

pb-udhcpc: use plain kernel names for dhcp events

No need to prefix with /net/

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Fix uninitialised op variable

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

test/parser: Add test-grub2-single-line-if test

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Support var=value assignments

We want to allow assignments outside of the 'set' builtin.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

test/parser: Allow double-quotes in embedded config

We need to escape any double-quotes.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add true & false builtins

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

test/parser: Allow checks for NULL boot_args

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

test/parser: Add variable expansion in Ubuntu conf file test

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2/lexer: Require 1+ spaces for a TOKEN_DELIM

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add linux16 command

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Allow both delimiters and EOLs after a "then" reserved word

Putting an if-statement on one line should be possible:

 if true; then echo "true"; else echo "false"; fi

To do this, we'll need to allow both EOLs and TOKEN_DELIM after then and
else.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

test/parser: Fix spacing in grub2-f18-ppc64 test

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement test builtin

A faily simple implementation now, and could do with some testing...

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add nops for unnecessary builtins

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement statement_block_execute

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: unknown commands should fail

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Populate option ids

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Allow EOF as a statement terminator

Files may not have a trailing newline.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add print_token

For better trace output.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: fix comment handling

Remove '#' from the list of word chars. Also, don't need the '$' char in
the comment regex

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement 'elif'

Rather than just having one conditional in an if statement, we use a
list of conditionals instead. This allows us to implement elif.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Create 'conditional' statements

Rather than holding the condition and conditional-statements in struct
grub2_statment_if, create a new conditional type that contains these. We
can then use this to implement elif statements.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Allow empty statements

Re-arrange the script/statements parse rules to allow empty statements.
We also move the EOL token out of the 'statement' definition, and use
it to separate 'statements'.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement submenu

Just recurse into the submenu items, as we flatten everything into the
one menu.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add function support

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Improve error handling

Only run the script if the parse succeeded, and and improve the
error reporting by enabling verbose errors and line numbers.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement 'else' blocks

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement 'search' command

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement 'linux' & 'initrd' commands

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Function infrastructure improvements

For user-defined functions, we'll need a data pointer to the function's
execution callback. Add this as a void *, and change references from
'command' to 'function'.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add option state in menuentry processing

This will allow menuentry-specific commands to populate boot option
data.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add a reference to the context from grub2_script

We'll need the context to add boot options.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Hook up flex/bison parser to discover server

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Move parser-api definitions to parser.y

No need for a separate file here.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement multiple-arv variable splitting

If we expand a variable containing word-delimiter chars, we need to
create new argv items.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add initial command execution

.. with a simple 'set' command to update the environment

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add menuentry execution

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Fix word append

The parser was setting word->next on the top-level word, resulting in
word chains of a maximum length of 2. Instead, use a last pointer to
always append to the last word.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Handle var tokens in lexer

Rather than post-processing to expand variables, use the lexer to
identify variable tokens as a type of grub2_word.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Remove debug printfs

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Initial environment handling

A simple linked-list implementation of string pairs.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add initial execution code

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Fix statement reduce rule

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add var expansion code

Still todo: splitting.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add initial script infrastructure

Now that we can parse scripts, we want some infrastructure for
execution.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Use subclass-style structs for AST

Use embedded 'struct grub2_statement' data to create a subclass-style
syntax tree of statements.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement statement blocks

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement reduce rules

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Add structures & API for parser

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Implement parser

Add our simple parser to handle a grub2 script. Since we're not building
a parse tree at the moment (we have no reduce rules), we just have a
simple word-based %union for our tokens.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/grub2: Initial grub2 lexer

Add a simple flex lexer to tokenise our grub2 config files.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

lib: Add array-size header

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Add booting status message

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Allow printf formatting in update_status

It'd be nice to include arbitrary format strings here, so do a
taloc_vasprintf in update_status.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Fix log message with no newline

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: clear default_timeout waiter on timeout

We don't want cancel_default to try to remove an already-removed waiter.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Don't register multiple boot_timeout waiters

Rather than adding another timeout waiter, just override
handler->default_boot_option and extend the timeout a little.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

test/parser: check for multiple default options

Update check_boot_option_count to check if a parser returned more than
one boot option with is_default set.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/pxe: Implement default options

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Implement device priorities

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: register network devices with the device handler

We'd like to correlate incoming network boot options with a device, so
register the interface with the device hander.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/network: Manage network->interfaces with {add,remove}_interface

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/network: ignore devices with zero MTU

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Remove unnecessary event passing

Currently, we pass "events" between the udev, user-event and
device-handler layers. These events all get sent through
device_handler_event, then de-multiplexed to an appropriate handler,
depending on their source.

Instead, just export relevant device_handler functions, and have the
(old) event sources call these functions directly.

This also means we can include a lot more of the device hander code in
the parser tests.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover/boot: Fix use-after-free in boot()

We need boot_task for kexec_reboot.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

lib/pb-config: Initialise DNS server info

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

lib/log: Cleanup log API

Rather than exposing log internals (through always_flush and
set_stream), do all logging init through pb_log_init(). If pb_log_init()
hasn't been called, pb_log will drop messages.

Also, add a pb_debug() function, specifically for debugging information.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: Consolidate device path, name and ID.

This change cleans up our usage of device path, names and IDs.

Device ID is the kernel name for the device. We also expose this through
lookup_by_name.

Device path is the path to the dev node (ie, always starts with /dev/),
and is only used for mounting.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>

discover: populate udev device types

Now that we have device types, populate from the udev info.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>