discover/grub2: Allow both delimiters and EOLs after a "then" reserved word

Putting an if-statement on one line should be possible:

 if true; then echo "true"; else echo "false"; fi

To do this, we'll need to allow both EOLs and TOKEN_DELIM after then and
else.

Signed-off-by: Jeremy Kerr <jk@ozlabs.org>