Implement review feedback

- document a recommended convention for fail-fast tests

- document the requirement for jq. (And, add a fail-fast
  test for its presence; remove the duplicated checks
  in subtests)

- add further sanity checks to 'help' test. Add missing
  documentation. Remove a no-longer-needed workaround for
  usage-message bug fixed in #2486

- add a documented TEMPLATE

- and, since we're at 1.1, enable 'Remote API' check in
  version test

- better diagnostics in setup/teardown; add vim filetype hint;
  better formatting of actual-vs-expect errors

- new pod-top, logs, build tests

- improve error messages

- add $IMAGE alias for ridiculous $PODMAN_TEST_IMAGE_FQN

- final cleanup, in prep for merge

Signed-off-by: Ed Santiago <santiago@redhat.com>
This commit is contained in:
Ed Santiago
2019-03-05 09:58:30 -07:00
parent 681eae9bcc
commit 589248d2f3
15 changed files with 343 additions and 55 deletions

View File

@ -18,17 +18,28 @@ sometimes needs to massage the returned values; `015-run.bats` offers
examples of how to deal with the more typical such issues.
* `run_podman` - runs command defined in `$PODMAN` (default: 'podman'
but could also be 'podman-remote'), with a timeout. Checks its exit status.
but could also be './bin/podman' or 'podman-remote'), with a timeout.
Checks its exit status.
* `is` - compare actual vs expected output. Emits a useful diagnostic
on failure.
* `die` - output a properly-formatted message to stderr, and fail test
* `skip_if_rootless` - if rootless, skip this test with a helpful message.
* `random_string` - returns a pseudorandom alphanumeric string
Test files are of the form `NNN-name.bats` where NNN is a three-digit
number. Please preserve this convention, it simplifies viewing the
directory and understanding test order. Most of the time it's not
important but `00x` should be reserved for the times when it matters.
directory and understanding test order. In particular, `00x` tests
should be reserved for a first-pass fail-fast subset of tests:
bats test/system/00*.bats || exit 1
bats test/system
...the goal being to provide quick feedback on catastrophic failures
without having to wait for the entire test suite.
Analyzing test failures
@ -56,6 +67,12 @@ set `PODMAN_TEST_DEBUG="funcname"` where `funcname` is the name of
the function or perhaps just a substring.
Requirements
============
The `jq` tool is needed for parsing JSON output.
Further Details
===============