I'm running the BATS tests manually once in a while, and
catching several problems each week that make it past
the rest of CI. Since the BATS tests run at RPM gating
time, we need to catch problems earlier. Try running
the tests from Cirrus.
Tests will be skipped on Ubuntu due to a too-ancient
version of coreutils (8.28; the 'timeout -v' we use
requires 8.29).
Tests are run *after* integration tests, even though
these take three minutes and would be nice to have
fail quickly, because running before causes bizarre
CI failures. Shrug.
UPDATE: also fix run test, broken by #3311.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Various small fixes to get BATS tests working again.
Split from #2947 because that one keeps getting stalled,
and I'm hoping these separate changes get approved.
I consider these changes urgent because RHEL8 gating
tests are failing, and will fail even more if/when #2272
gets picked up and packaged for RHEL8, and I consider
it important to have clean passing tests for RHEL8.
* info test: 'insecure registries' is gone. A recent
commit (d1a7378aa) changed the format of 'podman info',
removing the 'insecure registries' key. Deal with it.
* info test: remove check for .host.{Conmon,OCIRuntime}.package;
the value on f28 and f29 is 'Unknown' (instead of an NVR).
We can live without this check.
* 'load' test: skip when running in CI, because stdin
is not a tty.
* container restore: fix arg processing. #2272 broke argument
processing: 'podman container restore', with no args, should
exit with 'argument required' error. Root cause is that the
new --import option takes the place of an argument, so the
checkAllAndLatest() call had to be changed to not exit on error.
Workaround is (sigh) to copy/paste the skipped checkAllAndLatest()
code, with minor tweaks to accommodate --import.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- podman-remote:
- enable log, run and build tests, they're working now
- well, except build + rootless. Skip that one.
- add explanation of why info test is skipped
- Giuseppe's permission test:
- validate GraphRoot and RunRoot values
- add verbose logging, to enable seeing full directory tree
permissions on error
Signed-off-by: Ed Santiago <santiago@redhat.com>
Various problems, one of which was causing the test to fail
completely (otherwise I wouldn't have caught the others):
- option is --uidmap, not --uidmapping
- run_podman cannot be piped (| grep /sys/kernel). That's
an unfortunate limitation of BATS. Any invocation of 'run'
saves results to $output, which then has to be tested
in a separate step.
- do so, using 'run' and 'grep' and 'is' to produce
readable messages on failure
- remove "$expected_rc", that looks like a copy/paste bug
from a few lines above.
Skip entire test if rootless. (The one without --net=host
passes, but it also passes with older podman as both root
and rootless. I don't think it's actually testing anything,
but agree with leaving it in to catch weird regressions).
We really need to get these tests running in CI.
Signed-off-by: Ed Santiago <santiago@redhat.com>
when we run in a user namespace, there are cases where we have not
enough privileges to mount a fresh sysfs on /sys. To circumvent this
limitation, we rbind /sys from the host. This carries inside of the
container also some mounts we probably don't want to. We are also
limited by the kernel to use rbind instead of bind, as allowing a bind
would uncover paths that were not previously visible.
This is a slimmed down version of the intermediate mount namespace
logic we had before, where we only set /sys to slave, so the umounts
done to the storage by the cleanup process are propagated back to the
host. We also don't setup any new directory, so there is no
additional cleanup to do.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
podman-remote now supports rm! That's what we needed to start
running BATS tests.
Although most tests don't actually work, some do, and maybe
the rest will start working over time. For now, disable them.
The only significant difference found is that podman-remote
strips fractional seconds from timestamps in JSON output.
Probably not something worth caring about.
Signed-off-by: Ed Santiago <santiago@redhat.com>
New:
- podman exec
- podman load (requires #2674)
- CLI parsing (regression test for #2574)
Improved:
- help: test "podman NoSuchCommand", and subcommands
- help: test "podman cmd" without required args
- pod: start with --infra=false; this allows running rootless
- log: also run 'logs' after container is run
- log: test -f with two containers
Also, use helpful descriptions for skip_if_rootless
Tested on f29, root and rootless. As soon as podman-remote
supports rm, I'll start testing that too.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- document a recommended convention for fail-fast tests
- document the requirement for jq. (And, add a fail-fast
test for its presence; remove the duplicated checks
in subtests)
- add further sanity checks to 'help' test. Add missing
documentation. Remove a no-longer-needed workaround for
usage-message bug fixed in #2486
- add a documented TEMPLATE
- and, since we're at 1.1, enable 'Remote API' check in
version test
- better diagnostics in setup/teardown; add vim filetype hint;
better formatting of actual-vs-expect errors
- new pod-top, logs, build tests
- improve error messages
- add $IMAGE alias for ridiculous $PODMAN_TEST_IMAGE_FQN
- final cleanup, in prep for merge
Signed-off-by: Ed Santiago <santiago@redhat.com>
Initial attempt at writing a framework for podman system tests.
The idea is to define a useful set of primitives that will
make it easy to write actual tests and to interpret results
of failing ones.
This is a proof-of-concept right now; only a small number of
tests, by no means comprehensive. I am requesting review in
order to find showstopper problems: reasons why this approach
cannot work. Should there be none, we can work toward running
these as gating tests for Fedora and RHEL8.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The tests can be filter by --focus and --skip to fit different test
target. Also be able to set global options and cmd options by export
it to ENV to fit different test matrix.
Signed-off-by: Yiqiao Pu <ypu@redhat.com>
