There's a whole slew of networking-related flakes whose common
element seems to be improper use of curl. Fix those by:
* add --retry --retry-connrefused; and/or
* add -S ("show errors". Plain -s silences everything!); and/or
* test exit status from curl; and/or
* add wait_for_port after "podman run -d", to avoid races
* log commands, to make debugging easier
Important note: wait_for_port() was not working with rootless
podman ports. Trivial proof:
$ podman run -d --name foo -p 8192:80 \
quay.io/libpod/testimage:20221018 \
/bin/busybox-extras httpd -f -p 80
$ grep :2000 /proc/net/tcp
[no results]
Solution: use ss tool; it seems to handle this just fine.
There may be a better solution.
Oh, also, add -t1 to a podman restart, to shave 18s from test run.
Fixes: #20335 and, I think, a handful of others
Signed-off-by: Ed Santiago <santiago@redhat.com>
When a userns and netns is used we need to let the runtime create the
netns otherwise the netns is not owned by the right userns and thus
the capabilities would not be correct.
The current restart logic tries to reuse the netns which is fine if no
userns is used but when one is used we setup a new netns (which is
correct) but forgot to cleanup the old netns. This resulted in leaked
network namespaces and because no teardown was ever called leaked ipam
assignments, thus a quickly restarting container will run out of ip
space very fast.
Fixes#18615
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Part of RUN-1906.
Followup to #19878 (check stderr in system tests): allow_warnings()
and require_warning() functions to make sure no unexpected messages
fall through the cracks.
Signed-off-by: Ed Santiago <santiago@redhat.com>
With few exceptions, commands that exit 0 should not emit any
messages with level=warning or =error. Let's start enforcing
that in run_podman.
Allow one-off exceptions, typically when we're testing an
actual warning condition (usual case: "podman stop" where it
times out to SIGKILL). Exceptions are specified via:
run_podman 0+w subcommand...
^^^---- or, rarely, 0+e
"0" stands for "expect exit status 0", which is the default
so it's implicit anyway. The +w / +e (or even +we) is the
new part. I have added it to tests where necessary.
And, because life is what it is, add two global exceptions:
- Debian. Because runc has too many flakes.
- kube. Ditto. Kube commands emit lots of nasty error
messages (yes, level=error) that don't seem to affect
results.
Similar to #18442
Signed-off-by: Ed Santiago <santiago@redhat.com>
Compat api for containers/stop should take -1 value
Add support for `podman stop --time -1`
Add support for `podman restart --time -1`
Add support for `podman rm --time -1`
Add support for `podman pod stop --time -1`
Add support for `podman pod rm --time -1`
Add support for `podman volume rm --time -1`
Add support for `podman network rm --time -1`
Fixes: https://github.com/containers/podman/issues/17542
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We do not use any special netns path for the netns=none case, however
callers that inspect that may still wish to join the netns path directly
without extra work to figure out /proc/$pid/ns/net.
Fixes#16716
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
BATS 1.8.0 introduces tags: metadata that can be applied to
a single test or one entire file, then used for filtering
in a test run.
Issue #19299 introduces the possibility of using OpenQA
for podman reverse dependency testing: continuous CI on
all packages that can affect podman, so we don't go two
months with no bodhi builds then get caught by surprise
when systemd or kernel or crun change in ways that break us.
This PR introduces one bats tag, "distro-integration".
The intention is for OpenQA (or other) tests to install
the podman-tests package and run:
bats --filter-tags distro-integration /usr/share/podman/test/system
Goal is to keep the test list short and sweet: we do not
need to test command-line option parsing. We *DO* need to
test interactions with systemd, kernel, nethack, and other
critical components.
Signed-off-by: Ed Santiago <santiago@redhat.com>
To silence my find-obsolete-skips script, remove the '#'
from the following issues in skip messages:
#11784#15013#15025#17433#17436#17456
Also update the messages to reflect the fact that the issues
will never be fixed.
Also remove ubuntu skips: we no longer test ubuntu.
Also remove one buildah skip that is no longer applicable:
Fixes: #17520
Signed-off-by: Ed Santiago <santiago@redhat.com>
We use the name as alias but using the hostname makes also sense and
this is what docker does. We have to keep the short id as well for
docker compat.
While adding some tests I removed some duplicated tests that were
executed twice for nv for no reason.
Fixes#17370
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Several tweaks to see if we can track down #17216, the unlinkat-ebusy
flake:
- teardown(): if a cleanup command fails, display it and its
output to the debug channel. This should never happen, but
it can and does (see #18180, dependent containers). We
need to know about it.
- selinux tests: use unique pod names. This should help when
scanning journal logs.
- many tests: add "-f -t0" to "pod rm"
And, several unrelated changes caught by accident:
- images-commit-with-comment test: was leaving a stray image
behind. Clean it up, and make a few more readability tweaks
- podman-remote-group-add test: add an explicit skip()
when not remote. (Otherwise, test passes cleanly on
podman local, which is misleading)
- lots of container cleanup and/or adding "--rm" to run commands,
to avoid leaving stray containers
Signed-off-by: Ed Santiago <santiago@redhat.com>
When a userns is set we setup the network after the bind mounts, at the
point where resolv.conf is generated we do not yet know the subnet.
Just like the other dns servers for bridge networks we need to add the
ip later in completeNetworkSetup()
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2182052
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
...not CONTAINERS_CONF. At least for most tests.
Nearly every system test currently using CONTAINERS_CONF=tmpfile
should be using CONTAINERS_CONF_OVERRIDE.
Simple reason: runtime (crun/runc), database_backend (bolt/sqlite),
logger, and other important settings from /etc/c.conf are not
usually written into the tmpfile. Those tests, therefore, are
not running podman as configured on the system.
Much more discussion: #15413
This PR is a prerequisite for enabling sqlite system tests. For
the sake of simplicity and sanity, I choose to submit the sqlite
switch as a separate PR once this passes and merges.
Signed-off-by: Ed Santiago <santiago@redhat.com>
...in hopes of fixing a flake with podman-remote. It's still
possible that there's a real problem with logs under remote,
and this will just sweep that under the rug.
Also, fix a nasty-red test warning (add cleanup), refactor
uses of $(pause_image), and improve a few test assertions.
Closes: #17286
Signed-off-by: Ed Santiago <santiago@redhat.com>
Since commit 06241077cc we use the aardvark per container dns
functionality. This means we should only have the aardvark ip in
resolv.conf otherwise the client resolver could skip aardvark, thus
ignoring the special dns option for this container.
Fixes#17499
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Replace existing tab indentations with spaces, and add
a test to CI to prevent new ones from sneaking in.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The main helpers.bash file is rather bloated and it's difficult to
find stuff there. Move networking functions to their own helper
file.
While at it, apply a consistent style, and rearrange logically
related functions into sections.
Suggested-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
One test was using a hardcoded fixed port, with a comment
referring to #10806. That issue seems fixed, so let's
try switching to a pseudorandom open port.
Does not actually fix#16289 but I'm going to close that
anyway, will reopen if it recurs.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Some system tests in `255-auto-update.bats` and `500-networking.bats`
fail under proxy environment.
This PR fixes this problem.
Signed-off-by: Tsubasa Watanabe <w.tsubasa@fujitsu.com>
This Patch will cause podman COMMAND rm --force bogus not fail
This is how Docker works, so Podman should follow this to allow existing
scripts to convert from Docker to Podman.
Fixes: #14612
Oprignal version of this patch came from wufan 1991849113@qq.com
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
...and enable the at-test-time confirmation, the one that
double-checks that if CI requests runc we actually use runc.
This exposed a nasty surprise in our setup: there are steps to
define $OCI_RUNTIME, but that's actually a total fakeout!
OCI_RUNTIME is used only in e2e tests, it has no effect
whatsoever on actual podman itself as invoked via command
line such as in system tests. Solution: use containers.conf
Given how fragile all this runtime stuff is, I've also added
new tests (e2e and system) that will check $CI_DESIRED_RUNTIME.
Image source: https://github.com/containers/automation_images/pull/146
Since we haven't actually been testing with runc, we need
to fix a few tests:
- handle an error-message change (make it work in both crun and runc)
- skip one system test, "survive service stop", that doesn't
work with runc and I don't think we care.
...and skip a bunch, filing issues for each:
- #15013 pod create --share-parent
- #15014 timeout in dd
- #15015 checkpoint tests time out under $CONTAINER
- #15017 networking timeout with registry
- #15018 restore --pod gripes about missing --pod
- #15025 run --uidmap broken
- #15027 pod inspect cgrouppath broken
- ...and a bunch more ("podman pause") that probably don't
even merit filing an issue.
Also, use /dev/urandom in one test (was: /dev/random) because
the test is timing out and /dev/urandom does not block. (But
the test is still timing out anyway, even with this change)
Also, as part of the VM switch we are now using go 1.18 (up
from 1.17) and this broke the gitlab tests. Thanks to @Luap99
for a quick fix.
Also, slight tweak to #15021: include the timeout value, and
reword message so command string is at end.
Also, fixed a misspelling in a test name.
Fixes: #14833
Signed-off-by: Ed Santiago <santiago@redhat.com>
Wrong variable. And, wrong index range. And, wrong bash
syntax for extracting end_port. And, add explicit check
for valid range, because die() inside 'foo=$(...)' will not
actually die. And, refactor some confusing code. And,
reformat/clean up a confusing and too-wide comment.
Fixes: #14854
Signed-off-by: Ed Santiago <santiago@redhat.com>
The test must ensure that all ports in the range are free not just
the first. This flakes often because port 5355 is always in use by
systemd-resolved on fedora.
Fixes#14716
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* Replace "setup", "lookup", "cleanup", "backup" with
"set up", "look up", "clean up", "back up"
when used as verbs. Replace also variations of those.
* Improve language in a few places.
Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
Changes:
- use --timestamp option to produce 'created' stamps
that can be reliably tested in the image-history test
- podman now supports manifest & multiarch run, so we
no longer need buildah
- bump up base alpine & busybox images
This turned out to be WAY more complicated than it should've been,
because:
- alpine 3.14 fixed 'date -Iseconds' to include a colon in
the TZ offset ("-07:00", was "-0700"). This is now consistent
with GNU date's --iso-8601 format, yay, so we can eliminate
a minor workaround.
- with --timestamp, all ADDed files are set to that timestamp,
including the custom-reference-timestamp file that many tests
rely on. So we need to split the build into two steps. But:
- ...with a two-step build I need to use --squash-all, not --squash, but:
- ... (deep sigh) --squash-all doesn't work with --timestamp (#14536)
so we need to alter existing tests to deal with new image layers.
- And, long and sordid story relating to --rootfs. TL;DR that option
only worked by a miracle relating to something special in one
specific test image; it doesn't work with any other images. Fix
seems to be complicated, so we're bypassing with a FIXME (#14505).
And, unrelated:
- remove obsolete skip and workaround in run-basic test (dating
back to varlink days)
- add a pause-image cleanup to avoid icky red warnings in logs
Fixes: #14456
Signed-off-by: Ed Santiago <santiago@redhat.com>
When a container with a userns is created the network setup is special.
Normally the netns is setup before the oci runtime container is created,
however with a userns the container is created first and then the network
is setup. In the second case we never saved the container state
afterwards. Because of it, podman inspect would not show the network info
and network teardown will not happen.
This worked with local podman because there was a save() call later in the
code path which then also saved the network status. But in the podman API
code path this save never happened thus all containers started via API had
this problem.
Fixes#14465
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When a container is run in the host network namespace we have to keep
the same resolv.conf content and not use the systemd-resolve detection
logic.
But also make sure we still allow --dns options.
Fixes#14055
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The files /etc/hosts, /etc/hostname and /etc/resolv.conf should always
be owned by the root user in the container. This worked correct for
/etc/hostname and /etc/hosts but not for /etc/resolv.conf.
A container run with --userns keep-id would have the reolv.conf file
owned by the current container user which is wrong.
Consolidate some common code in a new helper function to make the code more
cleaner.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When we connect or disconnect from a network we also have to update
/etc/hosts to ensure we only have valid entries in there.
This also fixes problems with docker-compose since this makes use of
network connect/disconnect.
Fixes#12533
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When we lookup the hostname for a given container we have to check if
the container is joined to another utsns and use this hostname then
instead.
This fixes a problem where the `hostname` command would use the correct
name but /etc/hostname would contain a different name.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Use the new logic from c/common to create the hosts file. This will help
to better allign the hosts files between buildah and podman.
Also this fixes several bugs:
- remove host entries when container is stopped and has a netNsCtr
- add entries for containers in a pod
- do not duplicate entries in the hosts file
- use the correct slirp ip when an userns is used
Features:
- configure host.containers.internal entry in containers.conf
- configure base hosts file in containers.conf
Fixes#12003Fixes#13224
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
From a security point of view, it would be nice to be able to map a
rootless usernamespace that does not use your own UID within the
container.
This would add protection against a hostile process escapping the
container and reading content in your homedir.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Problem: the system test 'is()' checker was poorly thought out.
For example, there is no way to check for inequality or for
absence of a substring.
Solution, step 1: introduce new assert(), copied almost verbatim
from buildah, where it has been successful in addressing the
gaps in is().
The logical next step is to search the tests for 'die' and
for 'run', looking for negative assertions which we can
replace with assert(). There were a lot, and in the process
I found a number of ugly bugs in the tests themselves. I've
taken the liberty of fixing these.
Important note: at this time we have both assert() and is().
Replacing all instances of is() would be impossible to review.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The slirp4netns port forwarder was not updated to make use of the new
port format. This results in a problem when port ranges are used since
it does not read the range field from the port.
Update the logic to iterate through all ports with the range and
protocols. Also added a system test for port ranges with slirp4netns,
rootlesskit and the bridge network mode.
Fixes#13643
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Append the podman dns seach domain to the host search domains when we
use the dnsname/aardvark server. Previously it would only use podman
seach domains and discard the host domains.
Fixes#13103
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This reverts commit a1bc8cb52c.
Please see resolv.conf(5) search domains must be on the same line. If
you use multiple seach key words only the last one is used. I tested this
with alpine and it works correctly when they are on the same line so I
am not sure what issues Dan had with it but this is not correct.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
