Commit Graph

956 Commits

Author SHA1 Message Date
OpenShift Merge Robot
aeb36e5dad Merge pull request #16825 from alexlarsson/quadlet-system-test
Quadlet system test
2022-12-18 07:16:50 -05:00
OpenShift Merge Robot
f21c64019f Merge pull request #16800 from Luap99/restore-static-ip
checkpoint restore: fix --ignore-static-ip/mac
2022-12-16 11:03:40 -05:00
Alexander Larsson
80878f20bc Add initial system tests for quadlets
This adds basic container and volume system tests for quadlet. These
install and run actual systemd units and ensure they work.

Signed-off-by: Alexander Larsson <alexl@redhat.com>
2022-12-16 15:35:51 +01:00
Valentin Rothberg
f4d0496b54 wait: add --ignore option
In the recent past, I met the frequent need to wait for a container to
exist that, at the same time, may get removed (e.g., system tests in [1]).

Add an `--ignore` option to podman-wait which will ignore errors when a
specified container is missing and mark its exit code as -1.  Also
remove ID fields from the WaitReport.  It is actually not used by
callers and removing it makes the code simpler and faster.

Once merged, we can go over the tests and simplify them.

[1] github.com/containers/podman/pull/16852

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-12-16 10:24:29 +01:00
OpenShift Merge Robot
18f1a8046b Merge pull request #16733 from rhatdan/caps
Vendor in latest containers/common with default capabilities
2022-12-16 03:58:06 -05:00
Daniel J Walsh
3718ac8e96 Vendor in latest containers/common with default capabilities
Also update vendor of containers/storage and image

Cleanup display of added/dropped capabilties as well

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-12-15 14:28:30 -05:00
Giuseppe Scrivano
290019c486 rootless: add cli validator
whenever the podman process is launched, it runs any file found in
these directories:

- /etc/containers/auth-scripts
- /usr/libexec/podman/auth-scripts

The current podman command line is passed as arguments to the
process.

If any of the processes fail, the error is immediately reported back
from podman that exits with the same error code.

[NO NEW TESTS NEEDED] requires a system-wide configuration.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-12-15 18:31:16 +01:00
Daniel J Walsh
4a5581ce0d stop reporting errors removing containers that don't exist
Init containers are removed once they exit, but podman
reports and error that the container does not exist, when
it was previously removed.  Stop reporting missing containers
when removing.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-12-14 14:09:56 -05:00
Paul Holzinger
45a40bf587 checkpoint restore: fix --ignore-static-ip/mac
With the 4.0 network rewrite I introduced a regression in 094e1d70de.
It only covered the case where a checkpoint is restored via --import.
The normal restore path was not covered since the static ip/mac are now
part in an extra db bucket. This commit fixes that by changing the config
in the db.

Note that there were no test for --ignore-static-ip/mac so I added a big
system test which should cover all cases (even the ones that already
work). This is not exactly pretty but I don't have to enough time to
come up with something better at the moment.

Fixes #16666

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2022-12-12 18:34:49 +01:00
Daniel J Walsh
43e307b84f Add podman buildx version support
Fixes: https://github.com/containers/podman/issues/16793

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-12-09 12:03:22 -05:00
Paul Holzinger
2dde30b93a remote: allow --http-proxy for remote clients
The remote client should be allowed to specify if the container should
be run with the proxy env vars. It will still use the proxy vars from
the server process and not the client. This makes podman-remote more
consistent with the local version and easier to use in environments
where a proxy is required.

Fixes #16520

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2022-12-08 17:08:37 +01:00
Valentin Rothberg
4fa307f149 kube sdnotify: run proxies for the lifespan of the service
As outlined in #16076, a subsequent BARRIER *may* follow the READY
message sent by a container.  To correctly imitate the behavior of
systemd's NOTIFY_SOCKET, the notify proxies span up by `kube play` must
hence process messages for the entirety of the workload.

We know that the workload is done and that all containers and pods have
exited when the service container exits.  Hence, all proxies are closed
at that time.

The above changes imply that Podman runs for the entirety of the
workload and will henceforth act as the MAINPID when running inside of
systemd.  Prior to this change, the service container acted as the
MAINPID which is now not possible anymore; Podman would be killed
immediately on exit of the service container and could not clean up.

The kube template now correctly transitions to in-active instead of
failed in systemd.

Fixes: #16076
Fixes: #16515
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-12-06 14:15:11 +01:00
Valentin Rothberg
8c3af71862 notify k8s system test: move sending message into exec
The flake in #16076 is likely related to the notify message not being
delivered/read correctly.  Move sending the message into an exec session
such that flakes will reveal an error message.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-12-05 14:32:06 +01:00
Daniel J Walsh
af86b4f62a Fix handling of readonly containers when defined in kube.yaml
The containers should be able to write to tmpfs mounted directories.

Also cleanup output of podman kube generate to not show default values.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-12-03 07:05:58 -05:00
Valentin Rothberg
71f92d263c container create: add inspect data to event
When the new `events_container_create_inspect_data` option is enabled in
containers.conf set the `ContainersInspectData` event field for each
container-create event.

The data was requested for the purpose of auditing (e.g., intrusion
detection).

Jira: https://issues.redhat.com/browse/RUN-1702
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-11-29 10:40:24 +01:00
Daniel J Walsh
f17479c711 Run codespell on code
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-28 15:13:43 -05:00
Valentin Rothberg
e2fa94e8ac container restart: clean up healthcheck state
When restarting a container, clean up the healthcheck state by removing
the old log on disk.  Carrying over the old state can lead to various
issues, for instance, in a wrong failing streak and hence wrong
behaviour after the restart.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2144754
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-11-24 10:22:22 +01:00
Ed Santiago
50d72bc639 ssh system test: skip until it becomes a test
The 900-ssh test is not an actual test, and I'm unable to
figure out how to make it one. Skip it for now, but add a
bunch of FIXMEs some someone can come in later and actually
implement it.

Also removed lots of dead code and misleading comments.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-11-21 08:02:00 -07:00
OpenShift Merge Robot
d8fed62c96 Merge pull request #16439 from sstosh/proxy-tests
system tests: fix a system test in proxy environment
2022-11-18 00:51:30 +00:00
OpenShift Merge Robot
54b1fd35bd Merge pull request #16518 from ashley-cui/noout
Fix podman --noout to suppress all output
2022-11-15 21:57:33 +00:00
OpenShift Merge Robot
d1ac0dcd49 Merge pull request #16513 from vrothberg/fix-16502
remove pod if creation has failed
2022-11-15 21:54:57 +00:00
Ashley Cui
7c1ad8a582 Fix podman --noout to suppress all output
Podman --noout was not suppressing output from commands that do not
create the podman engine. Now, podman --noout properly suppresses output
from every command.

Fixes: https://github.com/containers/podman/issues/16201

Signed-off-by: Ashley Cui <acui@redhat.com>
2022-11-15 08:16:46 -05:00
Valentin Rothberg
9610d4c7b4 remove pod if creation has failed
Make sure to remove the pod if its creation has failed.

Fixes: #16502
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-11-15 13:36:57 +01:00
Ed Santiago
1a839a96dc healthcheck system tests: try to fix flake
Weird one-off flake seen:

   # ... healthcheck run <containername>
   Error: container SHA is not running

The only way I can see this happening is if the healthcheck
auto-timer triggered, which seems impossible because that
should be 30s and the log timestamps show this test taking
18s. But, shrug, let's see if disabling the timer works. I
don't have high hopes that this will fix anything, but it's
probably a good idea regardless.

Also, since this test loops over different policies, include
policy name in error messages as a courtesy. (It's obtainable
anyway by scrolling up)

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-11-14 12:33:34 -07:00
Valentin Rothberg
3371c9d25e podman cp: fix copying with "." suffix
Fix a bug for special-casing "." where Podman has mistakenly been
looking for a "." suffix instead of interpreting it as a path.

Add regression tests for the host-to-container, container-to-host and
container-to-container use cases.  Have separate tests for each to
verify that previous Podman versions fail each case.

Fixes: #16421
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-11-14 09:38:02 +01:00
OpenShift Merge Robot
25c67d6b9b Merge pull request #16462 from rhatdan/manifest
Add hidden podman manifest inspect -v option
2022-11-12 10:51:53 +00:00
Daniel J Walsh
00b2bc9b65 Add hidden podman manifest inspect -v option
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-11 15:22:31 -05:00
Daniel J Walsh
836ca6c008 Add podman volume create -d short option for driver
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-11 14:43:07 -05:00
Daniel J Walsh
bc77c034f8 Add podman system events alias to podman events
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-11 12:57:49 -05:00
OpenShift Merge Robot
aaa88f4554 Merge pull request #16141 from sbrivio-rh/pasta
Add pasta networking mode
2022-11-10 17:44:38 +00:00
Hironori Shiina
314cba2590 Skip test if --events-backend is necessary with podman-remote
`podman-remote` does not support `--events-backend`, which overrides a
log driver. When `--events-backend` is necessary in a test for
`podman-remote`, the test should be skipped.

We don't need to fix the other cases with
`_additional_events_backend()` because `_log_test_follow()` already has
the same skipping logic and `_log_test_multi()` always skips a test when
testing `podman-remote`.

Signed-off-by: Hironori Shiina <shiina.hironori@fujitsu.com>
2022-11-08 16:10:30 -05:00
Toshiki Sonoda
616fca9ffd system tests: fix a system test in proxy environment
When we are using a proxy, 'podman build - basic test'
will be failed on remote.
This test needs to add the '--http-proxy' option.

Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
2022-11-08 15:13:43 +09:00
Stefano Brivio
cb8c9af5d3 test/system: Add, use tcp_port_probe() to check for listeners rather than binds
_test_skopeo_credential_sharing() used port_is_free() to check if a
port has no active listeners. With the new implementation, this is
not equivalent anymore: a port might be in TIME_WAIT, so it's not
free, but the listener might be long gone.

Add tcp_port_probe() to check if there's an active listener on a
given port, and use it in _test_skopeo_credential_sharing().

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:36 +01:00
Stefano Brivio
348c3f2833 test/system: Add tests for pasta(1) connectivity
These tests should cover all the basic networking functionality with
pasta(1). Namely, they check:

- IPv4 and IPv6 addressing and routing settings

- TCP and UDP port forwarding over IPv4 and IPv6

- data transfers and ICMP/ICMPv6 echo requests

- the (exceedingly simple) lifecycle handling

These tests need some new helpers, to obtain IPv4 and IPv4 addresses
and routes, as well as MTU and interface names. Those use jq(1) for
parsing.

Some availability checks are implemented as well, to skip tests if
pasta(1) is not available, or if IPv4 and IPv6 are not usable.

To get consistent outcomes across distributions, and to enable
uncomplicated termination for UDP tests based on zero-sized packets,
use socat(1), which, unlike netcat, doesn't suffer from option
inconsistencies depending on flavours (traditional, BSD, NMAP) and
versions.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:36 +01:00
Stefano Brivio
b3cf836841 test/system: Move network-related helpers to helpers.network.bash
The main helpers.bash file is rather bloated and it's difficult to
find stuff there. Move networking functions to their own helper
file.

While at it, apply a consistent style, and rearrange logically
related functions into sections.

Suggested-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:36 +01:00
Stefano Brivio
ea4f168b3a test/system: Use procfs to find bound ports, with optional address and protocol
Using bash /dev/tcp/ pseudo-device files to probe for bound ports has
indeed the advantage of simplicity, but comes with a few drawbacks:

- it will actually send data to unsuspecting services that might be
  running in the same network namespace as the tests, possibly
  causing unwanted interactions

- it doesn't allow for UDP probing

- it makes it impossible to clearly distinguish between different
  address bindings

Replace that approach with a new helper, port_is_bound(), that uses
procfs entries at /proc/net to detect bound ports, without the need
for active probing.

We can now implement optional parameters in callers, to check if a
port if free for binding to a given address, including any IPv4
(0.0.0.0) or any IPv6 (::0) address, and for a given protocol, TCP
or UDP.

Extend random_free_port() and random_free_port_range() to support
that.

The implementation of one function in the file
test/system/helpers.bash, namely ipv6_to_procfs(), and the
implementation of the corresponding own test, delimited by the
markers "# BEGIN ipv6_to_procfs" and "# END   ipv6_to_procfs" in the
file test/system/helpers.c was provided, on the public forum at:

  https://github.com/containers/podman/pull/16141

by Ed Santiago <santiago@redhat.com>, who expressly invited me to
include them in this code submission.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:35 +01:00
Stefano Brivio
7e3d04fbc6 test/system: Use port_is_free() from wait_for_port()
Currently, wait_for_port() duplicates the check logic implemented by
port_is_free().

Add an optional argument to port_is_free(), representing the bound
address to check, and call it, dropping the direct check in
wait_for_port().

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:35 +01:00
Ed Santiago
6dd508b8e0 More log-flake work
It looks like #16132 was my fault: a missing 'wait' for a container
to exit. Let's see if this fixes the flake.

And, while poking through flake logs, I found another missing wait.

And... in wait_for_output(), address a potential race.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-11-07 14:35:24 -07:00
OpenShift Merge Robot
013bd81417 Merge pull request #16394 from vrothberg/fix-16387
remove container/pod id file along with container/pod
2022-11-04 05:22:57 -04:00
OpenShift Merge Robot
0702b4cf4c Merge pull request #15359 from rhatdan/manifest
Add --insecure flag to podman manifest inspect for Docker compatibility
2022-11-04 05:14:46 -04:00
Valentin Rothberg
3fee351c35 remove container/pod id file along with container/pod
Remove the container/pod ID file along with the container/pod.  It's
primarily used in the context of systemd and are not useful nor needed
once a container/pod has ceased to exist.

Fixes: #16387
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-11-03 17:07:09 +01:00
Daniel J Walsh
274d0f4956 Add --insecure,--tls-verify,--verbose flags to podman manifest inspect
--insecure and --verbose flags for docker compatibility

--tls-verify for syntax compatibility and allow users to inspect
manifests at remote Container Registiries without requiring tls.

Helps fix: https://github.com/containers/podman/issues/14917

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-03 08:00:15 -04:00
Ed Santiago
d7e70c7489 CI: set and verify DESIRED_NETWORK (netavark, cni)
We have CI tests running in netavark mode when CNI is desired.
Add a new .cirrus.yml envariable, CI_DESIRED_NETWORK, which
we then force-check in e2e and system tests. Simple copy/paste
of #14912 (the RUNTIME check) with manual s/RUNTIME/NETWORK/
and other minor changes.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-11-03 04:40:42 -06:00
Daniel J Walsh
71f0c9f33a Eval symlinks on XDG_RUNTIME_DIR
Partial Fix for https://github.com/containers/podman/issues/14606

[NO NEW TESTS NEEDED]

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-10-28 14:32:39 -04:00
Ed Santiago
11e4c04038 System tests: teardown: clean up volumes
One of the system tests was creating a volume and not cleaning up
after itself. Fix that: do cleanup in the test itself. And, add
a 'volume rm -af' to global teardown() to leave things clean for
the next tests.

Also, OOPS! Correct some instances of 'podman' in two system
tests to 'run_podman'. And remove an unused (misleading) variable.

And, one more: in auto-update test, unit file, use $PODMAN,
not /usr/bin/podman

UGH! Yet one more: found/fixed a 'run<space>podman'

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-10-26 09:56:58 -06:00
Ed Santiago
fdc9ca076b system tests: runlabel: use podman-under-test
I have no idea what this usage means, but the test fails
on a system with no /usr/bin/podman ... and that suggests
to me that the test is broken, in that it's been using
/usr/bin/podman instead of the $PODMAN we're testing.

Solution: 'podman', not '/usr/bin/podman'. Per @Luap99,
podman will replace the string 'podman' with /proc/self/exe

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-10-26 05:35:10 -06:00
Ed Santiago
05bdc72941 system tests: podman network create: use random port
One test was using a hardcoded fixed port, with a comment
referring to #10806. That issue seems fixed, so let's
try switching to a pseudorandom open port.

Does not actually fix #16289 but I'm going to close that
anyway, will reopen if it recurs.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-10-25 12:38:35 -06:00
Valentin Rothberg
f0ba2d89e8 sig-proxy test: bump timeout
Bump the timeout waiting for the container to process the signal.
The comparatively short timeout is most likely responsible for
flakes in gating tests.

Fixes: #16091
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-10-25 15:12:56 +02:00
Valentin Rothberg
221cfc6872 container/pod id file: truncate instead of throwing an error
Truncate the container and pod ID files instead of throwing an error.
The main motivation is to prevent redundant work when starting systemd
units.  Throwing an error when the file already exists is not preventing
races or file corruptions, so let's leave that to the user which in
almost all cases are generated (and tested) systemd units.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-10-25 09:37:35 +02:00
OpenShift Merge Robot
5af1c42119 Merge pull request #16211 from vrothberg/fix-16135
system df: fix image-size calculations
2022-10-19 10:46:01 -04:00