17074 Commits

Author SHA1 Message Date
cb8c9af5d3 test/system: Add, use tcp_port_probe() to check for listeners rather than binds
_test_skopeo_credential_sharing() used port_is_free() to check if a
port has no active listeners. With the new implementation, this is
not equivalent anymore: a port might be in TIME_WAIT, so it's not
free, but the listener might be long gone.

Add tcp_port_probe() to check if there's an active listener on a
given port, and use it in _test_skopeo_credential_sharing().

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:36 +01:00
348c3f2833 test/system: Add tests for pasta(1) connectivity
These tests should cover all the basic networking functionality with
pasta(1). Namely, they check:

- IPv4 and IPv6 addressing and routing settings

- TCP and UDP port forwarding over IPv4 and IPv6

- data transfers and ICMP/ICMPv6 echo requests

- the (exceedingly simple) lifecycle handling

These tests need some new helpers, to obtain IPv4 and IPv4 addresses
and routes, as well as MTU and interface names. Those use jq(1) for
parsing.

Some availability checks are implemented as well, to skip tests if
pasta(1) is not available, or if IPv4 and IPv6 are not usable.

To get consistent outcomes across distributions, and to enable
uncomplicated termination for UDP tests based on zero-sized packets,
use socat(1), which, unlike netcat, doesn't suffer from option
inconsistencies depending on flavours (traditional, BSD, NMAP) and
versions.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:36 +01:00
b3cf836841 test/system: Move network-related helpers to helpers.network.bash
The main helpers.bash file is rather bloated and it's difficult to
find stuff there. Move networking functions to their own helper
file.

While at it, apply a consistent style, and rearrange logically
related functions into sections.

Suggested-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:36 +01:00
ea4f168b3a test/system: Use procfs to find bound ports, with optional address and protocol
Using bash /dev/tcp/ pseudo-device files to probe for bound ports has
indeed the advantage of simplicity, but comes with a few drawbacks:

- it will actually send data to unsuspecting services that might be
  running in the same network namespace as the tests, possibly
  causing unwanted interactions

- it doesn't allow for UDP probing

- it makes it impossible to clearly distinguish between different
  address bindings

Replace that approach with a new helper, port_is_bound(), that uses
procfs entries at /proc/net to detect bound ports, without the need
for active probing.

We can now implement optional parameters in callers, to check if a
port if free for binding to a given address, including any IPv4
(0.0.0.0) or any IPv6 (::0) address, and for a given protocol, TCP
or UDP.

Extend random_free_port() and random_free_port_range() to support
that.

The implementation of one function in the file
test/system/helpers.bash, namely ipv6_to_procfs(), and the
implementation of the corresponding own test, delimited by the
markers "# BEGIN ipv6_to_procfs" and "# END   ipv6_to_procfs" in the
file test/system/helpers.c was provided, on the public forum at:

  https://github.com/containers/podman/pull/16141

by Ed Santiago <santiago@redhat.com>, who expressly invited me to
include them in this code submission.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:35 +01:00
7e3d04fbc6 test/system: Use port_is_free() from wait_for_port()
Currently, wait_for_port() duplicates the check logic implemented by
port_is_free().

Add an optional argument to port_is_free(), representing the bound
address to check, and call it, dropping the direct check in
wait_for_port().

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:35 +01:00
aa47e05ae4 libpod: Add pasta networking mode
Conceptually equivalent to networking by means of slirp4netns(1),
with a few practical differences:

- pasta(1) forks to background once networking is configured in the
  namespace and quits on its own once the namespace is deleted:
  file descriptor synchronisation and PID tracking are not needed

- port forwarding is configured via command line options at start-up,
  instead of an API socket: this is taken care of right away as we're
  about to start pasta

- there's no need for further selection of port forwarding modes:
  pasta behaves similarly to containers-rootlessport for local binds
  (splice() instead of read()/write() pairs, without L2-L4
  translation), and keeps the original source address for non-local
  connections like slirp4netns does

- IPv6 is not an experimental feature, and enabled by default. IPv6
  port forwarding is supported

- by default, addresses and routes are copied from the host, that is,
  container users will see the same IP address and routes as if they
  were in the init namespace context. The interface name is also
  sourced from the host upstream interface with the first default
  route in the routing table. This is also configurable as documented

- sandboxing and seccomp(2) policies cannot be disabled

- only rootless mode is supported.

See https://passt.top for more details about pasta.

Also add a link to the maintained build of pasta(1) manual as valid
in the man page cross-reference checks: that's where the man page
for the latest build actually is -- it's not on Github and it doesn't
match any existing pattern, so add it explicitly.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-11-08 00:16:35 +01:00
ec03579aa2 Merge pull request #16435 from edsantiago/fix_static_race
Fix test flakes caused by improper podman-logs
2022-11-07 14:39:50 -05:00
781ecec469 Merge pull request #16386 from bjorndown/feature/pod-create-with-netns
Allow namespace path network option for pods.
2022-11-07 14:29:24 -05:00
d60b25fc29 Merge pull request #16433 from Luap99/systemd-check
fix incorrect systemd booted check
2022-11-07 14:27:00 -05:00
3ebcfdbbc6 Fix test flakes caused by improper podman-logs
This one has been a thorn in my side: it's a podman-log issue,
but not remote, so I _almost_ retitled #16132 (removing "remote").

Nope, it's a bug in the tests themselves. One solution would be to
podman-wait, but I see no reason for logs to be involved, so I
went with podman start -a instead. This removes the k8s-log stuff
which is no longer necessary. Cleanup all around.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-11-07 11:29:06 -07:00
919678d2f8 fix incorrect systemd booted check
Check for the directory /run/systemd/system, this is described in
sd_booted(3). Reading /proc/1/comm will fail when /proc is mounted
with the `hidepid=2` option.

[NO NEW TESTS NEEDED]

Fixes #16022

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2022-11-07 18:23:55 +01:00
9e04bab722 Merge pull request #16418 from jakegt1/main
Fix compat build API returning two JSON objects at once
2022-11-07 09:49:57 -05:00
e300c269fa Merge pull request #16410 from rhatdan/codespell
Run codespell on code
2022-11-07 06:44:25 -05:00
9efa9d8e96 Merge pull request #16422 from dfr/freebsd-build
Makefile: don't install to tmpfiles.d on FreeBSD
2022-11-06 19:21:29 -05:00
d5031946ab Makefile: don't install to tmpfiles.d on FreeBSD
There is no equivalent equivalent on FreeBSD and this causes lint
failures when packaging.

[NO NEW TESTS NEEDED]

Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-11-06 11:08:02 +00:00
85f4d37173 Make sure we can build and read each line of docker py's api client
Signed-off-by: Jake Torrance <jaket1234@hotmail.com>
2022-11-06 10:30:18 +00:00
cdb00332da Docker compat build api - make sure only one line appears per flush
Closes #16360

Signed-off-by: Jake Torrance <jake.torrance@oracle.com>
Signed-off-by: Jake Torrance <jaket1234@hotmail.com>
2022-11-06 09:43:34 +00:00
afc84151fc Merge pull request #15591 from snprajwal/go-criu/v6
Upgrade go-criu to v6
2022-11-04 15:48:52 -04:00
e83bb91536 Merge pull request #16346 from eriksjolund/add_docs_userns_keep-id_uid_gid
[CI:DOCS] troubleshooting: document keep-id options
2022-11-04 11:53:17 -04:00
6d44d332a2 Merge pull request #16409 from rhatdan/VENDOR
Update vendor of containers/(image, storage, common)
2022-11-04 11:08:11 -04:00
efbad590d7 Run codespell on code
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-04 10:57:41 -04:00
571833d562 Update vendor of containers/(image, storage, common)
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-04 08:58:12 -04:00
5aa32e45ad Merge pull request #16377 from giuseppe/add-check-for-valid-pod-systemd-cgroup
runtime: add check for valid pod systemd cgroup
2022-11-04 07:59:22 -04:00
e3d174ae1d Merge pull request #16396 from cevich/sync_windows
Cirrus: Synchronize windows image
2022-11-04 06:53:12 -04:00
017521c283 Merge pull request #16399 from Luap99/no-apk
test/e2e: do not use apk in builds
2022-11-04 06:50:19 -04:00
049a5d82f1 Allow namespace path network option for pods.
Closes #16208.

Signed-off-by: Björn Mosler <dev@bjoern.mosler.ch>
2022-11-04 11:26:06 +01:00
013bd81417 Merge pull request #16394 from vrothberg/fix-16387
remove container/pod id file along with container/pod
2022-11-04 05:22:57 -04:00
0702b4cf4c Merge pull request #15359 from rhatdan/manifest
Add --insecure flag to podman manifest inspect for Docker compatibility
2022-11-04 05:14:46 -04:00
58cd3ff2b8 Merge pull request #16401 from cevich/fix_windows_cross
[CI:BUILD] Cirrus: Never skip running Windows Cross task
2022-11-03 17:13:55 -04:00
eb7da14015 Merge pull request #16357 from cevich/cron_auto_rerun
[CI:DOCS] [WIP] GHA: Auto. re-run failed cirrus-cron builds once
2022-11-03 16:37:42 -04:00
f3195c930b Cirrus: Never skip running Windows Cross task
All the other Windows tasks depend on access to a podman-remote build
from the Alt. Arch. `Windows Cross` task.  Re-arrange the test-skipping
call to never skip here only.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-11-03 16:17:24 -04:00
35523d560a GHA: Auto. re-run failed cirrus-cron builds once
With a seemingly ever growing list of cirrus-cron jobs running on
release branches, there are bound to be some hiccups.  Sometimes a lot
of them.  Normally any failures require a human to eyeball the logs
and/or manually re-run the job to see if it was simply a flake.  This
doesn't take long, but can be distracting and compounds over time.

Attempt to alleviate some maintainer burden by using a new github action
workflow to perform **one** automatic re-run on any failed builds.  This
task is scheduled an hour prior to a second failure check, and generation
of notification e-mail for review.

Note: If there are no failures, due to the auto. re-run or luck, no
e-mail is generated. If this proves useful in this repo, I intend to
re-use this workflow for other repo's cirrus-cron jobs.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-11-03 13:54:39 -04:00
3a85d537b6 GHA: Migrate inline script to file
Inline scripts make github-action workflow YAML harder to read/maintain.
Relocate the e-mail formation script to a dedicated file.  This also
permits better input-validation and re-use of a common `err()` function.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-11-03 13:54:39 -04:00
980d5b3622 GHA: Simplify script reference
This workflow was originally crafted to be (somehow) reused with
different scripts.  That never happened and the extra indirection is
confusing and hard to maintain.  Remove it.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-11-03 13:54:38 -04:00
4174901283 test/e2e: do not use apk in builds
As far as I can tell there is no reason to use apk in these tests. They
just build an image and check for it and never use the installed binary.
Network calls are always unstable and therefore should be avoided when
possible, this ensures no/less flakes.

Fixes #16391

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2022-11-03 17:32:49 +01:00
3fee351c35 remove container/pod id file along with container/pod
Remove the container/pod ID file along with the container/pod.  It's
primarily used in the context of systemd and are not useful nor needed
once a container/pod has ceased to exist.

Fixes: #16387
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-11-03 17:07:09 +01:00
442df29679 Cirrus: Synchronize windows image
It's important/useful to have all VM images built around the same time
as it prevents tooling/dependency divergence and therefore simpler
debugging.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-11-03 10:17:10 -04:00
6428ff180a Merge pull request #16389 from edsantiago/verify_network_backend
CI: set and verify DESIRED_NETWORK (netavark, cni)
2022-11-03 08:25:31 -04:00
a1fe0cd662 Merge pull request #16374 from rhatdan/man
[CI:DOCS] Fix documentation on read-only-tmpfs
2022-11-03 08:02:16 -04:00
274d0f4956 Add --insecure,--tls-verify,--verbose flags to podman manifest inspect
--insecure and --verbose flags for docker compatibility

--tls-verify for syntax compatibility and allow users to inspect
manifests at remote Container Registiries without requiring tls.

Helps fix: https://github.com/containers/podman/issues/14917

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-03 08:00:15 -04:00
cac4919bf6 runtime: add check for valid pod systemd cgroup
there is already the same check when using cgroupfs, but not when
using the systemd cgroup backend.  The check is needed to avoid a
confusing error from the OCI runtime.

Closes: https://github.com/containers/podman/issues/16376

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-11-03 12:23:44 +01:00
d7e70c7489 CI: set and verify DESIRED_NETWORK (netavark, cni)
We have CI tests running in netavark mode when CNI is desired.
Add a new .cirrus.yml envariable, CI_DESIRED_NETWORK, which
we then force-check in e2e and system tests. Simple copy/paste
of #14912 (the RUNTIME check) with manual s/RUNTIME/NETWORK/
and other minor changes.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-11-03 04:40:42 -06:00
6ec2bcb689 [CI:DOCS] troubleshooting: document keep-id options
Add new troubleshooting tip:
Podman run fails with "Error: unrecognized namespace mode keep-id:uid=1000,gid=1000 passed"

Update the troubleshooting tips:
"Passed-in devices or files can't be accessed in rootless container (UID/GID mapping problem)"
and
"Container creates a file that is not owned by the user's regular UID"
to use
"--userns keep-id:uid=$uid,gid=$gid"
instead of the command-line options --uidmap and --gidmap

Co-authored-by: Tom Sweeney <tsweeney@redhat.com>
Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
2022-11-02 22:18:29 +01:00
774e950374 Merge pull request #15760 from cevich/enable_kube_tests
Cirrus: Update CI VM images to F37beta
2022-11-02 17:04:30 -04:00
c35ed35081 Merge pull request #15091 from umohnani8/lift
Add podman kube apply command
2022-11-02 14:38:13 -04:00
8530724555 Cirrus: Guarantee CNI testing w/o nv/av present
Comments added to code for clarity.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-11-02 13:42:06 -04:00
fd9de876f1 Cirrus: temp. disable all Ubuntu testing
Building Ubuntu VM images is temporarily broken due to dependency
problems on (missing) netavaro for the (required) podman package.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-11-02 13:39:21 -04:00
ecd1927b4c Cirrus: Update to F37beta
Signed-off-by: Chris Evich <cevich@redhat.com>
2022-11-02 13:38:28 -04:00
0e367c7a18 Merge pull request #16257 from n1hility/win-smoke
Add Windows Smoke Testing
2022-11-02 10:58:50 -04:00
c772c00f6f Merge pull request #16384 from edsantiago/budtests_remote
buildah bud tests: better handling of remote
2022-11-02 10:48:24 -04:00