Commit Graph

5436 Commits

Author SHA1 Message Date
Paul Holzinger
8b49872669 Makefile: add ginkgo FOCUS/FOCUS_FILE options
When running ginkgo tests locally we often only want to test a small
subset. I think most people just add the `FIt` block but then you need
to remember to undo that before pushing the changes.

With this change you can just run:
```
make localintegration FOCUS="test name here"
make localintegration FOCUS_FILE="some_test.go"
```
I updated the test Readme to use this new syntax.
The options just map to the ginkgo options, see the upstream docs
linked in the readme for more information about syntax.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-16 14:44:05 +02:00
Ed Santiago
4bed2c285c e2e: refactor and document serialization
Followup to #18578: move Serial to Describe(), in case new
tests get added to this module. And, explain the reasoning.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-16 04:48:55 -06:00
OpenShift Merge Robot
e2d1ad1fb2 Merge pull request #18571 from containers/renovate/github.com-onsi-ginkgo-v2-2.x
fix(deps): update module github.com/onsi/ginkgo/v2 to v2.9.5
2023-05-16 04:05:36 -04:00
Ed Santiago
a485ba505a e2e: serialize gpg tests
Reason: gpg tests all run with a different GNUPGHOME, and gpg-agent
does not like that, and there's no longer any way to run gpg
without the agent. So, do not run these tests in parallel, and
clean up agent after each test.

Fixes: #17966 (I hope)

May also fix #18358 but it will take some time to be sure.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-15 14:47:13 -06:00
Ed Santiago
7580b8ae49 e2e: fix more test races (missing "wait")
...in three kube tests. And, missing error-message checks.
And, reverse the sense of a confusing Expect(), plus add
a description to the test failure. And, set never-restart,
otherwise our "podman wait" will spin for an indeterminate
time.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-15 11:53:04 -06:00
OpenShift Merge Robot
e0890cae14 Merge pull request #18563 from vrothberg/fix-15897
remote-save: fix permissions and dir formats
2023-05-15 13:20:18 -04:00
renovate[bot]
520979ba2e fix(deps): update module github.com/onsi/ginkgo/v2 to v2.9.5
Signed-off-by: Renovate Bot <bot@renovateapp.com>
2023-05-15 16:26:32 +00:00
Paul Holzinger
ab29ff2f66 test/e2e: dedup Before/AfterEach nodes
There is no reason to define the same code every time in each file, just
use global nodes. This diff should speak for itself.

CleanupSecrets()/Volume() no longer call Cleanup() directly, as the
global AfterEach node will always call Cleanup() this is no longer
necessary. If one AfterEach() node fails it will still run the others.

Also always unset the CONTAINERS_CONF env vars. This prevents people
from forgetting to unset it. And fix the special CONTAINERS_CONF logic
in the system connection tests, we do not want to preserve
CONTAINERS_CONF anyway so just remove this logic.

Ginkgo orders the BeforeEach and AfterEach nodes. They will be executed
from the outer-most defined to inner-most. This means our global
BeforeEach is always first. Only then the inner one (in the Describe()
function in each file). For AfterEach it is inverted, from the inner to
the outer.
Also see https://onsi.github.io/ginkgo/#organizing-specs-with-container-nodes

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-15 16:56:18 +02:00
Valentin Rothberg
d01ae436ee remote-save: fix permissions and dir formats
Make sure that the directory formats are not just substituted with their
archive counterparts but actually tar'ed up directories.  Also make sure
that the clients don't get chown errors by setting rootless user and
group ID instead of O when running in the user namespace.

Fixes: #15897
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-05-15 15:30:57 +02:00
OpenShift Merge Robot
493aac69e0 Merge pull request #18535 from rhatdan/mount
Fix handling of .containenv on tmpfs
2023-05-14 07:03:32 -04:00
OpenShift Merge Robot
d989c63751 Merge pull request #18542 from mheon/remove_image_annotations
Do not include image annotations when building spec
2023-05-13 06:04:35 -04:00
Daniel J Walsh
13f787842d Fix handling of .containenv on tmpfs
Fixes: https://github.com/containers/podman/issues/18531

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-05-13 06:03:21 -04:00
OpenShift Merge Robot
2783651005 Merge pull request #18481 from tony84727/fix/initctr-restart-policy-overridden
fix: initContainer restart policy overridden by pod
2023-05-12 09:09:21 -04:00
Matt Heon
2c0f404826 Do not include image annotations when building spec
These annotations can have security implications - crun, for
example, allows rootless containers to preserve the user's groups
through an annotation. We absolutely should not include
annotations from an untrusted image off the internet by default.

We may consider whitelisting some annotations (e.g. the legacy
WASM annotations), but given that there is now a more explicit
way of specifying an image uses the WASM runtime in the OCI image
spec, I'm just tearing this out entirely for now.

Signed-off-by: Matt Heon <mheon@redhat.com>
2023-05-11 23:48:23 -04:00
OpenShift Merge Robot
189b09d82e Merge pull request #18331 from TomSweeneyRedHat/dev/tsweeney/hooked
Add file switch for pre-exec hooks
2023-05-11 19:34:30 -04:00
OpenShift Merge Robot
20b15f07ed Merge pull request #18538 from ygalblum/play-multiple-configmaps
Kube Play - Support multi-doc YAML files for configmap argument
2023-05-11 08:25:37 -04:00
Ygal Blum
26f1e95ab4 Kube Play - Support multi-doc YAML files for configmap argument
Read the entire YAML file in case of a multi-doc file
Adjust the unit test
Add a system test
Add comment in the man page

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2023-05-11 13:13:08 +03:00
Ed Santiago
c33ba70f95 system tests: instrument, to try to catch unlinkat-ebusy
Several tweaks to see if we can track down #17216, the unlinkat-ebusy
flake:

 - teardown(): if a cleanup command fails, display it and its
   output to the debug channel. This should never happen, but
   it can and does (see #18180, dependent containers). We
   need to know about it.

 - selinux tests: use unique pod names. This should help when
   scanning journal logs.

 - many tests: add "-f -t0" to "pod rm"

And, several unrelated changes caught by accident:
 - images-commit-with-comment test: was leaving a stray image
   behind. Clean it up, and make a few more readability tweaks

 - podman-remote-group-add test: add an explicit skip()
   when not remote. (Otherwise, test passes cleanly on
   podman local, which is misleading)

 - lots of container cleanup and/or adding "--rm" to run commands,
   to avoid leaving stray containers

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-10 14:53:11 -06:00
Tony Duan
74a5b92b3a test: check restart policy of init containers
make the sure restart policy is "no" for init containers created by
`podman kube play`

Signed-off-by: Tony Duan <tony.duan@gapp.nthu.edu.tw>
2023-05-10 23:23:07 +08:00
OpenShift Merge Robot
5bf82d6ece Merge pull request #18518 from Luap99/e2e
[CI:DOCS] test: update README for integration tests
2023-05-10 09:34:40 -04:00
Ed Santiago
14001ad48a e2e: logs test: fix flakes
Yet another case of missing podman-wait. In these two, I see
no reason to run containers detached, so I just removed "-d"

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-10 04:53:44 -06:00
OpenShift Merge Robot
c307aeba37 Merge pull request #18506 from nalind/so-much-diffsize
libpod/Container.rootFsSize(): use recorded image sizes
2023-05-10 06:08:12 -04:00
OpenShift Merge Robot
3c781c4cda Merge pull request #18517 from edsantiago/quadlet_timeout
system tests: timeoutize quadlet, systemd
2023-05-10 06:03:03 -04:00
OpenShift Merge Robot
7a5daa0df3 Merge pull request #18492 from daw1012345/main
Ensure the consistent setting of the HOME env variable on container start
2023-05-10 05:34:02 -04:00
OpenShift Merge Robot
f8da6f0599 Merge pull request #18527 from edsantiago/wait_for_systemd_activate
e2e: stop podman.service test: wait for server
2023-05-10 04:09:20 -04:00
Ed Santiago
ff37615bfd e2e: stop podman.service test: wait for server
Another low-hanging fruit: test flake because podman-remote
trying to contact a server that hadn't come up.

Fixes: #17940

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-09 14:34:47 -06:00
renovate[bot]
58fd879596 fix(deps): update module golang.org/x/tools to v0.9.1
Signed-off-by: Renovate Bot <bot@renovateapp.com>
2023-05-09 16:56:11 +00:00
OpenShift Merge Robot
287a4193e7 Merge pull request #18520 from ygalblum/quadlet-test-notify
Quadlet system tests - fix socket notification
2023-05-09 12:09:14 -04:00
OpenShift Merge Robot
dc01ce8768 Merge pull request #18515 from vrothberg/fix-18486
quadlet: support `HostName`
2023-05-09 11:21:28 -04:00
Dawid Kulikowski
01e20818cc Ensure the consistent setting of the HOME env variable on container start
Signed-off-by: Dawid Kulikowski <git@dawidkulikowski.pl>
2023-05-09 16:34:28 +02:00
Ygal Blum
8aaa13accb Quadlet system tests - fix socket notification
In some tests use Notify=yes while in others remove the notification

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2023-05-09 17:27:47 +03:00
Ed Santiago
d24bec0d03 system tests: timeoutize quadlet, systemd
Run $QUADLET and all systemctl/journalctl commands using 'timeout'.
Nothing should ever, ever take more than the default 2 minutes.

Followup to #18514, in which quadlet tests are found to be
taking 9-10 minutes.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-09 07:46:12 -06:00
Paul Holzinger
3d6894426c test: update README for integration tests
- document env vars that can be used
- list up to date dependencies
- remove unnecessary GOPATH mention, no longer needed with gomodules
- use make targets to tests everything (much faster due `-p` option)
- remove tests in container section as make shell is not a valid target

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-09 15:45:43 +02:00
Nalin Dahyabhai
c400cc7ead libpod/Container.rootFsSize(): use recorded image sizes
In rootFsSize(), instead of calculating the size of the diff for every
layer of the container's base image, ask the storage library for the sum
of the values it recorded when it first wrote those layers.

In a similar fashion, teach rwSize() to use the library's
ContainerSize() method instead of trying to roll its own.

Replace calls to pkg/util.SizeOfPath() with calls to
github.com/containers/storage/pkg/directory.Size(), which does the same
thing.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2023-05-09 09:33:37 -04:00
Valentin Rothberg
b97be3406b quadlet: support HostName
Add a new `HostName` field to Quadlet `.container` files.

Fixes: #18486
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-05-09 13:35:54 +02:00
Ed Santiago
9dc107e749 e2e: fix race in a play-kube test
Yet another case where tests expect play-kube to be synchronous.
There are probably dozens more of these.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-09 05:19:05 -06:00
OpenShift Merge Robot
edcb29b19f Merge pull request #18511 from containers/renovate/golang.org-x-tools-0.x
fix(deps): update module golang.org/x/tools to v0.9.0
2023-05-09 05:03:56 -04:00
renovate[bot]
40b21fa7d5 fix(deps): update module golang.org/x/tools to v0.9.0
Signed-off-by: Renovate Bot <bot@renovateapp.com>
2023-05-09 06:13:19 +00:00
Daniel J Walsh
332b47fefb Check on client side for Containerfile, if none specified
Fixes: https://github.com/containers/podman/issues/16354

Currently we check on the server side, which ends up generating a bad
error message.

$ podman --remote build foo/
ERRO[0000] While reading directory /home/dwalsh/go/src/github.com/containers/podman/foo: EOF
Error: stat /var/tmp/libpod_builder1249622306/build/Dockerfile: no such file or directory

With this change you will get

./bin/podman --remote build foo/
Error: Containerfile not specified and no Containerfile or Dockerfile found in context directory, /home/dwalsh/podman/foo

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-05-08 13:58:37 -04:00
Ed Santiago
9ec630f305 Buildah treadmill: several fixes
- treadmill script: run root & rootless in parallel, not
  sequentially. It's only four jobs, and it seems dumb
  to fix root tests, repush, then discover a rootless failure.

- apply-podman-deltas: implement skip_if_rootless(), and
  use it to skip a nasty longstanding flake

- bud-tests-in-podman diffs: ugly code to fix a rootless hang.
   background: rootless remote tests hang
   cause: stray podman server process
   root cause: no idea. No clue at all. I just gave up
   workaround: seek out and kill stray server processes

  Rootless buildah-bud tests are not run in regular CI,
  only in the buildah treadmill.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-08 05:28:11 -06:00
renovate[bot]
1bf057e8cf chore(deps): update dependency docker to ~=6.1.0
Signed-off-by: Renovate Bot <bot@renovateapp.com>
2023-05-05 19:24:45 +00:00
renovate[bot]
773b457048 chore(deps): update dependency requests to ~=2.30.0
Signed-off-by: Renovate Bot <bot@renovateapp.com>
2023-05-04 16:51:16 +00:00
OpenShift Merge Robot
d1a696acb7 Merge pull request #18385 from dcermak/setup-env-in-init-containers
Run generate.CompleteSpec() for initContainers as well
2023-05-04 10:12:40 -04:00
OpenShift Merge Robot
8bfb27a3ce Merge pull request #18462 from Luap99/no-cleanup-twice
libpod: do not Cleanup() more than once
2023-05-04 09:55:06 -04:00
Paul Holzinger
95557a532e libpod: do not Cleanup() more than once
If the container was already cleaned up we should not try to do it
again. Podman stop will always try to call Cleanup() if you look at the
podman event log and just keep calling podman stop --all you see a
cleanup event every time. This is not wanted. Also in case of the host
pidns we report a error every single time, see the linked issue.

Fixes #18460

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-04 13:53:40 +02:00
Paul Holzinger
df9344ac4b compat container create: match duplicate mounts correctly
The logic which checks for duplicated volumes here did not work
correctly because it used filepath.Clean(). However the writes to the
volDestinations map did not thus the string no longer matched when you
included a final slash for example.

So we can either call Clean() on all or no paths. I decided to call it
on no path because this is what we do right now. Just the check did it.

Fixed #18454

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-04 11:57:02 +02:00
Dan Čermák
75d92f41d8 Run generate.CompleteSpec() for initContainers as well
initContainers in kubernetes deployments had no call to CompleteSpec in the
generation, which means that the default environment is not configured for
these. This causes issues with missing default environment variables like $HOME
or $PÄTH.

Also, switch to using logrus.Warn() instead of fmt.Fprintf(os.Stderr)

This fixes https://github.com/containers/podman/issues/18384

Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Dan Čermák <dcermak@suse.com>
2023-05-04 08:21:57 +02:00
renovate[bot]
d25356c303 fix(deps): update module github.com/onsi/ginkgo/v2 to v2.9.4
Signed-off-by: Renovate Bot <bot@renovateapp.com>
2023-05-03 22:33:15 +00:00
Ed Santiago
f877d7dcd0 Replace egrep/fgrep with grep -E/-F
There are days when I really, really, really hate GNU. Remember
when someone decided that 'head -1' would no longer work, and
that it was OK to break an infinite number of legacy production
scripts? Someone now decided that egrep/fgrep are deprecated,
and our CI logs (especially pr-should-include-tests) are now
filled with hundreds of warning lines, making it difficult
to find actual errors.

I expect that those warnings will be removed quickly after
furious community backlash, just like the 'head -1' fiasco
was quietly reverted, but ITM the warnings are annoying
so I capitulate.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-03 07:32:42 -06:00
renovate[bot]
1b2b0d0315 fix(deps): update module github.com/onsi/ginkgo/v2 to v2.9.3
Signed-off-by: Renovate Bot <bot@renovateapp.com>
2023-05-03 05:35:43 +00:00