18937 Commits

Author SHA1 Message Date
28573fa9d2 Merge pull request #18255 from edsantiago/cleanup
e2e test cleanup
2023-05-02 12:15:49 -04:00
ae5305134f Revert "logformatter: anchors: link to test summary, not name"
It looks like tell/seek don't work in CI-land: important test log
information is being lost. Revert this commit, maybe some day I'll
come up with a better solution.

This reverts commit 1bff0108f6275cc8290e649833f3a194cd6e9ee1.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-02 10:03:29 -06:00
d494af9647 Merge pull request #18414 from vrothberg/fix-17763
history: correctly set tags
2023-05-02 11:57:23 -04:00
fbe62f329a More cleanup: volumes: do not export to stdout
This one got complicated, and deserves its own commit.

Problem: ginkgo logs have a lot of NUL characters, making them
difficult for logformatter to process and for humans to read.

Cause: Paul tracked it down to "podman volume export" without "-o"
(hence spitting out tar data to stdout).

Solution: add "-o tmpfile" to named podman-volume-export. In
the process, fix all sorts of other problems with that test.
And, since the e2e test no longer tests "volume export" by
itself, add a system test that does.

It is possible that there are other places that emit NULs.
One step at a time.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-02 08:36:26 -06:00
3050a64373 e2e test cleanup
- fix a typo that was resulting in a test being a NOP, and
   add actual testing to it.

 - fix two Expects() with incorrectly-ordered actual/expects

 - remove leading whitespace from an It() test name

 - To(BeTrue()) is evil. Wherever possible, replace it with
   useful string or field checks. When not possible, use
   the annotation field to indicate what failed. I got
   carried away here, #sorrynotsorry

 - remove unused system-test code

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-02 08:36:26 -06:00
fa1ba17bc1 Update kube gen & play to use pod restart policy
Podman kube generate now uses the pod's restart policy
when generating the kube yaml. If generating from containers
only, use the restart policy of the first non-init container.
Podman kube play applies the pod restart policy from the yaml
file to the pod. The containers within a pod inherit this restart
policy.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-05-02 10:30:07 -04:00
db4ad54f92 Add {{.Restarts}} to podman pod ps
Add Restarts column to the podman pod ps output to show the total number
of times the containers in a pod were restarted. This is the same as the
restarts column displayed by kubernetes with kubectl get pods. This will
only be displayed when --format={{.Restarts}}.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-05-02 10:30:07 -04:00
0fef113a4b Add {{.Restarts}} to podman ps
Add Restarts column to the podman ps output to show how many times a
container was restarted based on its restart policy. This column will be
displayed when --format={{.Restarts}}.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-05-02 10:30:07 -04:00
edbeee5238 Add --restart flag to pod create
Add --restart flag to pod create to allow users to set the
restart policy for the pod, which applies to all the containers
in the pod. This reuses the restart policy already there for
containers and has the same restart policy options.
Add "never" to the restart policy options to match k8s syntax.
It is a synonym for "no" and does the exact same thing where the
containers are not restarted once exited.
Only the containers that have exited will be restarted based on the
restart policy, running containers will not be restarted when an exited
container is restarted in the same pod (same as is done in k8s).

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-05-02 10:29:58 -04:00
f173db755d Merge pull request #18163 from Luap99/ginkgo-v2
update to ginkgo v2
2023-05-02 10:29:41 -04:00
9e9bcec9ec history: correctly set tags
Requires vendoring fixes from c/common and to update the transformation
code.  Also add a test to avoid future regressions.

Fixes: #17763
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-05-02 16:21:09 +02:00
073cd4bb47 fix(deps): update module github.com/moby/term to v0.5.0
Signed-off-by: Renovate Bot <bot@renovateapp.com>
2023-05-02 13:51:31 +00:00
3858a833bf Makefile: do not run machine test in parallel
while reworking ginkgo to use -p by default we also forced the machine
tests to be run in parallel. Right now this does not work at all
(something that should be fixed).

Using -p is easier becuase that will let ginkgo decide how many parallel
nodes to use so it much faster on high core counts.

So use some makefile magic to instaed of using `GINKGONODES` use
`GINKGO_PARALLEL` and set it to `y` by default. The machine tests will
then use that var to disable it.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 12:30:18 +02:00
5af4339237 pkg/machine/e2e: switch to GinkgoWriter
Directly writing to stdout/err is not safe when run in parallel.
Ginkgo v2 fixed this buffering the output and syncing the output so it
is not mangled between tests.

This is the same as for the podman integration tests.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 12:07:42 +02:00
1720042d04 api: fix parsing filters
Podman and Docker clients split the filter map slightly different, so
account for that when parsing the filters in the image-listing endpoint.

Fixes: #18092
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-05-02 11:45:36 +02:00
c4b9f4b34e ginkgo-v2 cleanup workaround for #18180
Add a workaround for #18180 so the ginkgo work can be merged without
being blocked by the issue. Please revert this commit when the issue
is fixed.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:40 +02:00
5eb99a0ac8 test/e2e: fix custom timing reporting
This never worked when ginkgo runs with more than one thread, we use 3
in CI. The problem is that the SynchronizedAfterSuite() function accepts
two functions. The first one is run for each ginkgo node while the
second one is only run once for the whole suite.

Because the timings are stored as slice thus in memory we loose all
timings from the other nodes as they were only reported on node 1.
Moving the printing in the first function solves this but causes the
problem that the result is now no longer sorted. To fix this we let
each node write the result to a tmp file and only then let the final
after suite function collect the timings from all these files, then
sort them and print the output like we did before.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:40 +02:00
1bff0108f6 logformatter: anchors: link to test summary, not name
Super-complicated and ugly code merely to make in-page anchors
link three lines above the subtest name, to the subtest status
and timing line.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-02 11:27:40 +02:00
914ff01f94 WIP: logformatter: handle ginkgo v2 logs
Checkpoint as of Thursday April 13 afternoon.

Please do not merge with this; the .t file (tests) needs to
be updated, and there are probably many more fixes/improvements
I still need to make.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-02 11:27:39 +02:00
b3424f3708 test/e2e: unshare --rootless-netns cleanup slirp4netns
podman unshare --rootless-netns leaks the namespace and slirp4netns by
design as there is no safe way to remove it without any races.

To trigger a cleanup we can spin up a container and it will
automaticallt teardown the netns for us.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:39 +02:00
054154c77d test/e2e: run system reset test serial
USe the new ginkgo `Serial` decorator to make sure system reset is
never executed in parallel. system reset stops teh rootless pause
process which causes major issues when other process in parallel still
use this old namesapce.

Fixes #17903

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:39 +02:00
c5922cc6f0 test/e2e: fix CleanupVolume/Secrets()
Just like Cleanup() they should check the error codes.
While doing this it was clear that some volume tests were calling
Cleanup() twice so remove this.

Instead make sure they call Cleanup() themselves so callers only need to
do one call. This is required because we cannot use Expect().To() before
doing all the cleanup. An error causes panic does results in an early
return thus missing potentially important cleanup.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:39 +02:00
bc1ed07b56 ginkgo v2: fix new Skip() behavior
It looks like AfterEach() is now executed even after Skip(), this is a
good idea because the fact that it did't before caused us to leak tmp
directories. However in case Skip() is called before the podmanTest is
initialized it will no result in a panic. To fix it simply prevent such
panic by checking the pointer against nil and do nothing in such case.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:39 +02:00
cd46e72795 test/e2e: fix pause tests to unpause before cleanup()
The Cleanup() function tries to stop all containers, a paused contianer
cannot be stopped. The tests should make sure it works.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:38 +02:00
2ce4e935be ginkgo v2: drop localbenchmarks
Porting them over to v2 requires a full rewrite.
IT is not clear who actually uses these benchmarks, Valentin who wrote
them originally is in favor of removing them. He recommends to use
script from hack/perf instead.

This commit also drop the CI integration, it is not clear who actually
uses this data. If it is needed for something please speak up.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:38 +02:00
fb7a96638c test/e2e: switch to GinkgoWriter
Directly writing to stdout/err is not safe when run in parallel.
Ginkgo v2 fixed this buffering the output and syncing the output so it
is not mangled between tests.

This means we should use the GinkgoWriter everywhere to make sure the
output stays in sync.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:38 +02:00
cc19091e91 test/e2e: unset CONTAINERS_CONF before Cleanup()
If we do not unset CONTAINERS_CONF before tests that create a invlid
config will cause the Cleanup to fail.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:37 +02:00
8cc7a36183 ginkgo: run on all cores
Using -p autodetect the number of cores and thus spins up workers as
needed, this should help speeding up e2e tests, especially locally.

Also -vv for mor everbose logging.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:37 +02:00
9c1f713c75 test/e2e: fix Cleanup()
Only check exit codes last, othwerwise in case of errors it will return
early and miss other commands.
Also explicitly stop before rm, rm is not working in all cases (#18180).

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:37 +02:00
9bd833bcfd test/e2e: fix "podman run ipcns ipcmk container test"
The test will leak processes because the rm -fa in the cleanup failed.
This happens because podman tried to remove the contianers in the wrong
order and thus ppodman failed with:
`contianer XXX has dependent containers which must be removed before it`

For now I patch the test but it should be much better if we can fix it
in podman to remove in the correct order. `--all` should mean all I do
not care if there is a dependent container, just get rid of it.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:37 +02:00
629a6a6e45 test/e2e: actually check for cleanup errors
We blindy trust these commands to work but as it turns out they do not
under certain circumstances.

The "podman run ipcns ipcmk container test" can be used to fail this
reliably, if a container has dependencies the order of rm --all may
cause it to fail because the contianers are deleted in the wrong order.
This is th eonly one I found so far, adding this will uncover many more
of such problems without proper cleanup we leak processes and ginkgo v2
will block because of them.

Of course this cannot be merged without fixing these issues.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:36 +02:00
b9ba850440 Lower e2e timeout to not waste time when it hangs
Ginkgo v2 can result in the test just hanging at the end if there is
some bug which is leaking processes. The tests never take 90 mins
anyway. Just lower that time to 50 mins max. Usually they take between
30-40 mins so we should be fine with 50 mins.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:36 +02:00
fbb7c980aa test/e2e: containers.conf tests add missing Wait()
It is not clear why but without the wait is seems like the podman
process just hangs forever which now causes ginkgo to block until it
exits.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:36 +02:00
c564d9d7af ginkgo v2: remove CurrentGinkgoTestDescription()
This function is deprecated and replaced with CurrentSpecReport().
Also fix inconsitent callers.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:36 +02:00
666e314478 ginkgo v2: remove deprecated flags
- switch long flags to use `--`
- move the renamed flages noColor, flakeAttempts, outputdir
- remove no longer needed flag -progress

also see https://onsi.github.io/ginkgo/MIGRATING_TO_V2

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:35 +02:00
445815036f update to ginkgo v2
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:35 +02:00
867639e84d Merge pull request #18401 from eriksjolund/fix_troubleshooting_rootless_setup
[CI:DOCS] troubleshooting: fix subuid example
2023-05-02 05:21:21 -04:00
91da62d74c Merge pull request #18372 from Luap99/save-stdout
windows: podman save allow the use of stdout
2023-05-02 04:53:10 -04:00
5026545803 test/e2e: do not remove CNI directory
This is not safe at all when run in parallel, CNI needs that directory
to detect duplicated ips and also stores other important network info in
it. Removing it while container network is setup is not safe at all and
could cause a lot of weird flakes.

This "hack" was added in commit 55508c11 but provides zero context what
this was supposed to fix so I don't know what the actual issue is or was.

Fixes #18399

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 10:42:06 +02:00
3f5f906903 Merge pull request #18376 from Luap99/swagger-info
[CI:DOCS] swagger: fix Info name conflict
2023-05-02 04:39:58 -04:00
1c9d0330c5 Merge pull request #18395 from flouthoc/manifest-push-merge-dest
manifest, push: use `source` as `destination` if not specified
2023-05-02 04:17:22 -04:00
6e34fdb4b5 Merge pull request #18403 from edsantiago/login_logout_flakes
e2e: login_logout: use unique authfile for each test
2023-05-01 16:00:39 -04:00
7b25f362e6 e2e: login_logout: use unique authfile for each test
Set REGISTRY_AUTH_FILE to unique path for each subtest. This
should eliminate collisions where one test runs "podman logout"
just after another does "podman login".

Also, add a test to confirm that the authfile gets written
as expected.

Also, add actual tests for expected error messages, instead
of just ExitWithError()

Fixes: #18397

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-01 11:52:12 -06:00
a9a3b81ac2 Merge pull request #18388 from ashley-cui/uid
Fix clashing subuid
2023-05-01 13:51:54 -04:00
cd9a95922f Merge pull request #18359 from Luap99/machine-connection
machine: qemu only remove connection after confirmation
2023-05-01 13:07:56 -04:00
d06480d99e Fix clashing subuid
When initing a podman machine, we match core's UID to the UID of the user on the host. If the UID is in the subUID range, the machine throws an error. Check if the UID is within the default range for /etc/subuid (10000:1000000), and if it is, we adjust the range to not include it.

Signed-off-by: Ashley Cui <acui@redhat.com>
2023-05-01 10:39:54 -04:00
cda2b6e7ea Merge pull request #18379 from giuseppe/small-fixes
some small fixes
2023-05-01 10:36:34 -04:00
22047c6321 Merge pull request #18377 from containers/dependabot/go_modules/github.com/docker/docker-23.0.5incompatible
build(deps): bump github.com/docker/docker from 23.0.4+incompatible to 23.0.5+incompatible
2023-05-01 10:33:38 -04:00
24be46a517 [CI:DOCS] troubleshooting: fix subuid example
Fix incorrect number of required subuids in subuid example.

Fixes: https://github.com/containers/podman/issues/18400

Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
2023-05-01 16:03:04 +02:00
4dec42c6d8 Merge pull request #18396 from containers/renovate/github.com-moby-term-digest
Update github.com/moby/term digest to 0564e01
2023-05-01 10:01:06 -04:00