Commit Graph

1077 Commits

Author SHA1 Message Date
Ed Santiago
bd953fdc71 cirrus setup: special-case perl unicode
Perl is still stuck in the 1980s. Try to override that.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-10-19 09:30:10 -06:00
Chris Evich
106abd13be Cirrus: Execute Windows podman-machine e2e tests
Also, de-duplicate power-shell variables and functions as they're
beginning to sprawl.  This does not completely address all duplicates,
mainly those involved in the podman-machine CI workflow.  So,
nothing under `contrib/win-installer` has been touched.

[NO NEW TESTS NEEDED]

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-10-05 10:45:17 -04:00
OpenShift Merge Robot
f3aa35aa0e Merge pull request #20186 from cfergeau/gvproxy
Update mac installer to latest gvproxy release
2023-09-28 21:03:57 -04:00
Paul Holzinger
b284128d7b Revert "cirrus setup: install en_US.UTF-8 locale"
This reverts commit ed1f514d55.

The en_US.UTF-8 locale is now added in the images at build time,
https://github.com/containers/automation_images/pull/295

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-09-28 11:23:03 -06:00
Chris Evich
b28188de22 Cirrus: CI VM images w/ newer automation-library
The `v4.3.1` version of the library defines a common
`passthrough_envars()` so it doesn't need to be duplicated in podman and
buildah CI.  It also includes an update to build-push which should make
debugging easier.

Finally, these images include setting of the en_US.UTF-8 locale to enable
removal of a workaroud in a future commit.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-09-28 11:23:03 -06:00
Christophe Fergeau
58b5ae22e1 Update mac installer to latest gvproxy release
This brings DNS improvements/bug fixes, and docker.internal DNS entries.

Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
2023-09-28 14:52:04 +02:00
Paul Holzinger
0b5ea1e6ec test/e2e: default to netavark
When you run e2e tests locally they use CNI unless the NETWORK_BACKEND
env was set to netavark. Because our main focus is on netavark we should
test it by default.

For local tests this should help to prevent CNI/netavark conflicts as I
assume most systems where people run tests on are on netavark by now.

For  CI testing we hardcode NETWORK_BACKEND there to test both netavark
(on current fedora) and CNI (prior fedora). MAke sure to switch the
logic in the CI setup to reflect that.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-09-27 13:30:20 +02:00
OpenShift Merge Robot
ada67a2481 Merge pull request #20041 from lsm5/fcos-crun-wasm
[CI:BUILD] FCOS + podman-next image: pull in wasm
2023-09-21 07:34:55 -04:00
Lokesh Mandvekar
289be0c014 [CI:BUILD] FCOS + podman-next image: pull in wasm
This commit installs `crun-wasm` and `wasmedge-rt` in the FCOS image at
https://quay.io/repository/podman/fcos .

- crun-wasm is installed from rhcontainerbot/podman-next
- wasmedge-rt is installed from the official Fedora repos

Packages in Containerfile have also been rearranged in alphabetical
order.

[NO NEW TESTS NEEDED]

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2023-09-19 15:41:13 -04:00
Lokesh Mandvekar
fa71504518 [CI:BUILD] followup PR for fcos with podman-next
Followup on #19477

Remove commented out cirrus task for fcos image build with podman-next
and add 2 github actions: 1 for running a simple uni-arch image build
on every PR and another to actually build multiarch images and push to
quay after merge.

`podman --version` will also include git short sha for clarity.

[NO NEW TESTS NEEDED]

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2023-09-19 10:19:53 -04:00
Ed Santiago
ade0c4932a CI: trace setup and runner scripts
Every few months we get a new CI failure that requires
scanning through logs that give no indication of what
is happening or where. Tracking down the error can
cost many hours.

Solution: pepper cirrus scripts with showrun(), which echoes
the command _and_ displays the source filename + lineno.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-09-14 14:44:03 -06:00
Daniel J Walsh
b1e3e8d972 Run codespell on code
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-09-14 06:13:23 -04:00
Ed Santiago
7b91140238 Try to fix broken CI (gvisor-something)
Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-09-13 07:36:51 -06:00
Jason T. Greene
c2cd93cdc2 Split up alt binaries to speed up build
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
2023-09-08 23:57:20 -05:00
Jason T. Greene
7bf9f302a3 Switch installer task to EC2
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
2023-09-08 21:36:39 -05:00
Ed Santiago
afeab39d10 CI test runner: upgrade tests rely on system tests
Prevent future occurrences of #19894, by making upgrade tests
run any time there's a change to system tests. That's overly
broad: upgrade tests only rely on test/system/helpers.bash,
not test/system/anything-else. IMHO the cost of CI breaking
is higher than the cost of running unnecessary jobs.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-09-07 16:50:40 -06:00
Lokesh Mandvekar
50cd072bf6 [CI:BUILD] Podman FCOS image from main
Fixes: #19446, #19447, #19448

[NO NEW TESTS NEEDED]

Co-authored-by: Chris Evich <cevich@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2023-09-01 09:50:59 -04:00
Chris Evich
99834947dc Cirrus: Remove multi-arch podman image builds
These jobs have been failing since early August due to
technical/scripting problems.  Disable/remove entirely since a fix is
unlikely to be implemented anytime soon.

Ref: Abandoned recent attempt at debugging
https://github.com/containers/podman/pull/19720

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-08-28 16:15:20 -04:00
Chris Evich
ecf9f9fd8b Cirrus: Disable only hello multiarch build
The `contrib/hello` directory is needed for some CI tests here and in
buildah.  Further, the build job that produces images from this
directory are currently broken.  Disable the build job, and updte
the README.md to point people at the replacement repo. for this content.

Ref: https://github.com/containers/podman/pull/19730

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-08-24 16:07:25 -04:00
Ed Santiago
70f77efef9 Revert "Remove hello multi-arch image build"
This reverts commit 7e0130f75c.

It broke CI, not just on podman but on Buildah too. Buildah bud
tests require the hello subdirectory.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-08-24 05:16:01 -06:00
Chris Evich
7e0130f75c Remove hello multi-arch image build
Moved to https://github.com/containers/PodmanHello

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-08-23 12:10:22 -04:00
Paul Holzinger
ed1f514d55 cirrus setup: install en_US.UTF-8 locale
Make sure the en_US.UTF-8 locale is available so that we can use it in
tests, namely "podman logs with non ASCII log tag succeeds with env".

It is already there in fedora (except container image but we cannot use
journald there anyway) so only do this for debian. I think it makes
most sense to move this into the image build process in the future to
only do it once at build time.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-08-17 15:31:56 +02:00
Peter Hunt
9b4f1cdb97 cirrus/lib.sh: extend env to passthrough at start for locale work
Signed-off-by: Peter Hunt <pehunt@redhat.com>
2023-08-17 12:15:08 +02:00
Chris Evich
d79e3a23db CI FIXME removal/update.
Ref: https://github.com/containers/automation_images/pull/288/commits

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-08-08 14:34:24 -04:00
Dan Čermák
b46d72fece [ci] Remove the podman socket in remove_packaged_podman_files()
The socket can already exist and is not removed by disabling the systemd unit,
hence it needs to be deleted to get a "vanilla" system. This is a temporary
workaround suggested in:
https://github.com/containers/podman/pull/19478#issuecomment-1665350945

[NO NEW TESTS NEEDED] - this is a CI fix

Signed-off-by: Dan Čermák <dcermak@suse.com>
2023-08-04 16:30:55 +02:00
Dan Čermák
706c51b52b [ci] Correct the podman systemd file names
[NO NEW TESTS NEEDED] - this is a CI fix

Co-authored-by: Chris Evich <cevich@redhat.com>

Signed-off-by: Dan Čermák <dcermak@suse.com>
2023-08-04 16:24:58 +02:00
Jason T. Greene
2f21df7b3a Remove legacy msitools based msi installer
This was replaced by the setup.exe burn installer several releases ago,
and only kept around as a fallback. Remove it since it is no longer
maintained and not recommended for use.

Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
2023-07-24 15:21:30 -05:00
Daniel J Walsh
d4fc66af9a Clean up /var/tmp/ when using oci-archives when creating
containers

We need to remove /var/tmp/container_images_* and
/var/tmp/container_images_* which are podman temporary directories on each
boot which are created when creating containers from oci-archive tarballs
or other pull operations.

Signed-off-by: Joe Doss <joe@solidadmin.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-07-24 10:34:38 -04:00
Jason T. Greene
3b63432027 Fix windows installer
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
2023-07-18 00:05:21 +01:00
Paul Holzinger
499b8d13c5 CI: remove build without cgo task
Podman is basically unusable without cgo, checking if it compiles
without adds no value and just tricks people into thinking it works when
it does not.

This means we do not need extra to NOP out a lot of cgo calls with
functions that just return an error like `XXX is not supported without
cgo`.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-07-10 13:33:03 +02:00
Doug Rabson
e4644c3cb8 Cirrus: build FreeBSD binaries in a VM
Using a FreeBSD VM allows building a functioning binary via 'make
podman-release'. This uses Cirrus' freebsd_instance which provisions VMs
on Google Compute Engine.

I attempted to add FreeBSD binaries to the artifacts task but this
failed, apparently because the extra 50Mb of space exceeded a 1Gb limit
in Cirrus.

[NO NEW TESTS NEEDED]

Signed-off-by: Doug Rabson <dfr@rabson.org>
2023-07-03 16:19:43 +01:00
Ed Santiago
de494eb05a logformatter: ignore 'TOP-LEVEL' headings
When an e2e test fails in AfterEach, ginkgo "helpfully" adds
a heading in that test log block:

  TOP-LEVEL [AfterEach]
  /path/to/source.go
    Podman Desc Blah

That TOP-LEVEL line screws up our in-page links. Ignore it
both in the heading and in the bottom failure-summary lines.

Add test, including tests for bottom-summary.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-06-13 09:07:23 -06:00
Aditya R
02432fc692 cirrus,ci: default to overlay for debian env
In debian environment we are hitting an edge-case where older buildah
version is not compatible with newer podman version because both of them
are using different storage driver.

I.e
* Podmand defaults to native `overlay`.
* Older buildah version defaults to `vfs`.

See discussions below for more details
* containers#18510 (comment)

Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Aditya R <arajan@redhat.com>
2023-06-09 10:43:58 +05:30
Ed Santiago
137c8ef4d5 logformatter: better recognition of ginkgo test names
Ginkgo test names can have more than two levels: there can be
a nested series of Describes() before the final It(). (e.g.,
quadlet_test.go). Handle that.

Before: we just assumed that the third-or-maybe-fourth line
after a "-----" divider was the test name.

Now: examine every line after the "-----" divider, until the
first empty line. Lines with /path/to/source/file are ignored,
lines with text strings are assembled together to make anchors.
This is still imperfect but it's much better than before.

SPECIAL NOTE: in order to allow linking to timing results
in the AfterSuite, I've changed the test name from Leaf to Full.
This will now be a much longer string, and hence much less
readable, but I'm inclined to think it's more correct. Please
review carefully and lmk if I should revert.

Finally, as an unrelated add-on, add links (at top) to original
log, journal, and (if applicable) podman-remote server logs.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-06-06 14:32:15 -06:00
Ed Santiago
c51c6675ee logformatter: proper status color for failed tests
Fix an oops in my ginkgo-v2 handling: subtest-status was
getting re-reset back to "passed", resulting in incorrect
and misleading green titles on failed tests.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-06-05 05:59:11 -06:00
Lokesh Mandvekar
e6bfc81a8f RPM: bump gvisor-tap-vsock subpackage and fix packit scripts
gvisor-tap-vsock:
- bump to v0.6.1
- installs /usr/libexecdir/podman/gvforwarder

packit:
- fix pre-sync action in propose-downstream
Ref: https://dashboard.packit.dev/results/propose-downstream/2581

cleanup:
- remove `contrib/spec/python-podman.spec.in`. No longer needed.

[NO NEW TESTS NEEDED]

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2023-05-26 14:32:52 -04:00
OpenShift Merge Robot
d7b811677e Merge pull request #18652 from anjannath/update-qemu
pkginstaller: bump Qemu to version 8.0.0
2023-05-23 07:11:14 -04:00
Ed Santiago
94c65a659c TEMPORARY(?) instrumentation for unlinkat-ebusy
Instrument system tests in hopes of tracking down #17216,
the unlinkat-ebusy-hosed flake.

Oh, also, timestamp.awk: timestamps have always been UTC, but
add a 'Z' to make it unambiguous.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-22 10:34:37 -06:00
Anjan Nath
d623670e1a pkginstaller: bump Qemu to version 8.0.0
this updates the Makefile qemu version to make use new qemu released at
https://github.com/containers/podman-machine-qemu/releases/tag/v8.0.0-1

[NO NEW TESTS NEEDED]

Signed-off-by: Anjan Nath <kaludios@gmail.com>
2023-05-22 21:43:21 +05:30
OpenShift Merge Robot
abea786b15 Merge pull request #18545 from cevich/podman_next
Cirrus: Add support for podman-next magic
2023-05-18 06:50:20 -04:00
Chris Evich
99cedae317 Cirrus: Record the buildah version for reference
Apparently this matters, see
https://github.com/containers/podman/pull/18510#discussion_r1189812306

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-05-17 11:30:13 -04:00
Chris Evich
d1dcb0846b Cirrus: Add support for [CI:NEXT]
Rather than supporting a special-mode *just* for netavark/aardvark
testing [in podman CI], support testing with all the latest
`podman-next` COPR packages.

The idea here is very similar to the netavark/aardvark special mode it
replaces.  Most podman-dependencies do not have the level of
comprehensive CI as exist here.  This new CI-mode allows testing
upstream updates to podman-dependencies without needing to roll out
a whole new package/release for them.

Also update documentation for this new mode.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-05-16 12:18:00 -04:00
Chris Evich
eb4f0bc17d Cirrus: Remove support for [CI:COPR] magic
This magic string isn't often used and may cause confusion with future
magic-string additions.  Remove it.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-05-16 12:17:59 -04:00
Ed Santiago
3360214a0f system tests: add precision timestamps
In run_podman(), display a nanosecond-level timestamp next to
each command and its output.

Because this clutters the results, teach logformatter to grok
these new timestamps, strip them, and display a more human-readable
time delta in the left-hand timestamp column. logformatter started off
as a mess and is now, well, 🤮. I'm sorry. I just hope its results
make it worthwhile.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-16 09:34:33 -06:00
Ed Santiago
cb8e19677f logformatter: handle podman-machine test logs
With -p, ginkgo emits timing information on the first line
after dashes. Without -p, it's at the end.

Most tests use -p. Podman-machine does not. Deal with that.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-09 11:08:59 -06:00
Ed Santiago
f877d7dcd0 Replace egrep/fgrep with grep -E/-F
There are days when I really, really, really hate GNU. Remember
when someone decided that 'head -1' would no longer work, and
that it was OK to break an infinite number of legacy production
scripts? Someone now decided that egrep/fgrep are deprecated,
and our CI logs (especially pr-should-include-tests) are now
filled with hundreds of warning lines, making it difficult
to find actual errors.

I expect that those warnings will be removed quickly after
furious community backlash, just like the 'head -1' fiasco
was quietly reverted, but ITM the warnings are annoying
so I capitulate.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-03 07:32:42 -06:00
Ed Santiago
ae5305134f Revert "logformatter: anchors: link to test summary, not name"
It looks like tell/seek don't work in CI-land: important test log
information is being lost. Revert this commit, maybe some day I'll
come up with a better solution.

This reverts commit 1bff0108f6.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-02 10:03:29 -06:00
Ed Santiago
1bff0108f6 logformatter: anchors: link to test summary, not name
Super-complicated and ugly code merely to make in-page anchors
link three lines above the subtest name, to the subtest status
and timing line.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-02 11:27:40 +02:00
Ed Santiago
914ff01f94 WIP: logformatter: handle ginkgo v2 logs
Checkpoint as of Thursday April 13 afternoon.

Please do not merge with this; the .t file (tests) needs to
be updated, and there are probably many more fixes/improvements
I still need to make.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-05-02 11:27:39 +02:00
Paul Holzinger
2ce4e935be ginkgo v2: drop localbenchmarks
Porting them over to v2 requires a full rewrite.
IT is not clear who actually uses these benchmarks, Valentin who wrote
them originally is in favor of removing them. He recommends to use
script from hack/perf instead.

This commit also drop the CI integration, it is not clear who actually
uses this data. If it is needed for something please speak up.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-05-02 11:27:38 +02:00