Commit Graph

649 Commits

Author SHA1 Message Date
Ed Santiago
e39ddde709 Revert "Reapply "CI: test nftables driver on fedora""
This reverts commit 0e66a793bc.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-11-08 05:09:54 -07:00
Ed Santiago
0e66a793bc Reapply "CI: test nftables driver on fedora"
Temporary, until we get CI VMs with kernel 6.11.6.

I've lost track of where this is being discussed.

This reverts commit 7f836df303.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-11-06 11:03:35 -07:00
Paul Holzinger
ef20e75a77 cirrus: check consitent vendoring in test/tools
Ensure nobody modifies files directly there.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-11-01 13:23:01 +01:00
Paul Holzinger
bd75a4e0e0 logformatter: add cleanup tracer log link
I had a few cases where this is super handy in CI and if I can access the
link without one extra click it safes me some time and hopefully makes
it more discoverable for other to see how useful it can be sometimes.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-10-29 19:35:01 +01:00
Ed Santiago
b3302e8e18 Housekeeping: remove duplicates from success_task
Accidentally introduced in #21639.

Thanks to Paul for the Python code to prevent this from
happening again.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-10-15 10:09:15 -06:00
openshift-merge-bot[bot]
11ab0b7246 Merge pull request #24120 from Luap99/cirrus-build-speed
cirrus: use fastvm for builds
2024-10-14 16:04:12 +00:00
Paul Holzinger
7f836df303 Revert "CI: test nftables driver on fedora"
This reverts commit 43f6173cc6.

The netavark version with nftables default is in f41 and rawhide
already so this is no longer needed. While we do not yet test f41 in CI
we have rawhide which is good enough until we update.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-10-11 13:13:53 +02:00
Paul Holzinger
9e35fead03 cirrus: use zstd over bzip2 for repo archive
The repo tar process took over 1:20 min, with zstd it takes less than
10s so we safe over a minute by doing this.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-10-11 11:25:34 +02:00
Paul Holzinger
7eb4a6fe2b cirrus: speed up postbuild
In particular the main build task already did a make vendor and a
regeneration of the completion scripts. This means the first tre_status
would pick up both changes so the suggestion would be off. And rerunning
the same thing again here just makes thing slower than they need to be.
In particular there was the bug that make completion even rebuild podman
because generate-bindings obviously updates the timestamps of the files
as they are overwritten.

We do however must run generate-bindings as it was not run before.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-10-11 11:25:33 +02:00
Paul Holzinger
04dffbc2c7 cirrus: change alt arch task to only compile binaries
The current podman-release-%.tar.gz target does a lot more then just
checking if we can build for the given arch, in particular it first
builds a local podman-remote for the remote-docs.sh script. This makes
things slow as we compile several things and then builda and package the
docs. Given the docs are not arch specific there is realy no point in
doing all that work. All we care about is if the bianries can build on
other arches to catch compile issue for otherwise untested arches.

This should make the CI Alt Arch. tasks much faster.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-10-11 11:25:33 +02:00
Paul Holzinger
9082a14d76 cirrus: run make with parallel jobs where useful
The doc generation and the validate-binaries target can be run in
parallel as they do not depend on each other and a specific ordering. As
such we pass -j $(nproc) but also --output-sync=target to ensure the
output is not intermixed between several targets which could be harder
to read in case of errors.

Hower dus the complex podman-release target we can run podman-release
and validate-binaries at the same time as the dependencies are not right
and we run podman-release first in order to get the correct binaires
build.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-10-11 11:25:33 +02:00
Paul Holzinger
09ba5db035 CI: run and collect cleanup tracer logs
In order to get better debug data for cleanup flakes. The argv is
printed with 0 bytes so replace them with spaces to make the log
readable for humans.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-09-24 12:47:04 +02:00
Ed Santiago
6502e30cfd bats log: differentiate parallel tests from sequential
For tests run in parallel, show file number as |nnn| (vs [nnn])

Teach logformatter to distinguish the two, adding 'p' to anchors
in parallel tests. Necessary because in this scheme we run bats
twice, thus see 'ok 1' twice, and we want to differentiate them.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 11:19:37 -06:00
Ed Santiago
359092e463 Bump VMs. ShellCheck is now built-in
Minor bump. Fedora VMs now include ShellCheck, so we can
remove the 'dnf install' at CI run time.

Also, FWIW, Debian *vark are now at 1.12 (from 1.9)

VMs built in https://github.com/containers/automation_images/pull/385

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-09 06:16:20 -06:00
Paul Holzinger
e598bcafb8 cirrus: prebuild use f40 for extra tests
This check has a condition on the distro name to only run once, however
the prior fedora version doesn't have to exists necessarily as we might
have to drop support there due the outdated golang version.
The current fedora version should alway exists so this seems safer.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-09-03 15:13:42 +02:00
Paul Holzinger
6a0ab6f7bc cirrus: remove _bail_if_test_can_be_skipped
Since commit 55ad0d6e0e we do the conditions in the cirrus.yml directly
so there is no longer any need for this.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-08-29 18:59:47 +02:00
Paul Holzinger
d5c5261e6f cirrus: move renovate check into validate
The renovate config is used for the renovate bot, validating this in the
prior fedora prebuild setp is just confusing and hidden.

The problem is this image is very big so it is slow to download/extract.
To speed things up given it is only a single file we check the diff if
we even changed it.

Now one could argue this should be part of the validate Makefile target
but I given the size I do not want this run by default and I am not sure
if we should do the diff check in the Makefile.

Lastly remove -it, these is meant for interactive use and throws a
warning here because we have no actual tty attached.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-08-29 18:59:47 +02:00
Paul Holzinger
ac18b1a0af cirrus: remove 3rd party connectivity check
This doesn't help us at all, first the list is outdated. AFAICT we no
longer connect to docker.io, registry.fedoraproject.org or
podman.cachix.org (seems to be a cache site for nix.dev?) anywhere in
our tests.

Second a simple port check is not helpful, in the most cases the
CDN's and or load balancer accept connections but return internal server
errors when the registy goes down.

This is very similar to commit 5b6de98ee8.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-08-29 18:59:46 +02:00
Paul Holzinger
34a7d8dd10 cirrus: remove cross jobs for aarch64 and x86_64
We do build and test aarch64 and x86_64 natively so the cross job
doesn't seem to add value.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-08-29 18:59:46 +02:00
Alexander Larsson
17193af962 CI: On vX.Y-rhel branches, ensure that some downstream Jira issue is linked
In the RHEL specific branches we want to ensure that all MRs link to
at least one downstream Jira ticket. To do this we add a new test in
validate-source similar to the existing pr-should-include-tests. This
test only runs on actual pull requests.

The syntax for linking to a Jira is "Fixes " or "Fixes: ", followed by
one jira links, like so:

```
Fixes https://issues.redhat.com/browse/RHEL-50506
Fixes: https://issues.redhat.com/browse/RHEL-50506
```

Note: This is the same syntax as for a regular github issue reference.

Signed-off-by: Alexander Larsson <alexl@redhat.com>
2024-08-22 11:53:25 +02:00
Ed Santiago
5b6de98ee8 CI: remove build-time quay check
CI will fail if quay is down, but a build-time check does not
help us in any way. It just introduces another pain point
where we have to hit the Rerun button.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-08-14 08:17:45 -06:00
openshift-merge-bot[bot]
c3111c24c1 Merge pull request #23593 from cevich/fix_validate_renovate
[CI:ALL] Fix and validate renovate config
2024-08-12 19:08:03 +00:00
openshift-merge-bot[bot]
bd53a11630 Merge pull request #23225 from edsantiago/no-more-ci-docs
pr-should-include-tests: no more CI:DOCS override
2024-08-12 18:46:02 +00:00
Chris Evich
8f191618e4 Validate renovate config in every PR
Signed-off-by: Chris Evich <cevich@redhat.com>
2024-08-12 14:10:28 -04:00
Chris Evich
dcdb5408de De-duplicate docker-py testing
Previously there were two CI tasks that ended up both testing docker-py
compatibility.  Remove the duplicate from the `localapiv2-python` make
target, and symlink the identical requirements file.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-08-09 10:13:02 -04:00
Chris Evich
bf091abdc5 Fix not testing registry.conf updates
Previously, if anyone touched these files no extra testing would
trigger.  However, basically all testing depends on them.  Update the
condition and test that verifies it.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-08-08 10:40:22 -04:00
Nicola Sella
f041d059b5 Add passwd validate and generate steps
Add generate helper function.
Also, add a troubleshooting try/catch block in case we get more flakes
during Set-LocalUser step in Windows powershell.

Resolves: https://github.com/containers/podman/issues/23468

Signed-off-by: Nicola Sella <nsella@redhat.com>
2024-08-08 11:50:17 +02:00
Ed Santiago
87dad1caef CI: podman-machine: do not use cache registry
It's too difficult to keep the podman-machine image up-to-date.
And, we can't use the cache on Mac/Windows, so if quay is down
we're hosed no matter what.

Add a "nocache" mechanism to install_test_configs() and use that
in machine test setup.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-08-05 05:12:36 -06:00
openshift-merge-bot[bot]
69a5b28e6e Merge pull request #23455 from lsm5/main-5.3-bump
Version: bump to v5.3.0-dev
2024-08-03 02:20:30 +00:00
Chris Evich
029a9aa34e Add libkrun Mac task
Signed-off-by: Chris Evich <cevich@redhat.com>
2024-08-01 15:01:45 -04:00
Lokesh Mandvekar
fcf7cd7d5c Version: bump to v5.3.0-dev
We have v5.2 branch now.

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2024-07-31 13:30:35 -04:00
Ed Santiago
7bb3b83c17 CI: enable root user namespaces
Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-07-27 23:23:29 +02:00
openshift-merge-bot[bot]
2015137eee Merge pull request #23367 from rhatdan/codespell
Run codespell on source
2024-07-23 23:11:32 +00:00
openshift-merge-bot[bot]
b005b13274 Merge pull request #23328 from l0rd/win-installer-update-bug
Fix windows installer deleting machine provider config file
2024-07-23 13:14:46 +00:00
Daniel J Walsh
7768cf235e Run codespell on source
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2024-07-23 07:28:23 -04:00
Ed Santiago
a05a9cae07 logformatter: tweaks to pass html tidy
Plus, I think my ampersand-quot change earlier this month
caused problems for firefox. We no longer need it (pull-option
does not need the funky double-quoted curly-brace string),
so, remove it.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-07-22 07:58:32 -06:00
Mario Loriedo
2e4b640ad2 Fix windows installer deleting machine provider config file
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-07-18 16:56:39 +02:00
Giuseppe Scrivano
b1e535b911 contrib: use a distinct --pull-option= for each flag
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2024-07-12 12:51:39 +02:00
openshift-merge-bot[bot]
360c4f372d Merge pull request #23234 from Luap99/test-nftables
test netavark nftables driver
2024-07-11 22:19:32 +00:00
openshift-merge-bot[bot]
58c8803a1e Merge pull request #22726 from edsantiago/pull-from-local-registry
CI: Use local cache registry
2024-07-11 12:42:04 +00:00
Paul Holzinger
43f6173cc6 CI: test nftables driver on fedora
Make sure this passes podman CI before we push out a default change.

ref: https://fedoraproject.org/wiki/Changes/NetavarkNftablesDefault

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-07-11 14:08:27 +02:00
openshift-merge-bot[bot]
e570aacc4e Merge pull request #23212 from l0rd/vs-buildtools
Using Visual Studio BuildTools as a MinGW alternative
2024-07-11 11:25:03 +00:00
Ed Santiago
07b69943a4 CI: use local registry, part 3 of 3: for developers
New tool, get-local-registry-script, intended for developers
to get a local registry running in their environment. This is
not necessary for any tests, but may be desirable for performance
reasons and/or to recreate the CI environment.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-07-11 04:39:45 -06:00
Ed Santiago
dd1bcabae9 CI: use local registry, part 2 of 3: fix tests
This commit gets tests working under the new local-registry system:

  * amend a few image names, mostly just sticking to a consistent
    list of those images in our registry cache. Mostly minor
    tag updates.

  * trickier: pull_test: change some error messages, and remove
    a test that's now a NOP. Basically, with a local (unprotected)
    registry we always get "404 manifest unknown"; with a real
    registry we'll get "403 I can't tell you".

  * trickiest: seccomp_test: build our own images at run time,
    with our desired labels. Until now we've been pulling
    prebuilt images, but those will not copy to the local
    cache registry. Something about v1? Anyhow, I gave up
    trying to cache them, and the workaround is straightforward.

Also took the liberty of strengthening a few error-message checks

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-07-11 04:39:45 -06:00
Ed Santiago
2e8c074234 CI: use local registry, part 1 of 3: setup
As of https://github.com/containers/automation_images/pull/357
our CI VMs include a local registry preloaded with all(*)
images used in tests.

 * where "all" means "most".

This commit installs a new registries.conf that redirects docker
and quay to the new local registry. The hope is that this will
reduce CI flakes.

Since tests change over time, and new tests may require new
images, this commit also adds a mechanism for pulling in
remote images at test run time. Obviously this negates
the purpose of the cache, since it introduces a flake
pain point. The idea is: DO NOT DO THIS UNLESS ABSOLUTELY
NECESSARY, and then, if we have to do this, hurry up and
spin new CI VMs that include the new image(s).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-07-11 04:39:45 -06:00
Ed Santiago
d4c0e7ecbd CI: test composefs on rawhide
Run root e2e & system tests using composefs on rawhide.

Write magic settings to storage.conf. That part is easy.

e2e tests, however, ignore storage.conf. They require everything
to be specified on the command line. And "everything", in the
case of composefs, includes a long complicated --pull-options
string which in turn requires containers-storage PR 1966
which, as of this writing, is finally vendored into podman.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-07-10 14:51:58 -06:00
Chris Evich
1dd8a35b2f Drop minikube CI test
This test flakes frequently and its status is completely ignored in CI.
At the time of this commit, nobody has stepped up to debug or fix it.
Drop the test.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-07-09 11:30:00 -04:00
Paul Holzinger
4b7aaafb2d cirrus: check for header files in source code check
When we check if source code was changed also include header files.
There is only one header file currently but that can change and it may
be possible that changes in this file can break things so make sure it
is considered source code so that all tests are triggered.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-07-09 13:34:40 +02:00
Ed Santiago
5e9ab00063 pr-should-include-tests: no more CI:DOCS override
CI:DOCS is no more.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-07-08 15:46:56 -06:00
Mario Loriedo
81250cc5a9 Visual Studio BuildTools as a MinGW alternative
Building the MSI hook on Windows
(`contrib/win-installer/podman-msihooks/check.c`)
currently requires MinGW. This commit updates the build
script so that, when MinGW is absent but the C compiler
included in Visual Studio BuildTools is installed, the
latter is used to build the MSI hook.

Other than that, `winmake.ps1` has a new `installertest`
target to run the Windows installer tests that are
currently verified by Cirrus CI.

Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-07-08 17:18:15 +02:00