Commit Graph

1326 Commits

Author SHA1 Message Date
Chris Evich
c4e68bdae4 Cirrus: Reuse shared clone script on Mac
The `Makefile` makes assumptions about git repo. clone details.
Because fixing the `Makefile` would likely be problematic, fix
the clone operation used on the Mac so that it matches what's used
on Linux.

Also, simplify git repo clone operations.  At some point in the
distant past, a git identity was required for CI to function properly.
That has since changed, so remove the unnecessary complexities.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-21 15:20:16 -05:00
Chris Evich
f7d1726f93 Rearrange CI tasks for safety + efficiency
There's are sometimes conflicting purposes in podman CI:

1. Have the pipeline proceed in an orderly and progressive manner
   to sometimes save resources and unnecessary runtime.
2. Complete all testing as quickly as possible in support of
   human-developers moving on to other areas of work.
3. Ideally/hopefully, accomplish both items above safely,
   preventing untested and/or unintended changes from merging.

This commit shifts the balance of these slightly more toward the second
point.  It rearranges most CI tasks into essentially three buckets with
a single (new) aggregation task in-between the first two:

1. Build + Verify all the things
2. Test all the things
3. Minor/accessory things

The intention is that while we may unnecessarily spin some number of
testing tasks while others have failed, the best-case scenario
(everything passes) has a much shorter runtime.  In other words, it
potentially wastes more resources in favor of a chance to have
developers wait less.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-21 13:27:16 -05:00
openshift-merge-bot[bot]
f756e5db68 Merge pull request #21747 from mheon/windows_lint
Fix Lint on Windows and enable the job
2024-02-21 00:47:04 +00:00
Matt Heon
ec68f07c04 Fix Lint on Windows and enable the job
[NO NEW TESTS NEEDED] Purely refactoring

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-20 08:06:18 -05:00
Paul Holzinger
66fd41ff19 cirrus logcollector: update package list
aardvark-dns, netavark and passt are installed on both debian and
fedora. cri-o-runc is not installed anymore and it just uses the normal
runc package on debian. containers-common is called
golang-github-containers-common on debian and also uses
golang-github-containers-image for further config files from c/image.
This makes sure we correctly log all the package versions on debian
correctly.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-20 13:58:57 +01:00
openshift-merge-bot[bot]
9fc45088cb Merge pull request #21587 from cevich/fix_podman_remote_image
[CI:DOCS] Remove disused Containerfile and docs
2024-02-20 09:46:31 +00:00
Chris Evich
068d4e81c7 Allow CI user to cleanup own files
According to https://go.dev/ref/mod#module-cache golang will leave
behind read-only bits.  It was observed that these cause the find/rm
cleanup operations to fail fail with `permission denied` on thousands
of files.  This is preventing cleanup of cruft from unrelated Cirrus-tasks
leading to unnecessary occupation of critical, local-ssd storage space.
Fix this by ensuring the user has at least write access to the entire
contents of `$TMPDIR` and `$HOME`, `ci` subdirs.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-14 10:29:51 -05:00
Ashley Cui
87947761ed [CI:BUILD] Add VFKit into pkginstaller, remove QEMU
Do not build and install QEMU in the pkginstaller. Instead, build, sign, and install VFKit.

Signed-off-by: Ashley Cui <acui@redhat.com>
2024-02-13 10:20:03 -05:00
openshift-merge-bot[bot]
fcd69c997e Merge pull request #21542 from jakecorrenti/bump-gvproxy
Enable passing logfile to gvproxy
2024-02-12 21:49:41 +00:00
Chris Evich
00180bd5b3 Enforce podman-machine mac CI results
Followup to https://github.com/containers/podman/pull/21551

Note: Fixed indentation of podman-machine mac task.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-12 09:28:23 -05:00
openshift-merge-bot[bot]
e16d82dbac Merge pull request #21441 from cevich/win_lint
Windows-cross linting
2024-02-09 18:08:01 +00:00
Chris Evich
69163af2a0 Remove gitleaks scanning
Ref:
https://github.com/containers/podman/pull/21570#issuecomment-1935709148

This tool is really intended/best used from git pre-commit on developers
local machines, to prevent addition of secret leaks.  When used as a
check against PRs, it tends to turn up more false-positives than helpful
warnings.  There's no good way to fix this, and maintaining the scanner
is an additional burden.  Rather than continue struggling to improve/fix
the situation, let's just remove the tool entirely.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-09 10:09:42 -05:00
Chris Evich
5b6b689764 Remove disused Containerfile and docs
Fixes: #21574

The documentation for this image references a quay repository that
doesn't exist.  It doesn't appear any of these files have been touched
since late 2022. Instead of updating the docs, let's just remove the
source.  It's trivial to recreate if anybody actually needs it for
something.

Users needing to access remote podman can simply use the `podman` binary
present in existing images `quay.io/containers/podman`,
`quay.io/podman/stable`, etc.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-09 10:02:27 -05:00
Jake Correnti
69d54c482e Bump gvisor-tap-vsock from 0.7.2 to 0.7.3
Bump the gvisor-tap-vsock version that podman uses from 0.7.2 to 0.7.3

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-08 08:31:50 -05:00
Ed Santiago
e20b70c2e6 Upgrade tests: reenable, but revamped
No longer bother testing any 2.x or 3.x. Only 4.1 and above.

Remove all CNI-related code. CNI is gone.

Add DatabaseBackend tests, confirming that we can handle
both boltdb and sqlite.

Require BATS >= 1.8.0, and use "run -0" to do exit-status checks.

Update docs.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-02-08 06:22:56 -07:00
Chris Evich
43a01d7dc2 Temporarily ignore windows-linting failures
As of this commit, there are several pages worth of lint findings for
windows.  Once they're all addressed, this commit may be reverted to
enable continuous checking.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-06 16:28:38 -05:00
Chris Evich
defbf42376 Lint before windows-cross build
Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-06 16:28:38 -05:00
Lokesh Mandvekar
01a8539ef0 Warn if cgroups-v1
Podman v5 will not support cgroups-v1. This commit will print a warning
if it detects a cgroups-v1 system. The warning can be hidden by setting
envvar `PODMAN_CGROUPSV1_WARNING`.

This warning is patched out for RHEL 9 builds as cgroups-v1 will still
be supported on RHEL 9 systems.

Resolves: https://issues.redhat.com/browse/RUN-1957

[NO NEW TESTS NEEDED]

Co-authored-by: Ed Santiago <santiago@redhat.com>
Co-authored-by: Sascha Grunert <sgrunert@redhat.com>
Co-authored-by: Giuseppe Scrivano <gscrivan@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
2024-02-06 16:18:05 +05:30
Ashley Cui
26cd01ee51 Do not test CNI in CI
CNI is deprecated and is build tagged out for 5.0. Don't test it in our CI.
This commit also disables upgrade tests for now - those need more work since the old version of Podman only uses CNI. Upgrade tests will be re-vamped in a later commit.
Signed-off-by: Ashley Cui <acui@redhat.com>
2024-02-05 11:01:21 -05:00
Ed Santiago
1ab55defbb Test new CI VMs
From https://github.com/containers/automation_images/pull/325

Major change: netavark and aardvark are now included in prior-fedora,
so CNI can be fully eliminated from CI (#21410)

FIXME FIXME FIXME: skip two e2e tests, waiting for new netavark

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-02-01 09:50:41 -07:00
openshift-merge-bot[bot]
0655bf3d34 Merge pull request #21445 from anjannath/update-qemu
pkginstaller: bump Qemu version to 8.2.1
2024-01-31 03:29:32 +00:00
openshift-merge-bot[bot]
2632edc871 Merge pull request #21408 from cevich/re-enable_pm_win
Revert "Enable win podman-machine test failure"
2024-01-30 20:59:10 +00:00
Anjan Nath
76791a21c6 pkginstaller: bump Qemu version to 8.2.1
this updates the Makefile qemu version to use the qemu
8.2.1 build released at:
https://github.com/containers/podman-machine-qemu/releases/tag/v8.2.1-1

[NO NEW TESTS NEEDED]

Signed-off-by: Anjan Nath <kaludios@gmail.com>
2024-01-31 01:34:47 +05:30
Chris Evich
0ec84b8ec5 Revert "Enable win podman-machine test failure"
This reverts commit f9e8585c53.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-01-29 09:30:04 -05:00
Daniel J Walsh
8d14d41555 Run codespell on code
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2024-01-28 07:30:52 -05:00
Christophe Fergeau
c566551e96 gvproxy: Update to 0.7.2 release
This updates the Windows installer and macOS installer to ship the newer
binaries, as well as the vendored gvisor-tap-vsock code.

[NO NEW TESTS NEEDED]

Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
2024-01-17 16:59:35 +01:00
Łukasz Stolcman
e51bab3c97 Remove duplicated content in docs
Signed-off-by: Łukasz Stolcman <lukasz.stolcman@protonmail.com>
2023-12-15 23:54:03 +01:00
Łukasz Stolcman
7c72d8c2f3 Fix path to example files
Signed-off-by: Łukasz Stolcman <lukasz.stolcman@protonmail.com>
2023-12-15 00:37:53 +01:00
Chris Evich
95eb22c71c Support a machine-test only mode
Because the podman machine tests normally run at the end of the chain,
it's time consuming for developers to validate machine-only changes.
Support a special CI runtime mode, only when a PR is marked as a draft.
Update related documentation

Also: Only run machine tests on PRs, never for branches or new tags.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-12-11 11:02:32 -05:00
openshift-merge-bot[bot]
d513749085 Merge pull request #20914 from baude/machinehypervregression
Fix regression in e2e machine test suite
2023-12-08 16:16:24 +00:00
Lokesh Mandvekar
f9b2da3eb7 [CI:BUILD] override crun-wasm in fcos + podman-next image build
crun-wasm depends on the same epoch:version-release as crun so
overriding `crun` but not `crun-wasm` will cause installation issues
like:
```
error: Could not depsolve transaction; 1 problem detected:
 Problem: package crun-wasm-1.11.1-1.fc39.x86_64 from @System requires crun = 1.11.1-1.fc39, but none of the providers can be installed
  - cannot install both crun-102:1.12-1.20231205201336970037.main.19.g90b21dd.fc39.x86_64 from @commandline and crun-1.11.1-1.fc39.x86_64 from @System
  - cannot install both crun-102:1.12-1.20231205201336970037.main.19.g90b21dd.fc39.x86_64 from @commandline and crun-1.11.1-1.fc39.x86_64 from updates-archive
  - conflicting requests
```

This commit overrides both crun and crun-wasm from what's found in
podman-next.

Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
2023-12-08 17:10:41 +05:30
Brent Baude
1ebd507fbf Fix regression in e2e machine test suite
A simple regression was introduced to the test suite that overrode the
default image for hyperv testing.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-12-07 21:06:46 -06:00
Jason T. Greene
3d740674b3 Improve error handling in win-lib.ps1
- Modified Check-Exit to take a relative stack postition so that reusing
  functions like Run-Command report on their callers as opposed to the source
  position of the wrapper.
- Record and print the last command executed as it likely scrolled off with
  test output.

[NO NEW TESTS NEEDED]

Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
2023-12-06 21:49:33 -06:00
openshift-merge-bot[bot]
e8f30989b0 Merge pull request #20691 from cevich/mac_podman_machine_ci
Implement bare-metal Mac M1 podman-machine testing
2023-12-05 19:22:49 +00:00
Chris Evich
f1dc126bf6 Do not aggregate failing mac test status
Issue Ref: #20853

Allow the tests to fail, but don't block merging PRs.
This commit should be reverted when #20853 is resolved.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-12-04 10:40:13 -05:00
Chris Evich
2d46d05373 Implement bare-metal Mac M1 podman-machine testing
Setup and execute podman machine testing on bare-metal M1 Macs
using a pool of shared and semi-persistent hosts.  Automated
and manual processes outside this repository are responsible
for providing and maintaining all hosts.  Ref.
https://github.com/containers/automation/tree/main/mac_pw_pool

Update the `localmachine` make target to standardize execution
across platforms.  Update/simplify podman-machine e2e README to
reflect current reality.

Warning: This CI setup and supporting infrastructure was developed
in favor of expediency vs reliability and stability.  There are
many possible failure-modes (known and unknown) which may lead
to undefined test behaviors.  Future work may address some of
these as they are encountered or discovered.

[NO NEW TESTS NEEDED]

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-12-04 10:40:13 -05:00
Chris Evich
b7a81c1ce1 Fix command failure not resulting in task failure
For whatever reason (I don't understand this stuff well) the
`win-podman-machine-main.ps1` script exits successfully despite the
final `Check-Exit` showing a non-zero exit code was detected.  Attempt
to fix this by throwing an exception instead of calling `Exit`.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-12-04 10:38:05 -05:00
openshift-merge-bot[bot]
1672318439 Merge pull request #20852 from cevich/win_fail_ok_todo
Enable win podman-machine test failure
2023-12-03 11:52:49 +00:00
Chris Evich
f9e8585c53 Enable win podman-machine test failure
Intended to serve as motivation to fix them.  Removed from status
aggregator so the failures don't block PR merging.  Updated comment text
to reference related open issue, #20548.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-11-30 10:38:48 -05:00
Sander Maijers
a5d11f4041 Set BUILDAH_ISOLATION=chroot within Podman containers
See: https://developers.redhat.com/blog/2019/08/14/best-practices-for-running-buildah-in-a-container

See: 06c41b614d/docs/source/markdown/options/isolation.md

Signed-off-by: Sander Maijers <3374183+sanmai-NL@users.noreply.github.com>
2023-11-29 14:45:21 +01:00
Giuseppe Scrivano
cd21973f47 pkg/util: use code from c/storage
[NO NEW TESTS NEEDED] no new functionalities are added

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2023-11-23 21:36:42 +01:00
Ed Santiago
a10b88cb2f CI: test overlay and vfs
We're only testing vfs in CI. That's bad. #18822 tried to
remedy that but that only worked on system tests, not e2e.

Here we introduce CI_DESIRED_STORAGE, to be set in .cirrus.yml
in the same vein as all the other CI_DESIRED_X. Since it's 2023
we default to overlay, testing vfs only in priorfedora.

Fixes required:
 - e2e tests:
   - in cleanup, umount ROOT/overlay to avoid leaking mounts

 - system tests:
   - fix a few badly-written tests that assumed/hardcoded overlay
   - buildx test: add weird exception to device-number test
   - mount tests: add special case code for vfs
   - unprivileged test: disable one section that is N/A on vfs

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-11-17 05:48:50 -07:00
Daniel J Walsh
5dc807487f Pass secrets from the host down to internal podman containers
This change will allow RHEL subscriptions from the host to flow
to internal containers.

Fixes: https://github.com/containers/common/issues/1735

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-11-13 10:56:04 -05:00
Daniel J Walsh
c2de6d34ca Run codespell on podman
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-11-06 09:23:16 -06:00
Ed Santiago
2a17baa29d CI: default to sqlite
Followup to #20318: now that sqlite is the podman default,
enforce that in CI as well. Test boltdb only in Prior Fedora.

In the process, discovered & cleaned up some duplication
and unused YAML anchors.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-11-02 13:17:21 -06:00
Chris Evich
4c67a6aed2 Fix secrets scanning GHA Workflow
The podman in `ubuntu-latest` environment apparently is too old to
support `--userns=keep-id:uid=1000,gid=1000`.  Employ workaround in GHA
workflow and in `prebuild.sh` check.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-10-30 12:06:17 -04:00
Chris Evich
1146f2ca78 Merge pull request #20136 from cevich/credential_scanning_config
[CI:DOCS] Implement secrets/credential scanning
2023-10-30 11:43:01 -04:00
Urvashi Mohnani
ebe01ca292 Add e2e tests for farm build
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-10-24 17:32:47 -04:00
Ed Santiago
bd953fdc71 cirrus setup: special-case perl unicode
Perl is still stuck in the 1980s. Try to override that.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-10-19 09:30:10 -06:00
Chris Evich
6cb10425d4 Implement secrets/credential scanning
As an effort to catch potential secrets and/or credential leaks, add a
github-actions workflow which is untouchable in a PR context.
To additionally guard against accidents, also check recent branch
history.  This is especially important on newly created
release-branches, which may begin with content from who-knows-where.

Finally, since the new workflow bypasses PR-level changes to the scanner
config and base-line.  Add a Cirrus-CI invocation of the scanning tool
to help catch tool-breaking changes from being merged.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-10-05 11:16:19 -04:00