1313 Commits

Author SHA1 Message Date
e4123026b5 [CI:DOCS] Add contrib/podmanimage/stable path back in repo
The path mentioned above is linked in the sysadmin
article on running podman inside containers. The content
has since been moved and users are getting a 404 there now.
Add the path back with a readme pointing to the new location
of the content.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2024-05-31 10:38:34 -04:00
fb4ddf86cc Add Hyper-V option in windows installer
podman.msi GUI has a radio-button to select WSL or Hyper-V

The checkbox in podman.msi GUI allow the user to specify if
the machine provider installation (WSL or Hyper-V) should
be part of podman installation or not.

podman-setup.exe supports 2 new variables: MachineProvider
(valid values are `wsl` and `hyperv`) and HyperVCheckbox
(valid values are `0` and `1`)

Installation creates the configuration file
`99-podman-machine-provider.conf` under folder
`%APPDATA\containers\containers.conf.d` with the selected
machine provider

Cirrus CI `win_installer_task` tests the installation with
both `hyperv` and `wsl` and verifies the configuration.
Uninstallation is tested too.

Note that podman-setup.exe GUI doesn't allow to choose the
provider yet. See https://github.com/containers/podman/issues/22492

Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-05-29 12:09:37 +02:00
74e8f98772 Mac PM test: Require pre-installed rosetta
Previously, the mac podman-machine tests installed rosetta before
executing any tests.  As a best-practice (and because the Macs in CI are
shared) tests should never permanently modify the system.  As of this
commit, the system setup script used for the CI Macs does the rosetta
installation.  Remove the test setup code that installed rosetta and
add a CI-level confirmation that it's been pre-installed.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-05-23 10:23:06 -04:00
f03dc68f00 Merge pull request #22654 from ashley-cui/val
[CI:DOCS] Improvements to make validatepr
2024-05-13 20:36:22 +00:00
5901bf52b9 CI tests: enforce TMPDIR on tmpfs
First, setup a custom TMPDIR to ensure we have no special assumptions
about hard coded paths. Second, make sure it is actually on a tmpfs so
we can catch regressions in the VM setup immediately.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-05-13 17:27:10 +02:00
279e2d0cb2 [CI:DOCS] Improvements to make validatepr
Small usability improvements for our containerized validate target.

- Responds to SIGINT
- Exits if build fails, only validate if builds succeed
- Warns about potential of insufficient memory
- Document `make validatepr`

Signed-off-by: Ashley Cui <acui@redhat.com>
2024-05-09 10:06:47 -04:00
cdb354fc98 Merge pull request #22623 from cevich/cleanup_passthrough_envars
[CI:DOCS] Clarify passthrough_envars() comments
2024-05-08 09:05:07 +00:00
b520e909e0 CI Docs: Clarify passthrough_envars() comments
A long time ago, `passthrough_envars()` was defined in `lib.sh`.  It has
since been moved, but the related comments were never updated.  Update
the env. var. comments pointing future maintainers to the function that
relies on them.  Otherwise a simple search w/in this repo. won't turn up
anything.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-05-06 13:26:55 -04:00
87747ca820 Skip machine tests if they don't need to be run
Followup to #13936 : add an exclusion to localmachine tests
so we can avoid running those on test- or doc-only PRs.
Reason: #22551, the machine-start-timeout flake, is causing
hours of wasted time.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-05-06 09:55:16 -06:00
eaf60c7fe7 e2e tests: use /var/tmp, not $TMPDIR, as workdirs
TMPDIR is typically /tmp which is typically(*) a tmpfs.

This PR ignores $TMPDIR when $CI is defined, forcing all
e2e tests to set up one central working directory in /var/tmp
instead.

Also, lots of cleanup.

 (*) For many years, up to and still including the time of
     this PR, /tmp on Fedora CI VMs is actually NOT tmpfs,
     it is just / (root). This is nonstandard and undesirable.
     Efforts are underway to remove this special case.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-04-26 14:29:56 -06:00
d1bfdc7900 e2e and bindings tests: fix $PATH setup
Both tests need the podman-registry script in $PATH, this never worked
locally as only the cirrus specific CI setup scripts configured this.

To make it work correctly locally add the hack dir to $PATH for these
Makefile targets.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-04-26 14:37:27 +02:00
2c61ee9a31 CI: remove compose v1 tests
compose v1 has been deprecated for some time now, since July 2023 it no
longer receives any updates[1]. As such testing it on every PR is
pointless, it also does not provide any more coverage then compose v2.
At least I never saw only compose v1 test fails (except for flakes) so
it doesn't help us to catch regressions.
We tried to remove it before but decided against it at that time[2].

[1] https://docs.docker.com/compose/migrate/
[2] https://github.com/containers/podman/issues/18688

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-04-18 14:48:27 +02:00
5e9725983d install swagger from source
First of all this removes the need for a network connection, second
renovate can update the version as it is tracked in go.mod.

However the real important part is that the binary downloads are
broken[1]. For some reason the swagger created with them does not
include all the type information for the examples. However when building
from source the same thing works fine.

[1] https://github.com/go-swagger/go-swagger/issues/2842

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-04-12 15:26:34 +02:00
e773ed42b3 [CI:DOCS]Initial PR validation
This PR is only a first step towards being able to validate developer
code locally prior to pushing a PR and using CI.  Right now, we have a
prepared image in a temporary spot (will change when done).  That image
can be used to exercise various podman builds, make validate, and DCO
check.

The idea here is we have a make target that spins a podman container (or
machine) and then execute a small script to perform the actual builds.
Note, these builds are to verify code, not make production binaries so
corners are cut.  As of now, we choose to not build cross-arch binaries
because most of our problems thus far have been operating system builds
and not arch.

Of course this can be expanded in the future.  This is just step one to
start getting some of it in place.  The rest of the work is tracked in
JIRA under two cards.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-04-11 07:49:03 -05:00
ff133a544d Windows: clean up temporary perl install
Followup to #21991. Strawberry Perl is now installed by default
in CI VMs[1], so we no longer need the temporary install-perl code.

 [1] https://github.com/containers/automation_images/pull/337

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-04-03 08:53:55 -06:00
c0e32e93c0 Use logformatter for podman_machine_windows_task
https://github.com/containers/podman/issues/21760

Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-03-27 12:59:49 +01:00
f16ab115ed logformatter: handle Windows logs
C-colon and backslashes and ^Ms oh my

Tested on sample failure and success logs, seems to work.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-03-18 15:28:24 -06:00
dbf38779bd Makefile: drop tests-included from validate target
make validate should work locally, this check makes no sense in a local
context as it checks for a github label.
To fix this remove this check from the validate target and only use it
as part of the CI validate run.

While at it remove old dnf install step, the issue has been closed for a
long time and it should already be part of our base images.

Fixes #22031

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-03-13 18:17:43 +01:00
cc495a9fac Merge pull request #22014 from baude/norqdpolicy
do not require policy.json
2024-03-13 15:21:12 +00:00
d7bc7b7b41 do not require policy.json
we are having second thoughts about *requiring* a policy.json on podman
machine hosts.  we are concerned that we need to work out some more use
cases to be sure we do not make choices now that limit us in the near
term future. for example, should the policy files be the same for
container images and machine images? And should one live on the host
machine and the other live in the machine?

therefore, if a policy.json *is* present in the correct location, we will use and honor it; however, if it does not, we will allow the machine image to be pulled without a policy.

Signed-off-by: Brent Baude <baude@redhat.com>
Co-authored-by: Paul Holzinger <45212748+Luap99@users.noreply.github.com>
Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-03-13 09:07:51 -05:00
abaa179aa0 Add target win-gvproxy in winmake.ps1
For consistency with linux/osx makefile
I have added the win-gvproxy target as
an alias of win-sshproxy

[NO NEW TESTS NEEDED]

Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-03-13 00:45:10 +01:00
38e22d443c Merge pull request #21950 from cevich/move_podmanimage
[CI:DOCS] Migrate podman container image
2024-03-11 17:28:28 +00:00
45068124f0 Merge pull request #21985 from n1hility/fin-policy-json
Complete policy.json inclusion (Also add to win installer)
2024-03-07 21:13:38 +00:00
d54a373996 Complete policy.json inclusion
- Sets default search location to always be the peer directory
  + make podman-remote now creates binaries that work the same as release zips
- Updates release zip to match expected search location
- Updates win installer to include the file if present in the repo cross-build
  archive

[NO NEW TESTS NEEDED]

Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
2024-03-07 14:05:22 -06:00
75fa38d52e [CI:BUILD] Build universal Podman binary for Mac installer
Build universal Podman binary and installer for Mac. Update GitHub action to build it too.

Signed-off-by: Ashley Cui <acui@redhat.com>
2024-03-07 09:09:32 -05:00
d6d2601742 [CI:DOCS] Migrate podman container image
Moved to: https://github.com/containers/image_build

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-03-05 16:11:02 -05:00
530782e11d logformatter: fixes for Macintosh
Lots of small special-case tweaks to logformatter because Macs
have to be different.

Also fix:
 - Wrong slash in printf-newline, leading to gray [It] blocks
 - echo gitCommit, so we can link to sources
 - --image-path is deprecated

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-03-04 05:52:00 -07:00
99bb2bfce4 CI: must-add-tests check: use GH label, not text
Old way: edit commit message, add magic string, re-push

New way: repo maintainer adds a Github label to PR, hits Rerun

I've looked and looked for the history behind this script
and why I didn't do it this way in the first place. I've
concluded that I just never thought of it.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-03-01 05:32:06 -07:00
6957c78199 CI: run logformatter on mac
It works (verified in #17831). Imperfectly, because Macs don't
have a useful awk, so we can't get timestamps. I will, in time,
look into adding the timestamp functionality to logformatter
itself.

Also imperfect because it's not linkifying: source code paths
are dead text. I need to fix that, too, in logformatter, by
having it recognize /Users/Mac paths.

Imperfect as it may be, it was quick, and I think could provide
good bang for the buck in these Mac-intensive debugging days.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-02-28 20:14:05 -07:00
fd1d951262 Move ignition functions into Containerfiles
We used to use ignition to perform any customization required for podman
machine because our input was a generic FCOS image.  Now that we are
building our own images, some of this customization can be migrated to
the Containerfile itself and be less of a burden in our code at boot up.

At the time of this PR, the Containerfile can be found at
https://github.com/baude/podman-machine-images/tree/main.  It is only
present for a so-called daily image.  There is little liklihood that
this would the final location for the Containerfile so consider it a
working version only.

Split WSL and rest apart in the e2e tests so we no longer ppull the
generic FCOS image for testing.

Note: the change to the pull image name is so PRs are not immediately
broken that are already in the queue.

[NO NEW TESTS REQUIRED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-28 16:05:50 -06:00
4704418c13 Merge pull request #21841 from edsantiago/rawhide_rc6
new vms, with rc6 rawhide kernel
2024-02-27 19:40:56 +00:00
9f5c20f9bf macos installer: Add vfkit entitlement
vfkit needs the com.apple.security.virtualization entitlement or it
wont' be able to start virtual machines:

Error: Error Domain=VZErrorDomain Code=2 Description="Invalid virtual machine configuration. The process doesn’t have the “com.apple.security.virtualization” entitlement." UserInfo={
    NSLocalizedFailure = "Invalid virtual machine configuration.";
    NSLocalizedFailureReason = "The process doesn\U2019t have the \U201ccom.apple.security.virtualization\U201d entitlement.";
}

This fixes https://github.com/containers/podman/issues/21842

Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
2024-02-27 17:47:49 +01:00
15734f8c5a macos-installer: Remove hvf.entitlements
That's a left-over from 87947761ed

Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
2024-02-27 17:44:04 +01:00
74b87874b6 macos installer: Default to using ad-hoc signing
When there is no signing identity to pass to the macOS `codesign` tool,
we can use `-` instead as the identity to perform ad-hoc signing.

From `man codesign`:
> If identity is the single letter "-" (dash), ad-hoc signing is
> performed.  Ad-hoc signing does not use an identity at all

This makes it easier to test the sign() code-path in package.sh as
we'll run the same code regardless of `NO_CODESIGN` being set or not.

Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
2024-02-27 17:44:01 +01:00
c2a2113bb6 new vms, with rc6 rawhide kernel
Source: https://github.com/containers/automation_images/pull/331#issuecomment-1966677347

Kludgy VM build, because rawhide rc6 kernel is still not stable.
I would like to merge this anyway, because the rawhide hang is
hurting us badly. (I am not guaranteeing that this fixes the hang).

Also: new Windows VM has WiX 3.14 (up from 3.11).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-02-27 09:42:43 -07:00
1e5b5a8892 macos installer: install default policy.json file
Include a default policy.json file in the macos package so users do not
have to add this manually.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-27 15:31:09 +01:00
bb44510db6 Merge pull request #21639 from cevich/require_build
Rearrange CI tasks for safety + efficiency
2024-02-22 21:05:08 +00:00
c4e68bdae4 Cirrus: Reuse shared clone script on Mac
The `Makefile` makes assumptions about git repo. clone details.
Because fixing the `Makefile` would likely be problematic, fix
the clone operation used on the Mac so that it matches what's used
on Linux.

Also, simplify git repo clone operations.  At some point in the
distant past, a git identity was required for CI to function properly.
That has since changed, so remove the unnecessary complexities.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-21 15:20:16 -05:00
f7d1726f93 Rearrange CI tasks for safety + efficiency
There's are sometimes conflicting purposes in podman CI:

1. Have the pipeline proceed in an orderly and progressive manner
   to sometimes save resources and unnecessary runtime.
2. Complete all testing as quickly as possible in support of
   human-developers moving on to other areas of work.
3. Ideally/hopefully, accomplish both items above safely,
   preventing untested and/or unintended changes from merging.

This commit shifts the balance of these slightly more toward the second
point.  It rearranges most CI tasks into essentially three buckets with
a single (new) aggregation task in-between the first two:

1. Build + Verify all the things
2. Test all the things
3. Minor/accessory things

The intention is that while we may unnecessarily spin some number of
testing tasks while others have failed, the best-case scenario
(everything passes) has a much shorter runtime.  In other words, it
potentially wastes more resources in favor of a chance to have
developers wait less.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-21 13:27:16 -05:00
f756e5db68 Merge pull request #21747 from mheon/windows_lint
Fix Lint on Windows and enable the job
2024-02-21 00:47:04 +00:00
ec68f07c04 Fix Lint on Windows and enable the job
[NO NEW TESTS NEEDED] Purely refactoring

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-20 08:06:18 -05:00
66fd41ff19 cirrus logcollector: update package list
aardvark-dns, netavark and passt are installed on both debian and
fedora. cri-o-runc is not installed anymore and it just uses the normal
runc package on debian. containers-common is called
golang-github-containers-common on debian and also uses
golang-github-containers-image for further config files from c/image.
This makes sure we correctly log all the package versions on debian
correctly.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-20 13:58:57 +01:00
9fc45088cb Merge pull request #21587 from cevich/fix_podman_remote_image
[CI:DOCS] Remove disused Containerfile and docs
2024-02-20 09:46:31 +00:00
068d4e81c7 Allow CI user to cleanup own files
According to https://go.dev/ref/mod#module-cache golang will leave
behind read-only bits.  It was observed that these cause the find/rm
cleanup operations to fail fail with `permission denied` on thousands
of files.  This is preventing cleanup of cruft from unrelated Cirrus-tasks
leading to unnecessary occupation of critical, local-ssd storage space.
Fix this by ensuring the user has at least write access to the entire
contents of `$TMPDIR` and `$HOME`, `ci` subdirs.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-14 10:29:51 -05:00
87947761ed [CI:BUILD] Add VFKit into pkginstaller, remove QEMU
Do not build and install QEMU in the pkginstaller. Instead, build, sign, and install VFKit.

Signed-off-by: Ashley Cui <acui@redhat.com>
2024-02-13 10:20:03 -05:00
fcd69c997e Merge pull request #21542 from jakecorrenti/bump-gvproxy
Enable passing logfile to gvproxy
2024-02-12 21:49:41 +00:00
00180bd5b3 Enforce podman-machine mac CI results
Followup to https://github.com/containers/podman/pull/21551

Note: Fixed indentation of podman-machine mac task.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-12 09:28:23 -05:00
e16d82dbac Merge pull request #21441 from cevich/win_lint
Windows-cross linting
2024-02-09 18:08:01 +00:00
69163af2a0 Remove gitleaks scanning
Ref:
https://github.com/containers/podman/pull/21570#issuecomment-1935709148

This tool is really intended/best used from git pre-commit on developers
local machines, to prevent addition of secret leaks.  When used as a
check against PRs, it tends to turn up more false-positives than helpful
warnings.  There's no good way to fix this, and maintaining the scanner
is an additional burden.  Rather than continue struggling to improve/fix
the situation, let's just remove the tool entirely.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-09 10:09:42 -05:00
5b6b689764 Remove disused Containerfile and docs
Fixes: #21574

The documentation for this image references a quay repository that
doesn't exist.  It doesn't appear any of these files have been touched
since late 2022. Instead of updating the docs, let's just remove the
source.  It's trivial to recreate if anybody actually needs it for
something.

Users needing to access remote podman can simply use the `podman` binary
present in existing images `quay.io/containers/podman`,
`quay.io/podman/stable`, etc.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-09 10:02:27 -05:00