21503 Commits

Author SHA1 Message Date
d5eb8f3b71 AppleHV - make gz ops sparse
gz by definition is not able to preserve the sparse nature of files.  using some code from the crc project and gluing it together with our decompression code, we can re-create the sparseness of a file.  one downside is the operation is a little bit slower, but i think the gains from the sparse file are well worth it in IO alone.

there are a number of todo's in this PR that would be ripe for quick hitting fixes.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <baude@redhat.com>
2024-02-07 09:19:09 -06:00
85d8281484 create machine dirs at discovery
in various use cases, the required machine dirs are not created.  the
machine dirs are runtimedir, datadir, and configdir.  Example in Linux
would be:

configDir /<HOME>/.config/containers/podman/machine/<provider>
dataDir /<HOME>/.local/share/containers/podman/machine/<provider>
runtimeDir /run/user/1000/podman/machine

now we blindly create them without checking for their existence (because
it is faster).

this fixes a bug where runtimedir does not exist on macos after a reboot

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <baude@redhat.com>
2024-02-07 09:19:00 -06:00
b1ce6ef9a8 podman machine 5 - hyperv
this pr represents the podman 5 maching refactoring for HyperV.  with
the exception of already skipped tests, all local tests pass.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-07 09:18:51 -06:00
6b02c4894b Podman 5 machine refactor - applehv
this is the second provider done (qemu first).  all tests pass on arm64 hardware locally ... the hybrid pull from oci registries limit this to arm64 only.

calling gvproxy, waiting for it, and then vfkit seems to still be problematic.  this would be an area that should be cleaned up once all providers are implemented.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-07 09:18:45 -06:00
e8501ca991 Implement generic providers
The intial refactor used specifically qemu for testing and infra bring
up.  But the whole point was to have things interfaced.  This PR results
in an interface experience like podman 4 using the same term `provider`
to generically represent 'a provider' like qemu/applehv/etc.

This PR is required to move forward with new providers.

Also renamed pkg/machine/p5 to pkg/machine/shim.

[NO NEW TESTS REQUIRED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-07 09:18:38 -06:00
9bb191df51 [CI:MACHINE]Podman5 QEMU refactor
The following PR is the leading PR for refactoring podman machine with
the following goals:

* less duplication/more re-use
* common configuration file between providers
* more consistentency in how machines are handled by providers

The goal of this PR is the rough refactor.  There are still rough spots
for sure, specifically around the podman socket and pipe.  This
implemention is only for Linux. All other providers are still present
but will not compile or work.  This is why tests for them have been
temporarily suspended.

The ready socket code is another area that needs to be smoothed over.
Right now, the ready socket code is still in QEMU.  Preferably it would
be moved to a generic spot where all three approaches to readiness
socket use can be defined.

It should also be noted:

* all machine related tests pass.
* make validate for Linux passes
* Apple QEMU was largely removed
* More code pruning is possible; will become clearer when other
  providers are complete.

the dir pkg/machine/p5 is not permanent.  i had to seperate this from
machine initially due to circular import problems.  i think when all
providers are done (or nearly done), it can be placed and named
properly.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-07 09:18:36 -06:00
b9bcfa4749 podman5 machine reconfig 1
this is a logical place to get changes upstream before they grow out of
control.  this pr is the first in an effort to deduplicate machine code
and streamline code flow.

a lot of code is simply moved to eliminate circular imports.  names and
specific paths can ultimately be changed.  i dont like some of the
descriptive interface names, etc.  ultimately, i think once we have the
"old" code sanitized, we can re-use some of those.

clearly some of what is in here is temporary and will either be deleted,
changed, or moved again as this effort comes to a close.

right now, the machine code does not use any of the "new" code.  you
will see in `init` and `rm` some commented out code that hooks it. i'm
afraid things will get worse before they get better (way worse).

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-07 08:17:43 -06:00
c89dd4a92f Use tmpfs mounts when creating a memory-backed emptyDir volume
Signed-off-by: Andre Marianiello <andremarianiello@users.noreply.github.com>
2024-02-07 08:17:43 -06:00
f4f96a226f Merge pull request #21518 from Luap99/strongunits
vendor: update c/common + libhvee to latest main and use pkg/strongunits from c/common
2024-02-07 01:46:06 +00:00
22b1650619 Merge pull request #21530 from Luap99/netavark-skip
test/e2e: unskip netavark macvlan/ipvlan tests
2024-02-06 19:21:05 +00:00
46fe7ef3bb use pkg/strongunits from c/common
The code has been moved to c/common so it can be shared with libhvee.

[NO NEW TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-06 16:39:16 +01:00
5de4bd5d13 vendor: update c/common + libhvee to latest main
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-06 16:39:02 +01:00
a997d7d819 test/e2e: unskip netavark macvlan/ipvlan tests
These should all work with the latest netavark. The ipvlan case needs a
subnet because it does not support DHCP.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-06 15:49:34 +01:00
855a7cf0f4 Merge pull request #21413 from cevich/machine_cron
Include machine-tests in cirrus-cron jobs
2024-02-06 14:41:03 +00:00
75e6564138 Merge pull request #21497 from vikas-goel/volsfrom
Use semi-colon as the field separator for internal volumes-from inspect annotation
2024-02-06 14:35:34 +00:00
f439f4e9da Merge pull request #21431 from lsm5/cgroupv1-warn
Warn if cgroups-v1
2024-02-06 14:13:35 +00:00
01a8539ef0 Warn if cgroups-v1
Podman v5 will not support cgroups-v1. This commit will print a warning
if it detects a cgroups-v1 system. The warning can be hidden by setting
envvar `PODMAN_CGROUPSV1_WARNING`.

This warning is patched out for RHEL 9 builds as cgroups-v1 will still
be supported on RHEL 9 systems.

Resolves: https://issues.redhat.com/browse/RUN-1957

[NO NEW TESTS NEEDED]

Co-authored-by: Ed Santiago <santiago@redhat.com>
Co-authored-by: Sascha Grunert <sgrunert@redhat.com>
Co-authored-by: Giuseppe Scrivano <gscrivan@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
2024-02-06 16:18:05 +05:30
a2f0a44501 Merge pull request #21410 from ashley-cui/cnici
Do not test CNI in CI
2024-02-06 10:41:48 +00:00
8be1ce296a Merge pull request #21501 from kanielrkirby/correct-stars-markdown
[CI:DOCS] Corrected markdown documentation for `--stars`
2024-02-05 22:02:48 +00:00
2eee3cd812 Corrected markdown documentation for --stars
The `--stars` option was incorrectly documented as meaning
the number of stars to filter by. This has been corrected
to indicate that it is the minimum number of stars to filter
by.

Tweaked wording of podman-search.md stars filter text

Signed-off-by: Kaniel Kirby <piratey7007@runbox.com>
2024-02-05 11:37:01 -06:00
26cd01ee51 Do not test CNI in CI
CNI is deprecated and is build tagged out for 5.0. Don't test it in our CI.
This commit also disables upgrade tests for now - those need more work since the old version of Podman only uses CNI. Upgrade tests will be re-vamped in a later commit.
Signed-off-by: Ashley Cui <acui@redhat.com>
2024-02-05 11:01:21 -05:00
fc3ae73961 Merge pull request #21494 from containers/renovate/github.com-docker-docker-25.x
fix(deps): update module github.com/docker/docker to v25.0.2+incompatible
2024-02-05 15:43:21 +00:00
bb6a561583 Merge pull request #21457 from containers/renovate/github.com-mattn-go-sqlite3-1.x
fix(deps): update module github.com/mattn/go-sqlite3 to v1.14.21
2024-02-05 14:53:48 +00:00
ee923358c3 Vendor c/common
@main

Signed-off-by: Ashley Cui <acui@redhat.com>
2024-02-05 09:46:40 -05:00
a098899104 Use semi-colon as the field separator for internal volumes-from inspect annotation
The current field separator comma of the inspect annotation conflicts with the mount options of --volumes-from as the mount options itself can be comma separated.

Signed-off-by: Vikas Goel <vikas.goel@gmail.com>
2024-02-05 00:16:53 -08:00
daf7a2c069 Merge pull request #21372 from cevich/win_mac_lint
Run lint for mac builds
2024-02-03 13:19:30 +00:00
9e3d61a9d5 fix(deps): update module github.com/docker/docker to v25.0.2+incompatible
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-02 22:02:19 +00:00
2431fb3f0b Merge pull request #21472 from mheon/vendor_runc_main
Update to runc main, removing pin to an older version
2024-02-02 22:01:02 +00:00
39f22c3821 Include machine-tests in cirrus-cron jobs
Irrespective of reason/cause, a commit was merged into main that broke
podman-machine, and went unnoticed for several days.  Improve the
situation by including podman-machine testing in the daily cirrus-cron
builds.  These are monitored by e-mail when `Total Success` reports a
failure.

Also: Add a comment for Windows & Mac build tasks, documenting the
reason they don't execute on RHEL release branches.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-02 14:46:54 -05:00
7de64b47f2 Vendor c/storage main
This picks up a further removal of the runc users package that
was breaking the build on Windows.

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-02 13:30:07 -05:00
11c37d5c95 Merge pull request #21272 from rhatdan/VENDOR
Add tests to make sure mounts.conf does not warn on missing content
2024-02-02 16:03:39 +00:00
9fb57d346f Cease using deprecated runc userlookup
Instead switch to github.com/moby/sys/user, which we already had
as an indirect dependency.

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-02 11:02:43 -05:00
91b8f208a2 Bump to runc main
By using main instead of the v1.1 branch, we drop an unnecessary
dependency on cilium/ebpf, saving ~1mb of binary size.

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-02 10:06:20 -05:00
6a727fdbee Fix updated runc dep breaking pod devices cgroup
The update to runc broke creation of devices for containers in
the pod cgroup. We don't support the device cgroup for pods at
present, so just disable it for now, resolving the issue.

Thanks to Giuseppe for finding this one!

[NO NEW TESTS NEEDED] This is a fix for broken tests

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-02 09:15:35 -05:00
2818abf849 Update to runc main, removing pin to an older version
We were pinned to a specific commit to ensure that tests kept
passing. Hopefully they pass now, as we need to grab latest runc
for CVE fixes.

Also grab Buildah main to fix a build issue on FreeBSD. After a
botched manual vendor, I used Ed's treadmill script and squashed
it into this commit to make Git happy. Thanks bunches Ed.

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-02 08:11:17 -05:00
5e64d4f021 Merge pull request #21068 from alexlarsson/quadlet-templates
Support templates in quadlet
2024-02-02 10:36:18 +00:00
0da4054534 Verify the empty mounts.conf does not generate warnings.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2024-02-01 20:12:08 -05:00
7cb0c2ef09 Merge pull request #21000 from rhatdan/machine
Strip key/values of external quotes
2024-02-02 00:59:03 +00:00
a59a32dc87 Merge pull request #21464 from containers/renovate/go-github.com/opencontainers/runc-vulnerability
fix(deps): update module github.com/opencontainers/runc to v1.1.12 [security]
2024-02-01 19:48:30 +00:00
d8c58936a4 Merge pull request #21414 from umohnani8/farm-reg
Farm build should read server registries.conf
2024-02-01 19:37:17 +00:00
c3a32b170d Merge pull request #21366 from edsantiago/new-vms
Test new CI VMs
2024-02-01 19:01:32 +00:00
1ab55defbb Test new CI VMs
From https://github.com/containers/automation_images/pull/325

Major change: netavark and aardvark are now included in prior-fedora,
so CNI can be fully eliminated from CI (#21410)

FIXME FIXME FIXME: skip two e2e tests, waiting for new netavark

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-02-01 09:50:41 -07:00
3f3d1e4de7 Temporarily ignore mac-linting failures
As of this commit, there are several pages worth of lint findings for
the mac.  Once they're all addressed, this commit may be reverted to
enable continuous checking.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-01 09:49:53 -05:00
d9ad7cbc94 Run lint for mac builds
There are darwin-specific code paths which were not being linted prior
to this commit.  Fix this with a new, darwin-specific section of the lint
runner script.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-02-01 09:49:53 -05:00
90d6e92f2f fix(deps): update module github.com/opencontainers/runc to v1.1.12 [security]
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-01 00:24:04 +00:00
4fde98f380 Merge pull request #21459 from cevich/simpler_to_read
[skip-ci] Make leak-detection readable by humans
2024-01-31 21:09:41 +00:00
f7afaf9652 Merge pull request #21455 from vrothberg/resolve-endpoint
add new libpod/images/$name/resolve endpoint
2024-01-31 21:06:57 +00:00
a06685a548 Farm build should read server registries.conf
Fix the way we set skipTLSVerify on the client side
to ensure that the push stage in farm build takes into
account the configuration in the farm node's registries.conf
when the user hasn't set it on the client side.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2024-01-31 15:41:27 -05:00
1a8cb15aa6 Merge pull request #21384 from Luap99/connections
rework system connection and farm storage
2024-01-31 19:29:44 +00:00
28856b6c15 Make leak-detection readable by humans
Previously when a leak was detected under any circumstance, the workflow
would splat out a giant wall of gray, unreadable git-log text.  This often
enormous text might contain, somewhere, possibly, maybe, a little tiny
snippet of code that leaks a secret.

Improve the situation greatly by providing easy-to-use URLs that covers
the relevant changes based on the triggering context (new pr, force-push,
or merge).  Store the former (often) giant git-log output into a file
and stuff it into the artifacts in case it's ever useful.

Signed-off-by: Chris Evich <cevich@redhat.com>
2024-01-31 13:53:11 -05:00