Commit Graph

835 Commits

Author SHA1 Message Date
Valentin Rothberg
f8abd7ff75 machine config: make write atomic
As indicated in #21849, loading the machine config can flake/fail with
an EOF JSON error indicating an incomplete file.  Address the issue by
atomically writing the config.  This way, it is not possible to load an
incomplete or partially written file.  The lock can be acquired later on
to sync state.

[NO NEW TESTS NEEDED] as it's a hard-to-hit race.

Fixes: #21849
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2024-02-28 09:44:41 +01:00
openshift-merge-bot[bot]
9d1704396a Merge pull request #21572 from mheon/hyperv_9p_mkdir
Ensure HyperV 9p mounts work when a dir doesn't exist
2024-02-27 22:34:07 +00:00
Matt Heon
26ec570c65 Ensure HyperV 9p mounts work when a dir doesn't exist
Before, we required that the mount target exist and be a
directory for the 9p mount to successfully complete, which is not
how things are supposed to work - the user should be able to
mount anywhere. This should just be a simple mkdir, but with FCOS
the root directory is immutable so we need to undo that before we
can mkdir, and unfortunately we don't have a library that can do
chattr (and I didn't want to drag in a new dependency just for
that), so let's be gross and add it to the SSH command. I
aggressively dislike this but it does work.

[NO NEW TESTS NEEDED] Can worry about getting a more generic
mount test together for Machine later.

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-27 14:33:53 -05:00
Brent Baude
dfae51a4b8 Clean up gvproxy if machine start fails
Matt found a bug where if a machine start did not run to completion, a
gvproxy was left around running.  This gvproxy then subsequently stopped
the next attempt to start.

Signed-off-by: Brent Baude <bbaude@redhat.com>

[NO NEW TESTS NEEDED]
2024-02-27 13:33:43 -06:00
Mario Loriedo
7b6d9a586e Fixup: avoid overriding io.Copy error
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-27 16:48:47 +01:00
Mario Loriedo
59704665ae Avoid overriding io.Copy error
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-27 16:16:44 +01:00
Mario Loriedo
24a33a538c Move code for sparse optimized copy to a dedicated method
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-27 15:59:56 +01:00
Paul Holzinger
bed61806c8 pkg/machine/ocipull: add default policy.json file
So that this file can be inculded in our windows/macos packages and also
by other packagers.
Right now the default policy is allow everything but we plan to add
signing in the future.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-27 15:29:59 +01:00
Paul Holzinger
a02aa8f6a2 pkg/machine/ocipull: add custom policy.json location
The default policy file /etc/containers/policy.json location does not
work on windows and for packages that ship a default.

Now we search for the policy.json in the following overwrite locations:
macos and linux:
 - ~/.config/containers/policy.json
 - /etc/containers/policy.json
windows:
 - %APPDATA%\containers\policy.json

Also it offers an additional DefaultPolicyJSONPath var that should be
overwritten at built time with the path of the file that is shipped by
packagers. Thile file is used when none of the overwrite paths exist.

[NO NEW TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-27 15:29:58 +01:00
openshift-merge-bot[bot]
04f70320c1 Merge pull request #21594 from arixmkii/qemu-win-machine-5
Change QEMU netdev to Unix domain socket
2024-02-26 20:37:06 +00:00
Arthur Sengileyev
656bf7b764 Change QEMU netdev to Unix domain socket
This change migrates to new QEMU stream netdev added in 7.2.0.
It also unifies how gvproxy is used in QEMU and AppleHV.

Signed-off-by: Arthur Sengileyev <arthur.sengileyev@gmail.com>
2024-02-26 20:02:46 +02:00
Matt Heon
19b676f855 Enable lint for Darwin and fix identified issues
[NO NEW TESTS NEEDED] Just fixing lint issues

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-26 11:01:54 -05:00
Mario Loriedo
f099250beb Better file close and err handling
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:03 +01:00
Mario Loriedo
ea4553d590 integrating changes from #21768
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
fa99b9be93 renamed testfiles as testdata
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
0b3f3f0ef1 Use faster gzip reader
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
2245cf8dc4 Remove duplication and make consistent usage of the progress bar
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
c42d3a74ed Add a comment to explain why we look at file name for zip files
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
95aa2c4e68 Use github.com/stretchr/testify assert in compression_test.go
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
88af8852db Refactor machine decompress.go
Added some tests to verify that files extractions works
with different compression format.

Created a decompressor interface with 2 main methods:
  reader(): returns an io.Reader for the specific compression algorithm
  copy(): extracts the compressed file into the file provided as param

Created 5 decompressor types:
- gzip: extract gzip files
- xz: extract xz files
- zip: extract zip files
- generic: extract any other file using github.com/containers/image/v5/pkg/compression
- uncompressed: only do a copy of the file

Minor fix to the progress bar instances: added a call to bar.Abort(false)
that happens before Progress.Wait() to avoid that it hangs when a bar is
not set as completed although extraction is done.

Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
openshift-merge-bot[bot]
1fc9d5ae6e Merge pull request #21797 from mtrmac/sparse0
Reformulate sparseWriter
2024-02-25 14:36:08 +00:00
openshift-merge-bot[bot]
eabce71cd2 Merge pull request #21804 from n1hility/ready-race
Fix race conditions in hyperv backend
2024-02-24 14:07:38 +00:00
Jason T. Greene
b68d3c7a0e Fix race conditions in hyperv readiness checking
- Listen before starting the vm
- Fix a device race caused by lazy hv_vsock init by waiting on network manager

[NO NEW TESTS NEEDED]

Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
2024-02-24 01:43:28 -06:00
Miloslav Trmač
5d303ca267 Reformulate sparseWriter to deal with starting/ending zeroes explicitly
... instead of using a multi-variable state machine.

The net effect of this code is exactly the same as the previous implementation,
except:
- the operation after Write() returns an error might differ
- If the file ends with zeroes, we don't Seek(-1), and
  we don't create a hole at all if it is too small, preferring
  to save a syscall.

But this formulation is hopefully easier to prove correct.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2024-02-23 02:09:39 +01:00
Miloslav Trmač
c5434bf711 Expand sparseWriter tests
- Add more test cases
- Test that we create the expected (large) holes;
  don't enforce anything for the <zerosThresholt ones.

O)nly changes test code, should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2024-02-23 02:09:39 +01:00
openshift-merge-bot[bot]
36d8e27601 Merge pull request #21517 from jakecorrenti/fix-qemu-todos
machine: Address some QEMU TODOs
2024-02-22 21:21:50 +00:00
openshift-merge-bot[bot]
b338364fbf Merge pull request #21786 from Luap99/machine-gvproxy-cleanup
pkg/machine: ignore gvproxy pidfile not exists error
2024-02-22 16:13:41 +00:00
openshift-merge-bot[bot]
5a844511c8 Merge pull request #21652 from Luap99/machine-http-proxy
machine: implement http proxy logic for all providers
2024-02-22 15:40:39 +00:00
Jake Correnti
d7f7f07c9b Stop using rm'd inspect format templates in tests
Fixes the "machine rm --save-ignition --save-image" test so that it no longer
uses the `{{ .Image }}` format string.

Fixes the "init should cleanup on failure" test so that it no longer
uses the `{{ .Image }}` and `{{ .ConfigPath }}` format strings.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-22 08:34:41 -05:00
Jake Correnti
be9aba7ccd Remove image provenance from machine inspect
We don't care about the provenance of the machine image, so this is no
longer applicable to have when displaying info.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-22 08:34:27 -05:00
Jake Correnti
d68240b150 Update machine files rm order and add QEMU rm
Changes the order in which the machine-specific files are removed in
`Remove()`. Removes the system connections first, then removes the
`configPath` last. `configPath` is removed last, because in the case of
an error with any of the previous files, the removal can be attempted
again since the machine still "exists".

Made the errors in `Remove` hard errors instead of soft errors.

Added the implementation for the QEMU-specific file removal.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-22 08:31:55 -05:00
Paul Holzinger
6f6925cca4 pkg/machine: ignore gvproxy pidfile not exists error
When gvproxy exits it will delete the pidfile itself so we need to
account for that and juts ignore the case, it just means gvproxy was
able to exit successfully on its own.

Also remove the useless defer and return the error so we can get an
error exit code not just a print on stderr.

Currently it shows this error which is not helpful to any user:
unable to clean up gvproxy: "unable to read gvproxy pid file /run/user/1000/podman/gvproxy.pid: open /run/user/1000/podman/gvproxy.pid: no such file or directory"

[NO NEW TESTS NEEDED] TODO: make machine tests check stderr for such
things.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-22 12:39:20 +01:00
openshift-merge-bot[bot]
669e718561 Merge pull request #21636 from jakecorrenti/inspectinfo-hostuser
machine: Remove unnecessary TODOs
2024-02-22 10:29:51 +00:00
Paul Holzinger
f218f8430a machine: implement http proxy logic for all providers
Copy all proxy envs into the VM on each start, this allows for updates
without having to recrate the VM. This is implemented via shell script
that is passed via ssh to the VM after it is started.

With that we now use the same logic for all providers the old fw_cfg
logic for qemu has been removed and the WSL code as well which keeps the
behavior the same.

There is a small risk now because we only update the env via ssh that
processes started before will have the old incorrect env but it should
really only effect core system processes which likely do not need them
anyway. The podman system service should not be started at this point
so it should be good enough.

It also fixes the broken behavior with SSL_CERT_FILE/SSL_CERT_DIR which
were updated on each start which is not correct as the files are only
copied with ignition so these should not be updated and just set
statically when the VM was created.

e2e test has been added to ensure the behavior works as expected.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-22 11:23:45 +01:00
openshift-merge-bot[bot]
59b6f48d90 Merge pull request #21735 from jakecorrenti/inspect-conn-vals
machine: Add `ConnectionInfo` to inspect
2024-02-22 02:36:50 +00:00
openshift-merge-bot[bot]
0d1a8c0f2b Merge pull request #21768 from baude/zstd
zstd now default compression for podman machine
2024-02-21 14:46:38 +00:00
openshift-merge-bot[bot]
f756e5db68 Merge pull request #21747 from mheon/windows_lint
Fix Lint on Windows and enable the job
2024-02-21 00:47:04 +00:00
Brent Baude
a31e8d2a23 zstd now default compression for podman machine
given that we are moving to building our own machine images, we have
decided to use zstd compression as it is superior in speed to the
alternatives.  as such, this pr adds zstd to our machine code; and also
has to account for dealing with sparseness on darwin; which the default
zstd golang library does not.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-20 14:26:41 -06:00
openshift-merge-bot[bot]
1d4651bccc Merge pull request #21763 from giuseppe/sparse-file
machine: add sparse file writer
2024-02-20 19:35:33 +00:00
openshift-merge-bot[bot]
d9c706ea95 Merge pull request #21694 from arixmkii/wait-for-gvproxy
Extract waitForGvProxy into shared utility function
2024-02-20 18:59:46 +00:00
Giuseppe Scrivano
0b861350c2 machine: add sparse file writer
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2024-02-20 15:57:24 +01:00
Jake Correnti
09095acf2b machine: Add ConnectionInfo to MachineConfig
Adds `ConnectionInfo()` to the `MachineConfig` and fills out
`InspectInfo` accordingly. Additionally fixes the "inspect with go format" test.

Changes `ConfigPath` to `ConfigDir` to better represent the
output.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-20 09:08:11 -05:00
Matt Heon
ec68f07c04 Fix Lint on Windows and enable the job
[NO NEW TESTS NEEDED] Purely refactoring

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-20 08:06:18 -05:00
Jake Correnti
66f39eee5e machine: Remove unnecessary TODOs
Remove TODO to swap `Rootful` in Inspect with `HostUser`

It is unnecessary to remove the vfkit logfile in the provider-specific Remove function. Vfkit is fed the default logfile provided by mc.LogFile which is removed by the generic Remove function.

Removes TODO regarding moving the location of Stop. False TODO.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-19 19:51:49 -05:00
Daniel J Walsh
4f13b836bf Codespell code
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2024-02-19 15:23:56 -05:00
Arthur Sengileyev
49400ecce1 Extract waitForGvProxy into shared utility function
Signed-off-by: Arthur Sengileyev <arthur.sengileyev@gmail.com>
2024-02-19 18:00:00 +02:00
openshift-merge-bot[bot]
89587a5d59 Merge pull request #21638 from ashley-cui/buildtag
Build tag out QEMU for Darwin
2024-02-19 13:31:58 +00:00
openshift-merge-bot[bot]
630bfbfdbd Merge pull request #21686 from arixmkii/qemu-machine-5-bugfixes
Improve cross platform support in QEMU machine sources
2024-02-18 23:27:12 +00:00
Brent Baude
10d748f584 Introduce Podman machine reset
Podman machine reset is a new command that will "reset" your podman
machine environment.  Reset is defined as:

* Stop and Remove all VMs
* Remove the following directories:
    - configuration dir i.e. ~/.config/containers/podman/machine/qemu
    - data dir i.e. ~/.local/.share/containers/podman/machine/qemu

When deleting, if errors are encountered, they will be batched and spit
out at the end.  Podman will try to proceed even in error in doing what
it was told.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-18 08:43:12 -06:00
openshift-merge-bot[bot]
fbb4d5dca6 Merge pull request #21692 from Luap99/machine-cleanup
machine init: validate machine name and username
2024-02-17 12:33:23 +00:00