63 Commits

Author SHA1 Message Date
7dd33b307d Merge pull request #20540 from victortoso/usb-host-passthrough
qemu: add usb host passthrough
2023-11-13 16:03:30 +00:00
79eec47f5f Merge pull request #20626 from jakecorrenti/ignition-builder
Refactor Ignition configuration for virt providers
2023-11-09 19:54:00 +00:00
39d647a2f6 Fix socket mapping socket mapping nits
Fixes nits that were suggested in #20420. The caller of
`ListenAndWaitOnSocket` did not use the value returned by the conn
channel, therefore it was better to just close the conn in the
`ListenAndWaitOnSocket` function instead.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-11-09 10:41:50 -05:00
c23963d7a8 machine: qemu: add usb host passthrough
QEMU usb-host driver which is the one for passthrough, supports two
options for selecting an USB devices in the host to provide it to the
VM:
 - Bus and Device number the device is plugged
 - Vendor and Product information of the USB devices

    https://qemu-project.gitlab.io/qemu/system/devices/usb.html

This commit allows a user to configure podman machine with either of
options, with new --usb command line option for podman machine init.

Examples
  podman machine init tosovm4 --usb vendor=13d3,product=5406
  podman machine init tosovm3 --usb bus=1,devnum=4 --usb bus=1,devnum=3

This commit also allows a user to change the USBs configured with
--usb command line option for podman machine set.

Note that this commit does not handle host device permissions nor
verify that the USB devices exists.

Signed-off-by: Victor Toso <victortoso@redhat.com>
2023-11-08 23:38:53 +01:00
b2c56365b2 Refactor Ignition configuration for virt providers
Creates a common SetIgnitionFile function in pkg/machine/ignition.go which
creates the new VMFile that will represent the machine's ignition file. It
assigns the VMFile to the provided location.

Creates an IgnitionBuilder type to generate the ignition configuration for a
given virt provider.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-11-08 13:39:34 -05:00
a45ba06d02 Refactor key machine objects
In #20538, I was asked to consider refactoring the new OCI pull code
from within the generic machine directory.  This is something I had
tried when originally coding it but it became apparent that a much
larger refactor to prevent circular deps was needed.  Because I did not
want to pollute the initial PR with that refactor, I asked for the PR to
merge first.  This is the refactor that needed to be done.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-11-07 08:30:44 -06:00
5fc7585334 Merge pull request #20420 from jakecorrenti/refactor-socket-mapping
Refactor machine socket mapping
2023-11-03 18:33:54 +00:00
ce9c1b9b86 Refactor machine socket mapping
Refactors machine socket mapping to prevent using similar/the same code
paths. Moves the shared code to `pkg/machine/sockets.go`.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-11-01 23:09:17 -04:00
7153124f97 Log gvproxy and server9 to file on log-level=debug
Logging to os.Stdout and os.Stderr does not seem to work in
Powershell. I am not entirely certain why.

Logfiles are the best alternative I can think of.

Signed-off-by: Matt Heon <mheon@redhat.com>
2023-10-31 10:14:06 -04:00
642fa98976 Initial addition of 9p code to Podman
This includes two new hidden commands: a 9p server,
`podman machine server9p`, and a 9p client,
`podman machine client9p` with `server9p` currently only
configured to run on Windows and serve 9p via HyperV vsock, and
`client9p` only configured to run on Linux. The server is run by
`podman machine start` and has the same lifespan as gvproxy
(waits for the gvproxy PID to die before shutting down). The
client is run inside the VM, also by `podman machine start`, and
mounts uses kernel 9p mount code to complete the mount. It's
unfortunately not possible to use mount directly without the
wrapper; we need to set up the vsock and pass it to mount as an
FD.

In theory this can be generalized so that the server can run
anywhere and over almost any transport, but I haven't done this
here as I don't think we have a usecase other than HyperV right
now.

[NO NEW TESTS NEEDED] This requires changes to Podman in the VM,
so we need to wait until a build with this lands in FCOS to test.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2023-10-31 10:14:02 -04:00
d6f44d956d hyperV: Update lastUp time
LastUp now correctly reports the lastUp time for podman machine on
hyperv, for both inspect and list.

Signed-off-by: Ashley Cui <acui@redhat.com>
2023-10-20 15:00:48 -04:00
aa0e96e781 Merge pull request #20274 from ashley-cui/cleanup
Machine: Teardown on init failure
2023-10-13 14:22:46 +00:00
987dc2b8bb SetLock for all virt providers
Implements a shared `GetLock` function for virtualization providers. Returns
a pointer to a lockfile used for serializing write operations.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-10-12 12:06:31 -04:00
61e0b64b91 Machine: Teardown on init failure
If init fails, or if a SIGINT is sent during init, podman machine should remove all files and configs
created during the init. This includes config jsons, image files, ssh
id's, and system connections. On Windows, the VM instances are also
unregistered.

Signed-off-by: Ashley Cui <acui@redhat.com>
2023-10-12 09:26:06 -04:00
995a6a8ac4 Add podman socket info to machine inspect
HyperV and appleHV machine inspect commands were missing podman socket
info. This also fixes machine inspect tests on hyperV

Signed-off-by: Ashley Cui <acui@redhat.com>
2023-09-26 15:50:00 -04:00
65ce034863 hyperV: Respect rootful option on machine init
Fixed a bug where the rootful option to the podman machine init command
would not be written to to the machine config, and the rootful docker
sock would not be set.

Signed-off-by: Ashley Cui <acui@redhat.com>
2023-09-25 16:13:55 -04:00
9ba0022905 Merge pull request #20115 from baude/hypervstarting
hyperv: set more realistic starting state
2023-09-24 19:24:50 -04:00
08936db354 hyperv: set more realistic starting state
the window for hyperv's "Starting" state is very narrow; so to more
mimic qemu, we follow suit.  starting bools are set when the vm boots
and when it communicates back on the read socket.

this allows pkg/machine/init_test.go to pass

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-09-24 10:13:43 -05:00
2bef573f34 hyperv: use StopWithForce with remove
When doing a machine rm -f (force removal of a machine) or a machine
reset (force removal of all machines), there is no need to use a
"polite/soft" stop.

this will also speed up pkg/machine/e2e tests.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-09-24 09:49:40 -05:00
c66aa3b7bb fixes for pkg/machine/e2e on hyperv
some problems were found in machine tests on hyperv.

in the case of rootful, it is currently not implemented.  an issue #20092 has been
created for that problem.

there also seems to be a timezone issue between ignition and fcos right
now.  inquiries are in for that but no issue generated for that.  this
problem is not exclusive to hyperv by any means.

both of the above have been skipped or commented out.

otherwise, this fixes machine state reporting for consistency.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-09-21 16:14:57 -05:00
5b3801776b Various updates for hyperv and machine e2e tests
This PR is a mishmash of updates needed so that the hyperv provider can
begin to passd the machine e2e tests.

Summary as follows:
* Added custom error handling for machine errors so that all providers
  can generate the same formatted error messages.  The ones implemented
  thus far are needed for the basic and init tests.  More will come as
  they are identified.
* Vendored new libhvee for better memory inspection.  The memory type
  changed from uint32 to uint64.
* Some machine e2e tests used linux-specific utilities to check various
  error conditions and messages (like pgrep).  Those were made into
  functions and implemented on an operating system level.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-09-21 08:52:02 -05:00
f3e9a6369c windows cannot "do" extra files
if you use the extra files function on exec commands, windows loses it's
mind.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-09-20 15:14:08 -05:00
289e59ee1f Implement gvproxy networking using cmdline wrapper
Converts the host networking code in `podman machine` to use the
`GvproxyCommand` type introduced in containers/gvisor-tap-vsock#258

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-09-19 09:20:26 -04:00
a62abfe3d6 Tweaks and cleanups to prepare hyperv for CI
Small fixes for bugs in the hyperv code that were made obvious when
manually preparing to run pkg/machine/e2e with windows and hyperv.

Also includes vendoring a new libhvee and solves bug where json config
was not being removed.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-08-30 12:44:50 -05:00
1085177fee hyperv ignition: use gvforwarder instead of vm
in gvisor-vsock-tap upstream, there is a binary called 'vm' which is
used for routing traffic from a tap over something like vsock.  In
Fedora, the binary is named 'gvforwarder'.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-08-29 09:34:40 -05:00
2a8e6acecb Set remote username earlier for hyperv
the remote username was being set too "late" for hyperv and the username
for ssh connections was blank.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-08-29 09:34:40 -05:00
da81bc13a1 Add rootful status to machine inspect
Podman machine inspect now shows if the machine is rootful

Signed-off-by: Ashley Cui <acui@redhat.com>
2023-08-25 11:27:08 -04:00
d3618719b1 Dedup and refactor image acquisition
As promised in #19596, this pr deduplicates and refactors image
acquisition.  All virt providers that use FCOS as its default now use
the same code.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-08-24 20:52:03 -05:00
0772d8ddb0 Stop gvproxy on hyperv machine stop
when we stop a machine, we need to also stop the gvproxy process that is
running.

JIRA: RUN-1828

also, remove unused applehv function for ssh

Signed-off-by: Brent Baude <bbaude@redhat.com>

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-08-17 15:21:06 -05:00
9e14e3ebe5 enabled hyperv image downloads
now that we have public hypverv fcos artifacts, we can download them
instead of requiring a special build.

Signed-off-by: Brent Baude <bbaude@redhat.com>

[NO NEW TESTS NEEDED]
2023-08-15 07:45:29 -05:00
69e883493b Fix regression for hyperv
the method caller for creating the ignition file was accidently
misordered for hyperv.  this regression was caused by aa6827a6.

Signed-off-by: Brent Baude <bbaude@redhat.com>

[NO NEW TESTS NEEDED]
2023-08-04 09:10:10 -05:00
21ebe0e90a Move writeConfig logic to shared function
Moves the shared logic from `writeConfig` into a shared function in
`pkg/machine/machine_common.go`

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-08-01 21:40:14 -04:00
597ccff0bc Move some logic of setRootful to a common file
Moves most of the logic of `setRootful` to the common file
`pkg/machine/machine_common.go`.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-08-01 21:40:06 -04:00
98cf8462ad move removeFilesAndConnections to a common file
Moves `removeFilesAndConnections` to the common file
`pkg/machine/connections.go` to be reused by multiple hypervisors.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-08-01 21:13:58 -04:00
55c7b5ceca Move addSSHConnectionsToPodmanSocket code to shared file
Moves the implementation of `addSSHConnectionsToPodmanSocket` into the
common file `pkg/machine/machine_common.go`. The implementation was
shared between the hypervisors and does not need to be implemented
multiple times.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-08-01 21:13:52 -04:00
906af5bbc6 Move getDevNullFiles into a common file
Moves `getDevNullFiles` into a new common file,
`pkg/machine/machine_common.go`, preventing the re-implementation of the
function across the different hypervisor implementations.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-08-01 08:52:23 -04:00
330e51960e Merge pull request #19364 from jakecorrenti/breakup-hyperv-machine-funcs
Breakup hyperv machine funcs
2023-07-31 16:25:03 +02:00
19deb6a346 Adds documentation to new functions that were added
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 19:58:03 -04:00
481143e584 startHostNetworking: get DevNull files
[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 18:39:45 -04:00
13da82466a Remove: remove network and ready sockets from registry
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 18:39:41 -04:00
d896431b28 Remove: remove files and connections
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 18:31:30 -04:00
32265e1f64 Remove: collect files to destroy
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 18:24:13 -04:00
381b103dd1 Init: read and split ign file
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 18:22:49 -04:00
aa6827a67a Init: write ign config
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 18:21:48 -04:00
cc30639d7b Init: add network and registry socks to registry
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 18:19:34 -04:00
78d3b549cd Init: add SSH conns to podman sock
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-30 18:16:32 -04:00
837bc25681 Fix HyperV loadMachineFromJSON function name
Re-names HyperV function that was copied from the applehv
implementation and not changed. Makes the function a method of
`HyperVMachine`.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-07-28 11:14:13 -04:00
9fb2f8e100 podman machine start: fix ready service
When debugging #17403, the logs of sshd indicates that Podman tried to
ssh into the machine too soon as the `core` user has not yet been fully
set up:

 > error: kex_exchange_identification: Connection closed by remote host
 > fatal: Access denied for user core by PAM account configuration [preauth]

@dustymabe found that the we may have to wait for systemd-user sessions
to be up.  Doing that reduces the flake rate on my M2 machine but does
not entirely fix the issue.

Since I have seen multiple symptoms of flakiness, I think it does not
hurt to add the systemd-user sessions to the dependencies of the ready
service and continue investigating.

[NO NEW TESTS NEEDED] - once we have a fix out, I want to exercise
frequent stop/start in the machine tests but they won't pass now.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-30 10:50:30 +02:00
516034215f Re-organize hypervisor implementations
Ensures that for each hypervisor implementation, their `config.go` file
deals with implementing the `VirtProvider` interface while the
`machine.go` file is for implementing the `VM` interface.

Moves the `Virtualization` type into a common file and
created wrappers for the individual hypervisors. Allows for shared
functions that are exactly the same while providing the flexibility to
create hypervisor-specific implementations of the functions.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2023-06-23 11:33:19 -04:00
c7a8d29f12 refactor: improve get ssh path duplicate code
Signed-off-by: Black-Hole1 <bh@bugs.cc>
2023-06-07 09:03:35 +08:00