13458 Commits

Author SHA1 Message Date
b24110ea56 Merge pull request #12333 from rst0git/file-locks
Add --file-locks checkpoint/restore option
2021-11-18 21:57:16 +01:00
7098463e78 Add test for checkpoint/restore with --file-locks
Signed-off-by: Radostin Stoyanov <radostin@redhat.com>
2021-11-18 19:23:48 +00:00
6d23ea60d2 Add --file-locks checkpoint/restore option
CRIU supports checkpoint/restore of file locks. This feature is
required to checkpoint/restore containers running applications
such as MySQL.

Signed-off-by: Radostin Stoyanov <radostin@redhat.com>
2021-11-18 19:23:25 +00:00
0376e6092c Merge pull request #12342 from vrothberg/fix-12334
remote checkpoint/restore: more fixes
2021-11-18 19:48:16 +01:00
fceecc3a5b remote checkpoint/restore: more fixes
* Support `checkpoint --pre-checkpoint`

* Support `checkpoint --with-previous`

* Disable `restore --import-previous` for the remote client since we had
  to send two files which in turn would require to tar them up and hence
  be a breaking change.  Podman 4.0 would be the chance and I hope we'll
  find time before that to remote-restore prettier.

Note that I did not run over swagger yet to check whether all parameters
are actually documented due to time constraints.

Fixes: #12334
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-18 16:52:15 +01:00
9b964945d6 Merge pull request #12343 from vrothberg/fix-ci
fix CI
2021-11-18 16:49:53 +01:00
6f6a6925b2 fix CI
Our fedora-minimal image on Quay bases on fedora-minimal:latest which
starting with F35 removed a number of binaries that our CI depends on.
Fix that by pulling `fedora-minimal:34` from the Fedora registry
directly.

Once the build bot on Quay has been disabled, we move the image over
there to make sure that it will not change over time.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-18 15:19:14 +01:00
69e1204e5e Merge pull request #12330 from etenzy/main
fix: take absolute path for dd on apple silicon
2021-11-18 12:44:24 +01:00
878d22e831 Merge pull request #12325 from giuseppe/rootless-use-auto-cleanup
rootless: use auto cleanup functions
2021-11-17 20:20:29 +01:00
55b2efc028 Merge pull request #12326 from giuseppe/catatonit-rootless
rootless: use catatonit to maintain user+mnt namespace
2021-11-17 19:53:36 +01:00
010bee04c3 fix: take absolute path for dd on apple silicon
Fixes #12329

[NO NEW TESTS NEEDED] podman machine

Signed-off-by: Michael Rödel <hello@mroedel.de>
2021-11-17 17:02:04 +01:00
3073543fd9 rootless: use catatonit to maintain user+mnt namespace
if catatonit is present, use it to keep the rootless user+mnt
namespace alive.

[NO NEW TESTS NEEDED] no new features added.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-17 12:30:33 +01:00
85733e0b4a Merge pull request #12281 from vrothberg/fix-12007
fix remote checkpoint/restore
2021-11-17 12:02:37 +01:00
bd6fbb190b Merge pull request #12303 from rhatdan/tmpdir
Set config environment variables early in Podman init
2021-11-17 11:42:39 +01:00
802f26ef10 Merge pull request #12323 from giuseppe/utils-not-unique-pause-scope-name
utils: use podman-pause-$RANDOM.scope name
2021-11-17 11:39:37 +01:00
cb9de1007c rootless: drop strerror(errno) calls
*printf functions already support printing the errno string with %m

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-17 10:57:44 +01:00
084e32336a rootless: reuse existing open_namespace function
there is already a function for opening a namespace path, reuse it.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-17 10:57:44 +01:00
277d526869 rootless: use auto cleanup functions
simplify code using auto cleanup functions

[NO NEW TESTS NEEDED] it is a refactoring of existing code

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-17 10:57:44 +01:00
ee62711136 utils: use podman-pause-$RANDOM.scope name
we try hard to re-use the existing podman-pause.scope name when it
already exists, causing any sort of race errors when the already
existing scope is terminating.

There is no such a requirement though, so just try with a random
name.

Closes: https://github.com/containers/podman/issues/12065

[NO NEW TESTS NEEDED] it fixes a race in the CI

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-17 10:13:33 +01:00
c661664878 Merge pull request #12319 from Luap99/nettypes-rename
rename libpod nettypes fields
2021-11-17 00:21:31 +01:00
059785c3ed Merge pull request #12322 from edsantiago/bats_helper_bins
hack/bats: deal with new bin helpers
2021-11-16 22:51:32 +01:00
e367f46147 hack/bats: deal with new bin helpers
Some time in the last month, podman started to depend on a bunch
of external helper binaries: rootlessport, pause, catatonit.
System tests fail without these.

Update the hack/bats script to pass $CONTAINERS_HELPER_BINARIES_DIR
(set to ./bin); podman will then use locally-built helpers. (This
requires https://github.com/containers/common/pull/823 , which as
of this PR is not yet vendored into podman. There is no harm in
merging this while we wait.)

Also: if bats helper is invoked as root, run only once; i.e.,
skip the "rootless" step.

Also (piggybacked): the name of the podman pause image has
changed, from pause to podman-pause. Adjust that in our
teardown so we don't leave droppings.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-11-16 12:51:17 -07:00
97c6403a1b rename libpod nettypes fields
Some field names are confusing. Change them so that they make more sense
to the reader.
Since these fields are only in the main branch we can safely rename them
without worrying about backwards compatibility.
Note we have to change the field names in netavark too.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-16 19:19:04 +01:00
197ebe8b5e Merge pull request #12218 from vrothberg/pause-catatonit
infra container: replace pause with catatonit
2021-11-16 18:16:34 +01:00
f031bd23c6 Merge pull request #12100 from rhatdan/env
Add option --unsetenv to remove default environment variables
2021-11-16 16:27:34 +01:00
8430ffc72e Merge pull request #12283 from Luap99/machine-ports
podman machine improve port forwarding
2021-11-16 14:53:40 +01:00
33ec8c6698 fix remote checkpoint/restore
Nothing was working before, and it's too much to summarize.  To make
sure we're not regressing in the future again, enable the remote e2e
tests.

Fixes: #12007
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-16 14:41:18 +01:00
be681ab518 Merge pull request #12294 from flouthoc/secret-mount-target
secret: honor custom `target=` for secrets with `type=mount` for ctr.
2021-11-16 01:45:27 +01:00
45d28c2219 Merge pull request #12285 from nalind/journal-follow-not-early
journald logs: keep reading until the journal's end
2021-11-15 22:09:29 +01:00
44d1618dd7 Add --unsetenv & --unsetenv-all to remove def environment variables
Podman adds a few environment variables by default, and
currently there is no way to get rid of them from your container.
This option will allow  you to specify which defaults you don't
want.

--unsetenv-all will remove all default environment variables.

Default environment variables can come from podman builtin,
containers.conf or from the container image.

Fixes: https://github.com/containers/podman/issues/11836

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-15 15:10:12 -05:00
230f0b622e Merge pull request #12130 from rhatdan/journal
Error logs --follow if events-backend != journald, event-logger=journald
2021-11-15 20:55:28 +01:00
364b242b70 Set config environment variables early in Podman init
Fixes: https://github.com/containers/podman/issues/12296

[NO NEW TESTS NEEDED] because there is no easy way to test this.
Tests are in containers/common.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-15 14:36:23 -05:00
63ef7135d9 journald logs: keep reading until the journal's end
When reading logs from the journal, keep going after the container
exits, in case it gets restarted.

Events logged to the journal via the normal paths don't include
CONTAINER_ID_FULL, so don't bother adding it to the "history" event we
use to force at least one entry for the container to show up in the log.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-11-15 13:38:36 -05:00
014cc4b9d9 secret: honor custom target for secrets with run
Honor custom `target` if specified while running or creating containers
with secret `type=mount`.

Example:
`podman run -it --secret token,type=mount,target=TOKEN ubi8/ubi:latest
bash`

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-15 23:19:27 +05:30
47eaf23039 Merge pull request #12300 from WanzenBug/fix-bindings-context
bindings: reuse context for API requests
2021-11-15 18:31:37 +01:00
9eaa6f4566 Merge pull request #12297 from edsantiago/no_remote_unshare
Network test: fix podman-remote-rootless corner case
2021-11-15 16:42:40 +01:00
e9d8ca22b8 Merge pull request #11076 from boaz0/closes_10275
Support template unit files in podman generate systemd
2021-11-15 16:30:38 +01:00
d40736fef1 Merge pull request #12257 from adrianreber/2021-11-10-print-stats
Add optional checkpoint/restore statistics
2021-11-15 16:17:37 +01:00
5df883e87d bindings: reuse context for API requests
One of the main uses of context.Context is to provide cancellation for
go-routines, including API requests. While all user-facing bindings
already used a context parameter, it was only used to pass the client
information around.

This commit changes the internal DoRequest wrapper to take an additional
context argument, and pass that to the http request. Previously, the context
was derived from context.Background(), which made it impossible to cancel
once started.

All the convenience wrappers already supported the context parameter, so the
only user facing change is that cancelling those context now works as one
would expect.

Signed-off-by: Moritz "WanzenBug" Wanzenböck <moritz@wanzenbug.xyz>
2021-11-15 15:42:39 +01:00
295d87bb0b podman machine improve port forwarding
This commits adds port forwarding logic directly into podman. The
podman-machine cni plugin is no longer needed.

The following new features are supported:
 - works with cni, netavark and slirp4netns
 - ports can use the hostIP to bind instead of hard coding 0.0.0.0
 - gvproxy no longer listens on 0.0.0.0:7777 (requires a new gvproxy
   version)
 - support the udp protocol

With this we no longer need podman-machine-cni and should remove it from
the packaging. There is also a change to make sure we are backwards
compatible with old config which include this plugin.

Fixes #11528
Fixes #11728

[NO NEW TESTS NEEDED] We have no podman machine test at the moment.
Please test this manually on your system.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-15 15:20:47 +01:00
c21259bf86 Network test: fix podman-remote-rootless corner case
Followup to #12229, in which I added a podman unshare for
flake debugging. Turns out that doesn't work in podman-remote.
It was not caught because CI doesn't run podman-remote rootless.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-11-15 07:01:22 -07:00
c8790bfbbb cirrus: force-install catatonit
A temporary workaround until the CI images are updated.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-15 12:53:25 +01:00
5934e4c9b5 infra container: replace pause with catatonit
Podman has been using catatonit for a number of years already.
Thanks to @giuseppe, catatonit is now able to run as a pause
process which allows us to replace the pause binary entirely.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-15 12:53:25 +01:00
58cf0d4622 Revert "add kubernetes pause"
This reverts commit 9d2b8d2791c23b83b6155b046099a83483860c56 since
catatonit's new pause functionality can replace the `pause` binary
entirely.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-15 12:53:25 +01:00
d28b39a90d Added test for checkpoint/restore --print-stats
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
914f4c8905 Update man pages for checkpoint/restore --print-stats
This commit updates the man pages for checkpoint and restore to describe
the '--print-stats' parameter.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
80e56fa12b Added optional container restore statistics
This adds the parameter '--print-stats' to 'podman container restore'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to restore a checkpoint and print out these
information. CRIU already creates process restore statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the restored container, Podman will now print
out JSON:

 # podman container restore --latest --print-stats
 {
     "podman_restore_duration": 305871,
     "container_statistics": [
         {
             "Id": "47b02e1d474b5d5fe917825e91ac653efa757c91e5a81a368d771a78f6b5ed20",
             "runtime_restore_duration": 140614,
             "criu_statistics": {
                 "forking_time": 5,
                 "restore_time": 67672,
                 "pages_restored": 14
             }
         }
     ]
 }

The output contains 'podman_restore_duration' which contains the
number of microseconds Podman required to restore the checkpoint. The
output also includes 'runtime_restore_duration' which is the time
the runtime needed to restore that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
6202e8102b Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:

 # podman container checkpoint --latest --print-stats
 {
     "podman_checkpoint_duration": 360749,
     "container_statistics": [
         {
             "Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
             "runtime_checkpoint_duration": 177222,
             "criu_statistics": {
                 "freezing_time": 100657,
                 "frozen_time": 60700,
                 "memdump_time": 8162,
                 "memwrite_time": 4224,
                 "pages_scanned": 20561,
                 "pages_written": 2129
             }
         }
     ]
 }

The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:24 +00:00
cca6df428c Merge pull request #12272 from hshiina/memory-swappiness
Enable 'podman run --memory-swappiness=0'
2021-11-13 13:12:48 +01:00
062c887718 Error logs --follow if events-backend != journald, event-logger=journald
Fixes: https://github.com/containers/podman/issues/11255

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-13 07:11:09 -05:00