2708 Commits

Author SHA1 Message Date
895b815188 Merge pull request #10836 from Luap99/diff
podman diff accept two images or containers
2021-07-03 06:17:03 -04:00
ed51e3f548 podman service reaper
Add a new service reaper package. Podman currently does not reap all
child processes. The slirp4netns and rootlesskit processes are not
reaped. The is not a problem for local podman since the podman process
dies before the other processes and then init will reap them for us.

However with podman system service it is possible that the podman
process is still alive after slirp died. In this case podman has to reap
it or the slirp process will be a zombie until the service is stopped.

The service reaper will listen in an extra goroutine on SIGCHLD. Once it
receives this signal it will try to reap all pids that were added with
`AddPID()`. While I would like to just reap all children this is not
possible because many parts of the code use `os/exec` with `cmd.Wait()`.
If we reap before `cmd.Wait()` things can break, so reaping everything
is not an option.

[NO TESTS NEEDED]

Fixes #9777

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-07-02 19:00:36 +02:00
8f6a0243f4 podman diff accept two images or containers
First, make podman diff accept optionally a second argument. This allows
the user to specify a second image/container to compare the first with.
If it is not set the parent layer will be used as before.

Second, podman container diff should only use containers and podman
image diff should only use images. Previously, podman container diff
would use the image when both an image and container with this name
exists.

To make this work two new parameters have been added to the api. If they
are not used the previous behaviour is used. The same applies to the
bindings.

Fixes #10649

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-07-02 17:11:56 +02:00
86c6014145 Implement --archive flag for podman cp
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-07-01 12:01:46 +02:00
f95b0995e5 remove pkg/registries
Pull the trigger on the `pkg/registries` package which acted as a proxy
for `c/image/pkg/sysregistriesv2`.  Callers should be using the packages
from c/image directly, if needed at all.

Also make use of libimage's SystemContext() method which returns a copy
of a system context, further reducing the risk of unintentionally
altering global data.

[NO TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-06-25 09:56:21 +02:00
fc34f35e3f Merge pull request #10765 from umohnani8/mtab
Create /etc/mtab with the correct ownership
2021-06-23 16:12:45 -04:00
3f3feaa015 Merge pull request #10716 from cdoern/podFlags
Podman Pod Create --cpus and --cpuset-cpus flags
2021-06-23 15:51:29 -04:00
9db534e53a [NO TESTS NEEDED] Create /etc/mtab with the correct ownership
Create the /etc and /etc/mtab directories with the
correct ownership based on what the UID and GID is
for the container. This was causing issue when starting
the infra container with userns as the /etc directory
wasn't being created with the correct ownership.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-06-23 14:46:40 -04:00
2b850ef055 Merge pull request #10696 from cdoern/libpodStats
Podman Stats additional features
2021-06-23 14:43:30 -04:00
bbd085ad1e Podman Pod Create --cpus and --cpuset-cpus flags
Added logic and handling for two new Podman pod create Flags.

--cpus specifies the total number of cores on which the pod can execute, this
is a combination of the period and quota for the CPU.

 --cpuset-cpus is a string value which determines of these available cores,
how many we will truly execute on.

Signed-off-by: cdoern <cbdoer23@g.holycross.edu>
2021-06-23 13:47:57 -04:00
f26fa53921 Podman Stats additional features
added Avg Cpu calculation and CPU up time to podman stats. Adding different feature sets in different PRs, CPU first.

resolves #9258

Signed-off-by: cdoern <cbdoer23@g.holycross.edu>
2021-06-23 09:23:40 -04:00
e159eb892b Merge pull request #10754 from Luap99/sync-lock
getContainerNetworkInfo: lock netNsCtr before sync
2021-06-23 04:25:44 -04:00
a84fa194b7 getContainerNetworkInfo: lock netNsCtr before sync
`syncContainer()` requires the container to be locked, otherwise we can
end up with undefined behavior.

[NO TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-22 16:51:21 +02:00
e014608539 Do not use inotify for OCICNI
Podman does not need to watch the cni config directory. If a network is
not found in the cache, OCICNI will reload the networks anyway and thus
even podman system service should work as expected.
Also include a change to not mount a "new" /var by default in the
rootless cni ns, instead try to use /var/lib/cni first and then the
parent dir. This allows users to store cni configs under /var/... which
is the case for the CI compose test.

[NO TESTS NEEDED]

Fixes #10686

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-22 16:00:47 +02:00
be15e69a61 Merge pull request #10742 from vrothberg/fix-10675
logs: k8s-file: restore poll sleep
2021-06-21 13:06:04 -04:00
ee4cab0e09 logs: k8s-file: restore poll sleep
Commit 84b55eec2796 attempted to fix a race waiting for the container
died event.  Previously, Podman slept for duration of the polling
frequence which I considerred to be a mistake.  As it turns out, I was
mistaken since the file logger will, in fact, NOT read until EOF and
then stop logging but stop logging immediately _after_ it woke up.

[NO TESTS NEEDED] as the race condition cannot be hit reliably.

Fixes: #10675
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-06-21 15:44:13 +02:00
f4ba433b18 logs: k8s-file: fix spurious error logs
Fix the suprious "Error: nil" messages.  Also add some more context to
logged error messages which makes error sources more obvious.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-06-21 14:11:27 +02:00
18bf92f7ea Merge pull request #10734 from mgoltzsche/fix-dns-detection
Fix systemd-resolved detection.
2021-06-21 07:06:34 -04:00
0fb165ed08 Fix systemd-resolved detection.
Previously podman failed when run in an environment where 127.0.0.53 is
the only nameserver but systemd-resolved is not used directly.
In practice this happened when podman was run within an alpine container
that used the host's network and the host was running systemd-resolved.

This fix makes podman ignore a file not found error when reading /run/systemd/resolve/resolv.conf.

Closes #10733

[NO TESTS NEEDED]

Signed-off-by: Max Goltzsche <max.goltzsche@gmail.com>
2021-06-20 17:17:00 +02:00
7864108ff1 fix systemcontext to use correct TMPDIR
Users are complaining about read/only /var/tmp failing
even if TMPDIR=/tmp is set.

This PR Fixes: https://github.com/containers/podman/issues/10698

[NO TESTS NEEDED] No way to test this.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-06-18 19:53:24 -04:00
2509a81c34 Merge pull request #10683 from Luap99/exec-resize
Fix resize race with podman exec -it
2021-06-16 15:29:34 -04:00
666f555aa5 Fix resize race with podman exec -it
When starting a process with `podman exec -it` the terminal is resized
after the process is started. To fix this allow exec start to accept the
terminal height and width as parameter and let it resize right before
the process is started.

Fixes #10560

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-16 16:43:30 +02:00
85bf412bb5 Merge pull request #10684 from Luap99/slirp-hosts
add correct slirp ip to /etc/hosts
2021-06-16 10:10:17 -04:00
9c81b8cf72 add correct slirp ip to /etc/hosts
The container name should have the slirp interface ip set in /etc/hosts
and not the gateway ip. Commit c8dfcce6db0a introduced this regression.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1972073

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-15 13:41:14 +02:00
81eb71fe36 Fix permissions on initially created named volumes
Permission of volume should match the directory it is being mounted on.

Fixes: https://github.com/containers/podman/issues/10188

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-06-14 11:56:48 -04:00
328174df27 Merge pull request #10635 from adrianreber/2021-06-04-privileged
Fix restoring of privileged containers
2021-06-12 14:56:42 -04:00
d1cb5d14e9 Merge pull request #10654 from Luap99/net-connect
Fix network connect race with docker-compose
2021-06-12 05:02:43 -04:00
45dc3d699d Merge pull request #10405 from mheon/always_cleanup_exec
Always spawn a cleanup process with exec
2021-06-11 16:32:43 -04:00
af9d690690 Merge pull request #10638 from Luap99/volume
Fix volumes with uid and gid options
2021-06-11 16:31:45 -04:00
44d9c453d3 Fix network connect race with docker-compose
Network connect/disconnect has to call the cni plugins when the network
namespace is already configured. This is the case for `ContainerStateRunning`
and `ContainerStateCreated`. This is important otherwise the network is
not attached to this network namespace and libpod will throw errors like
`network inspection mismatch...` This problem happened when using
`docker-compose up` in attached mode.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-11 16:00:12 +02:00
ad3b56c62f Fix volumes with uid and gid options
Podman uses the volume option map to check if it has to mount the volume
or not when the container is started. Commit 28138dafcc39 added to uid
and gid options to this map, however when only uid/gid is set we cannot
mount this volume because there is no filesystem or device specified.
Make sure we do not try to mount the volume when only the uid/gid option
is set since this is a simple chown operation.

Also when a uid/gid is explicity set, do not chown the volume based on
the container user when the volume is used for the first time.

Fixes #10620

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-11 13:27:08 +02:00
0c38ac44ef Merge pull request #10609 from giuseppe/ignore-named-hierarchies
container: ignore named hierarchies
2021-06-10 17:01:41 -04:00
62f4b0a195 Add ExecDied event and use it to retrieve exit codes
When making Exec Cleanup processes mandatory, I introduced a race
wherein attached exec sessions could be cleaned up and removed by
the cleanup process before the frontend had a chance to get their
exit code. Fortunately, we've dealt with this issue before in
containers, and the same solution can be applied here. I added an
event for an exec session's process exiting, `exec_died` (Docker
has an identical event, so this actually improves our
compatibility there) that includes the exit code of the exec
session. If the race happens and the exec session no longer
exists when we go to remove it, pick up exit code from the event
and exit cleanly.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-06-10 14:17:41 -04:00
341e6a1628 Always spawn a cleanup process with exec
We were previously only doing this for detached exec. I don't
know why we did that, but I don't see any reason not to extend it
to all exec sessions - it guarantees that we will always clean up
exec sessions, even if the original `podman exec` process died.

[NO TESTS NEEDED] because I don't really know how to test this
one.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-06-10 14:16:22 -04:00
240bbc3bfa Fix pre-checkpointing
Unfortunately --pre-checkpointing never worked as intended and recent
changes to runc have shown that it is broken.

To create a pre-checkpoint CRIU expects the paths between the
pre-checkpoints to be a relative path. If having a previous checkpoint
it needs the be referenced like this: --prev-images-dir ../parent

Unfortunately Podman was giving runc (and CRIU) an absolute path.

Unfortunately, again, until March 2021 CRIU silently ignored if
the path was not relative and switch back to normal checkpointing.

This has been now fixed in CRIU and runc and running pre-checkpoint
with the latest runc fails, because runc already sees that the path is
absolute and returns an error.

This commit fixes this by giving runc a relative path.

This commit also fixes a second pre-checkpointing error which was just
recently introduced.

So summarizing: pre-checkpointing never worked correctly because CRIU
ignored wrong parameters and recent changes broke it even more.

Now both errors should be fixed.

[NO TESTS NEEDED]

Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Adrian Reber <adrian@lisas.de>
2021-06-10 15:29:24 +02:00
3b6cb8fabb container: ignore named hierarchies
when looking up the container cgroup, ignore named hierarchies since
containers running systemd as payload will create a sub-cgroup and
move themselves there.

Closes: https://github.com/containers/podman/issues/10602

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-06-10 14:49:58 +02:00
d9a1c34e43 Fix restoring of privileged containers
Checkpointed containers started with --privileged fail during restore
with:

 Error: error creating container storage: ProcessLabel and Mountlabel must either not be specified or both specified

This commit fixes it by not setting the labels when restoring a
privileged container.

[NO TESTS NEEDED]

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-06-10 12:18:03 +02:00
b5e5730f0c Merge pull request #10600 from vrothberg/fix-10596
logs: k8s-file: fix race
2021-06-08 21:17:26 +02:00
18fa124dfc Improve systemd-resolved detection
When 127.0.0.53 is the only nameserver in /etc/resolv.conf assume
systemd-resolved is used. This is better because /etc/resolv.conf does
not have to be symlinked to /run/systemd/resolve/stub-resolv.conf in
order to use systemd-resolved.

[NO TESTS NEEDED]

Fixes: #10570

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-08 18:14:00 +02:00
84b55eec27 logs: k8s-file: fix race
Fix a race in the k8s-file logs driver.  When "following" the logs,
Podman will print the container's logs until the end.  Previously,
Podman logged until the state transitioned into something non-running
which opened up a race with the container still running, possibly in
the "stopping" state.

To fix the race, log until we've seen the wait event for the specific
container.  In that case, conmon will have finished writing all logs to
the file, and Podman will read it until EOF.

Further tweak the integration tests for testing `logs -f` on a  running
container.  Previously, the test only checked for one of two lines
stating that there was a race.  Indeed the race was in using `run --rm`
where a log file may be removed before we could fully read it.

Fixes: #10596
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-06-08 16:14:13 +02:00
8aa5340ade Add parameter to specify checkpoint archive compression
The checkpoint archive compression was hardcoded to `archive.Gzip`.

There have been requests to make the used compression algorithm
selectable. There was especially the request to not compress the
checkpoint archive to be able to create faster checkpoints when not
compressing it.

This also changes the default from `gzip` to `zstd`. This change should
not break anything as the restore code path automatically handles
whatever compression the user provides during restore.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-06-07 08:07:15 +02:00
df2e7e00fc add ipv6 nameservers only when the container has ipv6 enabled
The containers /etc/resolv.conf allways preserved the ipv6 nameserves
from the host even when the container did not supported ipv6. Check
if the cni result contains an ipv6 address or slirp4netns has ipv6
support enabled and only add the ipv6 nameservers when this is the case.

The test needs to have an ipv6 nameserver in the hosts /etc/hosts but we
should never mess with this file on the host. Therefore the test is
skipped when no ipv6 is detected.

Fixes #10158

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-03 10:19:36 +02:00
52dae693da Merge pull request #10514 from vrothberg/fix-10507
events: support disjunctive filters
2021-06-02 10:37:58 -04:00
37f39eefee events: support disjunctive filters
While different filters are applied in conjunction, the same filter (but
with different values) should be applied in disjunction.  This allows,
for instance, to query the events of two containers.

Fixes: #10507
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-06-02 13:37:56 +02:00
7ef3981abe Enable port forwarding on host
Using the gvproxy application on the host, we can now port forward from
the machine vm on the host.  It requires that 'gvproxy' be installed in
an executable location.  gvproxy can be found in the
containers/gvisor-tap-vsock github repo.

[NO TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-06-01 10:13:18 -05:00
9822c3309c create libimage-events channel in main routine
Move the creation of the channel outside of the sub-routine to fix a
data race between writing the channel (implicitly by calling
EventChannel()) and using that channel in libimage.

[NO TESTS NEEDED]

Fixes: #10459
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-05-27 16:52:56 +02:00
542d730987 Merge pull request #10476 from mheon/ensure_exists_on_remove
Ensure that container still exists when removing
2021-05-27 08:55:03 +02:00
d9eb126925 Merge pull request #10447 from mheon/rootlesscni_default
Add the option of Rootless CNI networking by default
2021-05-26 23:13:40 +02:00
5b4ffc7ba7 Merge pull request #10431 from vrothberg/journald-logs
journald logger: fix race condition
2021-05-26 22:38:37 +02:00
fad6e1d3ef Ensure that container still exists when removing
After #8906, there is a potential race condition in container
removal of running containers with `--rm`. Running containers
must first be stopped, which was changed to unlock the container
to allow commands like `podman ps` to continue to run while
stopping; however, this also means that the cleanup process can
potentially run before we re-lock, and remove the container from
under us, resulting in error messages from `podman rm`. The end
result is unchanged, the container is still cleanly removed, but
the `podman rm` command will seem to have failed.

Work around this by pinging the database after we stop the
container to make sure it still exists. If it doesn't, our job is
done and we can exit cleanly.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-05-26 15:33:28 -04:00