2192 Commits

Author SHA1 Message Date
db684f970f Merge pull request #7971 from rhatdan/blob
BlobInfoCacheDir is set incorrectly when copying images
2020-10-09 05:04:27 -04:00
f7e11d34b1 BlobInfoCacheDir is set incorrectly when copying images
It is not set based on the root image directory, and always
points at the defaults.  This change will get it to follow
filepath.Join(ir.store.GraphRoot(), "cache") set from libpod.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-08 16:22:55 -04:00
4d800a5f45 Store cgroup manager on a per-container basis
When we create a container, we assign a cgroup parent based on
the current cgroup manager in use. This parent is only usable
with the cgroup manager the container is created with, so if the
default cgroup manager is later changed or overridden, the
container will not be able to start.

To solve this, store the cgroup manager that created the
container in container configuration, so we can guarantee a
container with a systemd cgroup parent will always be started
with systemd cgroups.

Unfortunately, this is very difficult to test in CI, due to the
fact that we hard-code cgroup manager on all invocations of
Podman in CI.

Fixes #7830

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-10-08 15:25:06 -04:00
0e1d01103e Merge pull request #7382 from mheon/pod_parallel
Move pod jobs to parallel execution
2020-10-07 15:06:02 -04:00
9ae873e60e Merge pull request #7943 from baude/issue7807
prevent unpredictable results with network create|remove
2020-10-07 13:56:56 -04:00
fe3faa517e prevent unpredictable results with network create|remove
due to a lack of "locking" on cni operations, we could get ourselves in trouble when doing rapid creation or removal of networks.  added a simple file lock to deal with the collision and because it is not considered a performent path, use of the file lock should be ok.  if proven otherwise in the future, some generic shared memory lock should be implemented for libpod and also used here.

moved pkog/network to libpod/network because libpod is now being pulled into the package and it has therefore lost its generic nature. this will make it easier to absorb into libpod as we try to make the network closer to core operations.

Fixes: #7807

Signed-off-by: baude <bbaude@redhat.com>
2020-10-07 10:03:21 -05:00
2bb2425704 Move pod jobs to parallel execution
Make Podman pod operations that do not involve starting
containers (which needs to be done in a specific order) use the
same parallel operation code we use to make `podman stop` on
large numbers of containers fast. We were previously stopping
containers in a pod serially, which could take up to the timeout
(default 15 seconds) for each container - stopping 100 containers
that do not respond to SIGTERM would take 25 minutes.

To do this, refactor the parallel operation code a bit to remove
its dependency on libpod (damn circular import restrictions...)
and use parallel functions that just re-use the standard
container API operations - maximizes code reuse (previously each
pod handler had a separate implementation of the container
function it performed).

This is a bit of a palate cleanser after fighting CI for two
days - nice to be able to return to a land of sanity.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-10-07 10:00:11 -04:00
3ae47f7d2b Populate /etc/hosts file when run in a user namespace
We do not populate the hostname field with the IP Address
when running within a user namespace.

Fixes https://github.com/containers/podman/issues/7490

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-07 08:39:44 -04:00
80a2317ca2 Merge pull request #7929 from kolyshkin/nits-err
Nits
2020-10-06 10:15:04 +02:00
684d0079d2 Lowercase some errors
This commit is courtesy of

```
for f in $(git ls-files *.go | grep -v ^vendor/); do \
	sed -i 's/\(errors\..*\)"Error /\1"error /' $f;
done

for f in $(git ls-files *.go | grep -v ^vendor/); do \
	sed -i 's/\(errors\..*\)"Failed to /\1"failed to /' $f;
done

```

etc.

Self-reviewed using `git diff --word-diff`, found no issues.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-10-05 15:56:44 -07:00
4878dff3e2 Remove excessive error wrapping
In case os.Open[File], os.Mkdir[All], ioutil.ReadFile and the like
fails, the error message already contains the file name and the
operation that fails, so there is no need to wrap the error with
something like "open %s failed".

While at it

 - replace a few places with os.Open, ioutil.ReadAll with
   ioutil.ReadFile.

 - replace errors.Wrapf with errors.Wrap for cases where there
   are no %-style arguments.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-10-05 15:30:37 -07:00
348f2df0c0 Support max_size logoptions
Docker supports log-opt max_size and so does conmon (ALthough poorly).
Adding support for this allows users to at least make sure their containers
logs do not become a DOS vector.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-05 17:51:45 -04:00
436dda2385 Merge pull request #7913 from andylibrian/kube-generate-support-resource-limits-7855
Add support for resource memory limit to generate kube
2020-10-05 12:15:04 +02:00
62607e7f27 Add TODO for adding CPU limit support
Signed-off-by: Andy Librian <andylibrian@gmail.com>
2020-10-05 12:50:39 +07:00
d6258eb6c2 Add support for resource memory limit to generate kube
addresses #7855

Signed-off-by: Andy Librian <andylibrian@gmail.com>
2020-10-04 21:17:00 +07:00
d0f3c17912 Add SELinux support for pods
All containers within a Pod need to run with the same SELinux
label, unless overwritten by the user.

Also added a bunch of SELinux tests to make sure selinux labels
are correct on namespaces.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-02 15:14:18 -04:00
1e162edc8a Merge pull request #7882 from giuseppe/check-for-gids-before-adding-them
libpod: check there are enough gids before adding them
2020-10-02 14:45:27 -04:00
1132bcfeb5 Merge pull request #7893 from ashley-cui/journald
Fix Podman logs reading journald
2020-10-02 14:24:53 -04:00
b58980a43c Merge pull request #7887 from vrothberg/fix-7872
image prune: remove all candidates
2020-10-02 11:31:43 -04:00
c0d1954663 Fix Podman logs reading journald
A podman could not read logs written to journald properly, due to a tail config bug.
Added a system test to check this - since e2e tests don't like journald

Signed-off-by: Ashley Cui <acui@redhat.com>
2020-10-02 10:05:19 -04:00
d30121969f libpod: check the gid is present before adding it
check there are enough gids in the user namespace before adding
supplementary gids from /etc/group.

Follow-up for baede7cd2776b1f722dcbb65cff6228eeab5db44

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-10-02 13:13:44 +02:00
51851e10ba Merge pull request #7622 from hxtk/master
Fix for incorrect evaluation of error condition within libpod.LabelVolumePath.
2020-10-02 06:36:53 -04:00
6c151b98b6 image prune: remove all candidates
Make sure to remove images until there's nothing left to prune.
A single iteration may not be sufficient.

Fixes: #7872
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-10-02 11:04:05 +02:00
baede7cd27 Add additionalGIDs from users in rootless mode
There is a risk here, that if the GID does not exists
within the User Namespace the container will fail to start.

This is only likely to happen in HPC Envioronments, and I think
we should add a field to disable it for this environment,
Added a FIXME for this issue.

We currently have this problem with running a rootfull container within
a user namespace, it will fail if the GID is not available.

I looked at potentially checking the usernamespace that you are assigned
to, but I believe this will be very difficult to code up and to figure out.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-01 10:54:28 -04:00
8ff35a098c image look up: consult registries.conf
When looking up local images, take the unqualified-serach registries of
the registries.conf into account (on top of "localhost/").

Also extend the integration tests to prevent future regressions.

Fixes: #6381
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-09-30 16:30:41 +02:00
451f003ce9 Merge pull request #7841 from AkihiroSuda/fix-7789
rootless-cni-infra v3: fix cleaning up DNS entries
2020-09-30 12:41:57 +00:00
08d036cbd4 Merge pull request #7825 from rhatdan/exitcode
Fix handling of remove of bogus volumes, networks and Pods
2020-09-30 11:33:19 +00:00
19f080f1af Merge pull request #7827 from vrothberg/systemd-run-error
healthchecks: return systemd-run error
2020-09-30 10:50:56 +00:00
515b919438 libpod: bump up rootless-cni-infra to v3
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-09-30 19:01:52 +09:00
22474095ab Fix handling of remove of bogus volumes, networks and Pods
In podman containers rm and podman images rm, the commands
exit with error code 1 if the object does not exists.

This PR implements similar functionality to volumes, networks, and Pods.

Similarly if volumes or Networks are in use by other containers, and return
exit code 2.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-09-29 15:52:43 -04:00
b07eb0bc91 healthchecks: return systemd-run error
In case `systemd-run` errors when creating transient unit files (and
timers), create an error based on the combined output from stdout and
stderr.  Using the error from `exec.Command` contains the exit code
only which is not useful to debug (see #7484).

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-09-29 18:26:15 +02:00
63f0bb93f8 Merge pull request #7783 from ashley-cui/slirp
Add support for slirp network for pods
2020-09-29 15:34:37 +00:00
1b5853e647 Properly handle podman run --pull command
Currently the --pull missing|always|never is ignored

This PR implements this for local API.  For remote we
need to default to pullpolicy specified in the containers.conf
file.

Also fixed an issue when images were matching other images names
based on prefix, causing images to always be pulled.

I had named an image myfedora and when ever I pulled fedora, the system
thought that it there were two images named fedora since it was checking
for the name fedora as well as the prefix fedora.  I changed it to check
for fedora and the prefix /fedora, to prefent failures like I had.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-09-27 07:25:28 -04:00
b6176d8987 Add support for slirp network for pods
flag --network=slirp4netns[options] for root and rootless pods

Signed-off-by: Ashley Cui <acui@redhat.com>
2020-09-25 21:34:23 -04:00
00cca405d2 HTTP Attach: Wait until both STDIN and STDOUT finish
In the old code, there was a chance that we could return when
only one of STDIN or STDOUT had finished - this could lead to us
dropping either input to the container, or output from it, in the
case that one stream terminated early.

To resolve this, use separate channels to return STDOUT and STDIN
errors, and track which ones have returned cleanly to ensure that
we need bith in order to return from the HTTP attach function and
pass control back to the HTTP handler (which would assume we
exited cleanly and close the client's attach connection).

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-09-24 14:48:26 -04:00
a6b300ef7e Merge pull request #7754 from mheon/fix_logdriver_order
Fix a bug where log-driver json-file was made no logs
2020-09-23 20:12:29 +00:00
bbbc0655b9 Fix a bug where log-driver json-file was made no logs
When we added the None log driver, it was accidentally added in
the middle of a set of Fallthrough stanzas which all should have
led to k8s-file, so that JSON file logging accidentally caused
no logging to be selected instead of k8s-file.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-09-23 13:28:55 -04:00
99386c7ae7 Include cgroup manager in podman info output
This is very useful for debugging cgroups v2, especially on
rootless - we need to ensure people are correctly using systemd
cgroups in these cases.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-09-22 13:15:01 -04:00
dc5cc43563 Merge pull request #7698 from AkihiroSuda/rootless-cni-deflake-lo-up
rootless-cni-infra: fix flakiness during bringing up lo interface
2020-09-21 14:38:07 -04:00
0a46b9c9e6 Merge pull request #7696 from jwhonce/wip/version
Refactor version handling in cmd tree
2020-09-21 13:23:20 +00:00
caad102056 libpod: bumps up rootless-cni-infra to 2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-09-21 20:11:11 +09:00
c2068f11cc Merge pull request #7694 from mheon/fix_exec_supplemental_groups
Preserve groups in exec sessions in ctrs with --user
2020-09-19 06:32:59 -04:00
c4b49afad3 Refactor version handling in cmd tree
* Move from simple string to semver objects
* Change client API Version from '1' to 2.0.0

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-09-18 15:13:58 -07:00
2f605dcc1c Preserve groups in exec sessions in ctrs with --user
Podman wants to guarantee that exec sessions retain the groups of
the container they are started in, unless explicitly overridden
by the user. This guarantee was broken for containers where the
`--user` flag was specified; this patch resolves that.

Somewhere in the Exec rewrite for APIv2, I changed the location
where the container's User is passed into the exec session
(similar to groups, we also want to preserve user unless
overridden). The lower-level Exec APIs already handled setting
user and group appropriately if not specified when the exec
session was created, but I added duplicate code to handle this
higher in the stack - and that code only handled setting user,
not supplemental groups, breaking support in that specific case.

Two things conspired to make this one hard to track down: first,
things were only broken if the container explicitly set a user;
otherwise, the container user would still appear to be unset to
the lower-level code, which would properly set supplemental
groups (this tricked our existing test into passing). Also, the
`crun` OCI runtime will add the groups without prompting, which
further masked the problem there. I debated making `runc` do the
same, but in the end it's better to fix this in Podman - it's
better to be explicit about what we want done so we will work
with all OCI runtimes.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-09-18 13:54:25 -04:00
fc131a207e Merge pull request #7671 from zhangguanzhang/play-kube-handle-restartPolicy
handle the restartPolicy for play kube and generate kube
2020-09-18 06:19:19 -04:00
f0ccac199b handle the play kube and generate kube for with restartPolicy
Signed-off-by: zhangguanzhang <zhangguanzhang@qq.com>
2020-09-18 13:28:22 +08:00
5f6fff0899 fix the .Path and .Args when use the infra-command
Signed-off-by: zhangguanzhang <zhangguanzhang@qq.com>
2020-09-18 09:57:54 +08:00
661786808c update github.com/docker/docker and relevant deps
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-09-18 02:22:25 +09:00
ecae5f7ade image list: return all associated names
Always return all associated names / repo tags of an image and fix a bug
with malformed repo tags.

Previously, Podman returned all names only with `--all` but this flag
only instructs to list intermediate images and should not alter
associated names.  With `--all` Podman queried the repo tags of an image
which splits all *tagged* names into repository and tag which is then
reassembled to eventually be parsed again in the frontend.  Lot's of
redundant CPU heat and buggy as the reassembly didn't consider digests
which ultimately broke parsing in the frontend.

Fixes: #7651
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-09-17 12:23:58 +02:00
8d7e795454 Merge pull request #7621 from rhatdan/pods
Fix podman pod create --infra-command and --infra-image
2020-09-16 16:40:44 -04:00