add a new flag that allows to override the pull options configured in
the storage.conf file.
e.g.: --pull-option="enable_partial_images=false" can be specified to
Podman to disable partial pulls even if enabled.
Leave it as a hidden configuration flag for now since the API itself
is marked as experimental in c/storage.
Currently c/storage doesn't honor the overrides, being fixed with
https://github.com/containers/storage/pull/1966
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
This was added ages ago in commit c65b3599cc, however in the meantime
both podman and conmon can support longer socket paths as they use a
workaround to open the path via /proc/self/fd, see openUnixSocket() in
libpod/oci_conmon_attach_linux.go
Thus this restriction is not needed anymore and we can drop a workaround
in the tests.
Fixes#22272
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Moving from Go module v4 to v5 prepares us for public releases.
Move done using gomove [1] as with the v3 and v4 moves.
[1] https://github.com/KSubedi/gomove
Signed-off-by: Matt Heon <mheon@redhat.com>
Before this, for some special Podman commands (system reset,
system migrate, system renumber), Podman would create a first
Libpod runtime to do initialization and flag parsing, then stop
that runtime and create an entirely new runtime to perform the
actual task. This is an artifact of the pre-Podman 2.0 days, when
there was almost no indirection between Libpod and the CLI, and
we only used one runtime because we didn't need a second runtime
for flag parsing and basic init.
This system was clunky, and apparently, very buggy. When we
migrated to SQLite, some logic was introduced where we'd select a
different database location based on whether or not Libpod's
StaticDir was manually set - which differed between the first
invocation of Libpod and the second. So we'd get a different
database for some commands (like `system reset`) and they would
not be able to see existing containers, meaning they would not
function properly.
The immediate cause is obviously the SQLite behavior, but I'm
certain there's a lot more baggage hiding behind this multiple
Libpod runtime logic, so let's just refactor it out. It doesn't
make sense, and complicates the code. Instead, make Reset,
Renumber, and Migrate methods of the libpod Runtime. For Reset
and Renumber, we can shut the runtime down afterwards to achieve
the desired effect (no valid runtime after). Then pipe all of
them through the ContainerEngine so cmd/podman can access them.
As part of this, remove the SystemEngine part of pkg/domain. This
was supposed to encompass these "special" commands, but every
command in SystemEngine is actually a ContainerEngine command.
Reset, Renumber, Migrate - they all need a full Libpod and access
to all containers. There's no point to a separate engine if it
just wraps Libpod in the exact same way as ContainerEngine. This
consolidation saves us a bit more code and complexity.
Signed-off-by: Matt Heon <mheon@redhat.com>
these functions are not used anymore in the codebase, so drop them.
[NO NEW TESTS NEEDED] no new functionalities are added
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
All `[]string`s in containers.conf have now been migrated to attributed
string slices which require some adjustments in Buildah and Podman.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The processing and setting of the static and volume directories was
scattered across the code base (including c/common) leading to subtle
errors that surfaced in #19938.
There were multiple issues that I try to summarize below:
- c/common loaded the graphroot from c/storage to set the defaults for
static and volume dir. That ignored Podman's --root flag and
surfaced in #19938 and other bugs. c/common does not set the
defaults anymore which gives Podman the ability to detect when the
user/admin configured a custom directory (not empty value).
- When parsing the CLI, Podman (ab)uses containers.conf structures to
set the defaults but also to override them in case the user specified
a flag. The --root flag overrode the static dir which is wrong and
broke a couple of use cases. Now there is a dedicated field for in
the "PodmanConfig" which also includes a containers.conf struct.
- The defaults for static and volume dir and now being set correctly
and adhere to --root.
- The CONTAINERS_CONF_OVERRIDE env variable has not been passed to the
cleanup process. I believe that _all_ env variables should be passed
to conmon to avoid such subtle bugs.
Overall I find that the code and logic is scattered and hard to
understand and follow. I refrained from larger refactorings as I really
just want to get #19938 fixed and then go back to other priorities.
https://github.com/containers/common/pull/1659 broke three pkg/machine
tests. Those have been commented out until getting fixed.
Fixes: #19938
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The --syslog flag has not been passed to the cleanup process (i.e.,
conmon's exit args) complicating debugging quite a bit.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Motivation
===========
This feature aims to make --uidmap and --gidmap easier to use, especially in rootless podman setups.
(I will focus here on the --gidmap option, although the same applies for --uidmap.)
In rootless podman, the user namespace mapping happens in two steps, through an intermediate mapping.
See https://docs.podman.io/en/latest/markdown/podman-run.1.html#uidmap-container-uid-from-uid-amount
for further detail, here is a summary:
First the user GID is mapped to 0 (root), and all subordinate GIDs (defined at /etc/subgid, and
usually >100000) are mapped starting at 1.
One way to customize the mapping is through the `--gidmap` option, that maps that intermediate mapping
to the final mapping that will be seen by the container.
As an example, let's say we have as main GID the group 1000, and we also belong to the additional GID 2000,
that we want to make accessible inside the container.
We first ask the sysadmin to subordinate the group to us, by adding "$user:2000:1" to /etc/subgid.
Then we need to use --gidmap to specify that we want to map GID 2000 into some GID inside the container.
And here is the first trouble:
Since the --gidmap option operates on the intermediate mapping, we first need to figure out where has
podman placed our GID 2000 in that intermediate mapping using:
podman unshare cat /proc/self/gid_map
Then, we may see that GID 2000 was mapped to intermediate GID 5. So our --gidmap option should include:
--gidmap 20000:5:1
This intermediate mapping may change in the future if further groups are subordinated to us (or we stop
having its subordination), so we are forced to verify the mapping with
`podman unshare cat /proc/self/gid_map` every time, and parse it if we want to script it.
**The first usability improvement** we agreed on #18333 is to be able to use:
--gidmap 20000:@2000:1
so podman does this lookup in the parent user namespace for us.
But this is only part of the problem. We must specify a **full** gidmap and not only what we want:
--gidmap 0:0:5 --gidmap 5:6:15000 --gidmap 20000:5:1
This is becoming complicated. We had to break the gidmap at 5, because the intermediate 5 had to
be mapped to another value (20000), and then we had to keep mapping all other subordinate ids... up to
close to the maximum number of subordinate ids that we have (or some reasonable value). This is hard
to explain to someone who does not understand how the mappings work internally.
To simplify this, **the second usability improvement** is to be able to use:
--gidmap "+20000:@2000:1"
where the plus flag (`+`) states that the given mapping should extend any previous/default mapping,
overriding any previous conflicting assignment.
Podman will set that mapping and fill the rest of mapped gids with all other subordinated gids, leading
to the same (or an equivalent) full gidmap that we were specifying before.
One final usability improvement related to this is the following:
By default, when podman gets a --gidmap argument but not a --uidmap argument, it copies the mapping.
This is convenient in many scenarios, since usually subordinated uids and gids are assigned in chunks
simultaneously, and the subordinated IDs in /etc/subuid and /etc/subgid for a given user match.
For scenarios with additional subordinated GIDs, this map copying is annoying, since it forces the user
to provide a --uidmap, to prevent the copy from being made. This means, that when the user wants:
--gidmap 0:0:5 --gidmap 5:6:15000 --gidmap 20000:5:1
The user has to include a uidmap as well:
--gidmap 0:0:5 --gidmap 5:6:15000 --gidmap 20000:5:1 --uidmap 0:0:65000
making everything even harder to understand without proper context.
For this reason, besides the "+" flag, we introduce the "u" and "g" flags. Those flags applied to a
mapping tell podman that the mapping should only apply to users or groups, and ignored otherwise.
Therefore we can use:
--gidmap "+g20000:@2000:1"
So the mapping only applies to groups and is ignored for uidmaps. If no "u" nor "g" flag is assigned
podman assumes the mapping applies to both users and groups as before, so we preserve backwards compatibility.
Co-authored-by: Tom Sweeney <tsweeney@redhat.com>
Signed-off-by: Sergio Oller <sergioller@gmail.com>
Add support for `--imagestore` in podman which allows users to split the filesystem of containers vs image store, imagestore if configured will pull images in image storage instead of the graphRoot while keeping the other parts still in the originally configured graphRoot.
This is an implementation of
https://github.com/containers/storage/pull/1549 in podman.
Signed-off-by: Aditya R <arajan@redhat.com>
Add a hidden flag to set the database backend and plumb it into
podman-info. Further add a system test to make sure the flag and the
info output are working properly.
Note that the test may need to be changed once we settled on how
to test the sqlite backend in CI.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
We should have done this much earlier, most of the times CNI networks
just mean networks so I changed this and also fixed some function
names. This should make it more clear what actually refers to CNI and
what is just general network backend stuff.
[NO NEW TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This handles the transient store options from the container/storage
configuration in the runtime/engine.
Changes are:
* Print transient store status in `podman info`
* Print transient store status in runtime debug output
* Add --transient-store argument to override config option
* Propagate config state to conmon cleanup args so the callback podman
gets the same config.
Note: This doesn't really change any behaviour yet (other than the changes
in containers/storage).
Signed-off-by: Alexander Larsson <alexl@redhat.com>
Use `Default()` instead of re-loading containers.conf.
Also rework how the containers.conf objects are handled for parsing the
CLI. Previously, we were conflating "loading the defaults" with
"storing values from the CLI" with "libpod may further change fields"
which ultimately led to various bugs and test failues.
To address the issue, separate the defaults from the values from the CLI
and properly name the fields to make the semantics less ambiguous.
[NO NEW TESTS NEEDED] as it's not a functional change.
Fixes: containers/common/issues/1200
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
[NO NEW TESTS NEEDED]
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
* Replace "setup", "lookup", "cleanup", "backup" with
"set up", "look up", "clean up", "back up"
when used as verbs. Replace also variations of those.
* Improve language in a few places.
Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
Firstly, reset is now managed by the runtime itself as a part of
initialization. This ensures that it can be used even with
runtimes that would otherwise fail to be created - most notably,
when the user has changed a core path
(runroot/root/tmpdir/staticdir).
Secondly, we now attempt a best-effort removal even if the store
completely fails to be configured.
Third, we now hold the alive lock for the entire reset operation.
This ensures that no other Podman process can start while we are
running a system reset, and removes any possibility of a race
where a user tries to create containers or pull images while we
are trying to perform a reset.
[NO NEW TESTS NEEDED] we do not test reset last I checked.
Fixes#9075
Signed-off-by: Matthew Heon <mheon@redhat.com>
Libpod or packages under /pkg should never import from /cmd/...
This will quickly result in import cycles and weird code paths.
Also there is no reason to use this special code we can just use
syscall.SIGHUB as SIGNAL.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
From a security point of view, it would be nice to be able to map a
rootless usernamespace that does not use your own UID within the
container.
This would add protection against a hostile process escapping the
container and reading content in your homedir.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Since this option will also be used for netavark we should rename it to
something more generic. It is important that --cni-config-dir still
works otherwise we could break existing container cleanup commands.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Create a custom writer which logs the netavark output to logrus. This
will log to the syslog when it is enabled.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
To make testing easier we can overwrite the network backend with the
global `--network-backend` option.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Fixes: https://github.com/containers/podman/issues/10393
Currently if a user specifies a --root flag to override the location of
the container storage, we still enforce the storage-opts from
storage.conf. This causes issues with people trying to intereact with
the additional stores feature, and then forces them to use the obscure
--storage-opt="" option. I belive this should be the default and we
already do this when the user specifies the --storage-driver option.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We define in the man page that this overrides the default storage
options, but the code was appending to the existing options.
This PR also makes a change to allow users to specify --storage-opt="".
This will turn off all storage options.
https://github.com/containers/podman/issues/9852
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
this enables the ability to connect and disconnect a container from a
given network. it is only for the compatibility layer. some code had to
be refactored to avoid circular imports.
additionally, tests are being deferred temporarily due to some
incompatibility/bug in either docker-py or our stack.
Signed-off-by: baude <bbaude@redhat.com>
Currently infr-command and --infra-image commands are ignored
from the user. This PR instruments them and adds tests for
each combination.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules. While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.
Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`. The renaming of the imports
was done via `gomove` [1].
[1] https://github.com/KSubedi/gomove
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We need to default to building podman. If you specify no build
tags you will not build podman, not podman-remote.
Just using remote flag to indicate podman-remote and !remote for
podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Support the `X-Registry-Auth` http-request header.
* The content of the header is a base64 encoded JSON payload which can
either be a single auth config or a map of auth configs (user+pw or
token) with the corresponding registries being the keys. Vanilla
Docker, projectatomic Docker and the bindings are transparantly
supported.
* Add a hidden `--registries-conf` flag. Buildah exposes the same
flag, mostly for testing purposes.
* Do all credential parsing in the client (i.e., `cmd/podman`) pass
the username and password in the backend instead of unparsed
credentials.
* Add a `pkg/auth` which handles most of the heavy lifting.
* Go through the authentication-handling code of most commands, bindings
and endpoints. Migrate them to the new code and fix issues as seen.
A final evaluation and more tests is still required *after* this
change.
* The manifest-push endpoint is missing certain parameters and should
use the ABI function instead. Adding auth-support isn't really
possible without these parts working.
* The container commands and endpoints (i.e., create and run) have not
been changed yet. The APIs don't yet account for the authfile.
* Add authentication tests to `pkg/bindings`.
Fixes: #6384
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
