This also includes a number of significant changes to the SQLite
state made possible by removal of the legacy DB.
1. Enable database unit tests for SQLite state, with numerous
tweaks to get tests passing. Most notable changes are to
container removal - where we previously didn't return an error
if there was no container to remove - and RemovePodContainers,
which I don't think ever worked properly from my reading of
the failures.
2. Removal of AddContainerToPod/RemoveContainerToPod. On SQLite,
these functions are identical to AddContainer/RemoveContainer
and there is no reason to retain duplicates.
3. Removal of SafeRewriteContainerConfig - it's identical to
RewriteContainerConfig in SQLite, no reason to have duplicate
entrypoints.
As an exciting side-note, this removes Podman's requirement that
containers and pods cannot share a name, which was a BoltDB
restriction only.
Signed-off-by: Matt Heon <matthew.heon@pm.me>
Tremendous amount of changes in here, but all should amount to
the same thing: changing Go import paths from v5 to v6.
Also bumped go.mod to github.com/containers/podman/v6 and updated
version to v6.0.0-dev.
Signed-off-by: Matt Heon <mheon@redhat.com>
Problem: While removing cgroupsv1 code, I noticed my neovim Go config
automatically changed fileperms to the new octal format and I didn't
want that polluting my diffs.
Decision: I thought it best to switch to the new octal format in a dedicated PR.
Action:
- Cursor switched to new octal format for all fileperm ocurrences in Go
source and test files.
- vendor/, docs/ and non-Go files were ignored.
- Reviewed manually.
Ref: https://go.dev/ref/spec#Go_1.13
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
podman system check --quick currently only skips layer contents, but
practically it's not much quicker than without the flag.
This changes the flag to also skip checking layer digests which speed up
the check significantly.
In some cases, it is useful to opt for a quicker check if we prioritize
detecting and fixing severe corruption and can tolerate minor damage.
The check option is derived from CRI-O's internal repair:
9e4d86d823/internal/lib/container_server.go (L860)
Signed-off-by: Sonny Sasaka <sonnysasaka@gmail.com>
Instead of duplicating the NewArtifactStore() call in many places and
having to make sure we always pass the same path to it define it as
function on the runtime. This allows any caller with access to the
libpod runtime to create the store easily.
This is suing a sync.OnceValues() function so the store is initialized
only once and only when actually needed.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The `podman system prune` command is able to remove build containers that were created during the build, but were not removed because the build terminated unexpectedly.
By default, build containers are not removed to prevent interference with builds in progress. Use the **--build** flag when running the command to remove build containers as well.
Fixes: https://issues.redhat.com/browse/RHEL-62009
Signed-off-by: Jan Rodák <hony.com@seznam.cz>
Fix new issues found by usetesting, mainly we should use t.TempDir() in
test which makes the code better as this will be removed on test end
automatically so no need for defer or any error checking.
Also fix issues reported by exptostd, these mainly show where we can
switch the imports to the std maps/slices packages instead of the
golang.org/x/exp/... packages.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In the common scenario of podman-remote run --rm the API is required to
attach + start + wait to get exit code. This has the problem that the
wait call races against the container removal from the cleanup process
so it may not get the exit code back. However we keep the exit code
around for longer than the container so we can just look it up in the
endpoint. Of course this only works when we get a full id as param but
podman-remote will do that.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Currently podman run -d can exit 0 if we send SIGTERM during startup
even though the contianer was never started. That just doesn't make any
sense is horribly confusing for a external job manager like systemd.
The original motivation was to exit 0 for the podman.service in commit
ca7376bb11. That does make sense but it should only do so for the
service and only if the server did indeed gracefully shutdown.
So we rework how the exit logic works, do not let the handler perform
the exit. Instead the shutdown package does the exit after all handlers
are run, this solves the issue of ordering. Then we default to exit code
1 like we did before and allow the service exit handler to overwrite the
exit code 0 in case of a graceful shutdown.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Strictly speaking we don't need the path yet, but it existing
prevents a lot of strangeness in our path-checking logic to
validate the current Podman configuration, as it was the only
path that might not exist this early in init.
Fixes#23515
Signed-off-by: Matt Heon <mheon@redhat.com>
shutdown the containers store so that the home directory mount is not
leaked when "podman system service" exits.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
It seems that if some background tasks are queued in libpod's Runtime before the worker's channel is set up (eg. in the refresh phase), they are not executed later on, but the workerGroup's counter is still ticked up. This leads podman to hang when the imageEngine is shutdown, since it waits for the workerGroup to be done.
fixescontainers/podman#22984
Signed-off-by: Farya Maerten <me@ltow.me>
Add a `podman system check` that performs consistency checks on local
storage, optionally removing damaged items so that they can be
recreated.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
In cases where we fail to configure the error is returned as it and may
be missing useful context. Make sure we know the error happened as part
of the storage setup.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Podman needs to be able to detect when a system reboot occurs to
do certain types of cleanup operation (for example, reset
container states, clean up IPAM allocations, etc). our current
method for this is a sentinel file on a tmpfs filesystem. The
problem emerges that there is no directory that is guaranteed to
be a tmpfs and is also guaranteed to be accessible to rootless
users in the FHS. If the user has a systemd user session, we can
depend on /run/user/$UID, but we can't reliably say that they do.
This code will detect the no-tmpfs-but-reboot-occurred case by
writing the current system boot ID to our tmpfs sentinel file
when it is created, and checking that file every time Podman
starts to make sure that the current boot ID matches the cached
one in the sentinel file. If they don't match, a reboot occurred
and the sentinel file was not on a tmpfs and thus survived. In
that case, throw an error telling the user to remove certain
directories (the ones that are supposed to be tmpfs), so we can
proceed as expected.
Signed-off-by: Matt Heon <mheon@redhat.com>
This factors out the check for cgroupsv2 unified mode into a
platform-specific file and stops podman from generating a (harmless)
warning every time it is run on FreeBSD.
[NO NEW TESTS NEEDED]
Signed-off-by: Doug Rabson <dfr@rabson.org>
Commit 03f6589f3 added basic support for pull-error event from libimage
but it contains several problems:
1. storing the error as error type prevents it from being unmarshalled,
thus change it to a string
2. the error was never propagated from the libimage event to the podman
event struct
3. the error message was not wired into the cli and API
This commit fixes these problems.
Fixes#21458
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Moving from Go module v4 to v5 prepares us for public releases.
Move done using gomove [1] as with the v3 and v4 moves.
[1] https://github.com/KSubedi/gomove
Signed-off-by: Matt Heon <mheon@redhat.com>
We were preserving ContainerStateExited, which is better than
nothing, but definitely not correct. A container that ran at any
point during the last boot should be moved to Exited state to
preserve the fact that they were run at least one. This means we
have to convert Running, Stopped, Stopping, Paused containers to
exited as well.
Signed-off-by: Matt Heon <mheon@redhat.com>
During system shutdown, Podman should go down gracefully, meaning
that we have time to spawn cleanup processes which remove any
containers set to autoremove. Unfortunately, this isn't always
the case. If we get a SIGKILL because the system is going down
immediately, we can't recover from this, and the autoremove
containers are not removed.
However, we can pick up any leftover autoremove containers when
we refesh the DB state, which is the first thing Podman does
after a reboot. By detecting any autoremove containers that have
actually run (a container that was created but never run doesn't
need to be removed) at that point and removing them, we keep the
fresh boot clean, even if Podman was terminated abnormally.
Fixes#21482
[NO NEW TESTS NEEDED] This requires a reboot to realistically
test.
Signed-off-by: Matt Heon <mheon@redhat.com>
Podman v5 will not support cgroups-v1. This commit will print a warning
if it detects a cgroups-v1 system. The warning can be hidden by setting
envvar `PODMAN_CGROUPSV1_WARNING`.
This warning is patched out for RHEL 9 builds as cgroups-v1 will still
be supported on RHEL 9 systems.
Resolves: https://issues.redhat.com/browse/RUN-1957
[NO NEW TESTS NEEDED]
Co-authored-by: Ed Santiago <santiago@redhat.com>
Co-authored-by: Sascha Grunert <sgrunert@redhat.com>
Co-authored-by: Giuseppe Scrivano <gscrivan@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
Before this, for some special Podman commands (system reset,
system migrate, system renumber), Podman would create a first
Libpod runtime to do initialization and flag parsing, then stop
that runtime and create an entirely new runtime to perform the
actual task. This is an artifact of the pre-Podman 2.0 days, when
there was almost no indirection between Libpod and the CLI, and
we only used one runtime because we didn't need a second runtime
for flag parsing and basic init.
This system was clunky, and apparently, very buggy. When we
migrated to SQLite, some logic was introduced where we'd select a
different database location based on whether or not Libpod's
StaticDir was manually set - which differed between the first
invocation of Libpod and the second. So we'd get a different
database for some commands (like `system reset`) and they would
not be able to see existing containers, meaning they would not
function properly.
The immediate cause is obviously the SQLite behavior, but I'm
certain there's a lot more baggage hiding behind this multiple
Libpod runtime logic, so let's just refactor it out. It doesn't
make sense, and complicates the code. Instead, make Reset,
Renumber, and Migrate methods of the libpod Runtime. For Reset
and Renumber, we can shut the runtime down afterwards to achieve
the desired effect (no valid runtime after). Then pipe all of
them through the ContainerEngine so cmd/podman can access them.
As part of this, remove the SystemEngine part of pkg/domain. This
was supposed to encompass these "special" commands, but every
command in SystemEngine is actually a ContainerEngine command.
Reset, Renumber, Migrate - they all need a full Libpod and access
to all containers. There's no point to a separate engine if it
just wraps Libpod in the exact same way as ContainerEngine. This
consolidation saves us a bit more code and complexity.
Signed-off-by: Matt Heon <mheon@redhat.com>
these functions are not used anymore in the codebase, so drop them.
[NO NEW TESTS NEEDED] no new functionalities are added
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Use sqlite as default but for upgrades it will still use boltdb to avoid
breaking anyone. This is done by checking if the boltdb file already
exists and if it does then we have to use it.
I added a e2e test to check the new logic and removed the system test
for it, the problem with the system test is that we share the storage
dir there so all following commands without --db-backend would try to
use boltdb as a single --db-backend boltdb command will create the file
and then all folllwing commands will use it because of the backwards
compat. In e2e tests each test uses their own --root so it is not an
issue there.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The processing and setting of the static and volume directories was
scattered across the code base (including c/common) leading to subtle
errors that surfaced in #19938.
There were multiple issues that I try to summarize below:
- c/common loaded the graphroot from c/storage to set the defaults for
static and volume dir. That ignored Podman's --root flag and
surfaced in #19938 and other bugs. c/common does not set the
defaults anymore which gives Podman the ability to detect when the
user/admin configured a custom directory (not empty value).
- When parsing the CLI, Podman (ab)uses containers.conf structures to
set the defaults but also to override them in case the user specified
a flag. The --root flag overrode the static dir which is wrong and
broke a couple of use cases. Now there is a dedicated field for in
the "PodmanConfig" which also includes a containers.conf struct.
- The defaults for static and volume dir and now being set correctly
and adhere to --root.
- The CONTAINERS_CONF_OVERRIDE env variable has not been passed to the
cleanup process. I believe that _all_ env variables should be passed
to conmon to avoid such subtle bugs.
Overall I find that the code and logic is scattered and hard to
understand and follow. I refrained from larger refactorings as I really
just want to get #19938 fixed and then go back to other priorities.
https://github.com/containers/common/pull/1659 broke three pkg/machine
tests. Those have been commented out until getting fixed.
Fixes: #19938
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
To debug a deadlock, we really want to know what lock is actually
locked, so we can figure out what is using that lock. This PR
adds support for this, using trylock to check if every lock on
the system is free or in use. Will really need to be run a few
times in quick succession to verify that it's not a transient
lock and it's actually stuck, but that's not really a big deal.
Signed-off-by: Matt Heon <mheon@redhat.com>
This is a general debug command that identifies any lock
conflicts that could lead to a deadlock. It's only intended for
Libpod developers (while it does tell you if you need to run
`podman system renumber`, you should never have to do that
anyways, and the next commit will include a lot more technical
info in the output that no one except a Libpod dev will want).
Hence, hidden command, and only implemented for the local driver
(recommend just running it by SSHing into a `podman machine` VM
in the unlikely case it's needed by remote Podman).
These conflicts should normally never happen, but having a
command like this is useful for debugging deadlock conditions
when they do occur.
Signed-off-by: Matt Heon <mheon@redhat.com>
Ref: https://pkg.go.dev/math/rand@go1.20#Seed
Note: For `runtime_test.go`, this test-case was never actually doing
what appears as it's intent . Fixing it to work as intended would be
require incredibly libpod-invasive changes. Do the least-worse thing and
simply confirm that consecutive generated names are different.
Signed-off-by: Chris Evich <cevich@redhat.com>
Currently --tmpdir changes the location of the pause.pid file. this
causes issues because the c code in pkg/rootless does not know about
that. I tried to fix this[1] by fixing the c code to not use the
shortcut. While this fix worked it will result in many pause processes
leaking in the integrration tests.
Commit ab88632 added this behavior but following the disccusion it was
never the intention that we end up having more than one pause process.
The issues that was trying to fix was caused by somthing else AFAICT,
the main problem seems to be that the pause.pid file parent directory
may not be created when we try to create the pid file so it failed with
ENOENT. This patch fixes it by creating this directory always and revert
the change to no longer depend on the tmpdir value.
With this commit we now always use XDG_RUNTIME_DIR/libpod/tmp/pause.pid
for all podman processes. This allows the c shortcut to work reliably
and should therefore improve perfomance over my other approach.
A system test is added to ensure we see the right behavior and that
podman system migrate actually stops the pause process. Thanks to Ed
Santiago for the improved test to make it work for both `catatonit` and
`podman pause`.
This should fix the issues with namespace missmatches that we can see in
CI as flakes.
[1] https://github.com/containers/podman/pull/18057Fixes#18057
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This contains the implementation of (most) container functions,
with stubs for all pod and volume functions. Presently accessed
via environment variable only for testing purposes.
Signed-off-by: Matt Heon <mheon@redhat.com>
Loading container states speed things up when listing all containers but
it comes with a price tag for many other call paths. Hence, make
loading the state conditional to allow for keeping `podman ps` fast
without other commands regressing in performance.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
always create a user namespace when running with euid != 0 since the
user is not owning the current mount namespace.
This issue happened on a Kubernetes cluster, where the pod was running
privileged but the UID was not 0, as it was configured in the image
itself.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
