Often users want their overlayed volumes to be `non-volatile` in nature
that means that same `upper` dir can be re-used by one or more
containers but overall of nature of volumes still have to be `overlay`
so work done is still on a overlay not on the actual volume.
Following PR adds support for more advanced options i.e custom `workdir`
and `upperdir` for overlayed volumes. So that users can re-use `workdir`
and `upperdir` across new containers as well.
Usage
```console
$ podman run -it -v myvol:/data:O,upperdir=/path/persistant/upper,workdir=/path/persistant/work alpine sh
```
Signed-off-by: Aditya R <arajan@redhat.com>
While trying to match permissions of target directory podman adds
extra `0111` which should not be needed if target path does not have
execute permission.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Continue eliminating GrepString() and BeTrue(), in tiny
incremental steps. Here I take the liberty of refactoring
some hard-to-read code by adding a helper.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Many ginkgo tests have been written to use this evil form:
GrepString("foo")
Expect(that to BeTrue())
...which yields horrible useless messages on failure:
false is not true
Identify those (automatically, via script) and convert to:
Expect(output to ContainSubstring("foo"))
...which yields:
"this output" does not contain substring "foo"
There are still many BeTrue()s left. This is just a start.
This is commit 1 of 2. It includes the script I used, and
all changes to *.go are those computed by the script.
Commit 2 will apply some manual fixes.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Found by my find-obsolete-skips script. Let's see which, if any,
of these skipped tests can be reenabled.
Some Skips are "this will never work", not "this is expected to
work one day". Update the message on those to reflect that.
Some were real bugs in the test framework. Fix those.
And, joy of joys, some work today. Remove those skips.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The :U flag can be used to change the ownership of source volumes based on
the UID, GID of the container. This is only supported by the --volume option,
this will allow to use --mount option as well.
Signed-off-by: Eduardo Vega <edvegavalerio@gmail.com>
Following PR allows containers to create and mount overlays on top of
named volumes instead of mounting actual volumes via already documented `:O`.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
There are use-cases where users would want to use overlay-mounts as
workdir. For such cases workdir should be resolved after all the mounts
are completed during the container init process.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
e2e test failures are rife with messages like:
Expected 1 to equal 0
These make me cry. They're anti-helpful, requiring the reader
to dive into the source code to figure out what those numbers
mean.
Solution: Go tests have a '.Should(Exit(NNN))' mechanism. I
don't know if it spits out a better diagnostic (I have no way
to run e2e tests on my laptop), but I have to fantasize that
it will, and given the state of our flakes I assume that at
least one test will fail and give me the opportunity to see
what the error message looks like.
THIS IS NOT REVIEWABLE CODE. There is no way for a human
to review it. Don't bother. Maybe look at a few random
ones for sanity. If you want to really review, here is
a reproducer of what I did:
cd test/e2e
! positive assertions. The second is the same as the first,
! with the addition of (unnecessary) parentheses because
! some invocations were written that way. The third is BeZero().
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Equal\((\d+)\)\)/Expect($1).Should(Exit($2))/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(\(Equal\((\d+)\)\)\)/Expect($1).Should(Exit($2))/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(BeZero\(\)\)/Expect($1).Should(Exit(0))/' *_test.go
! Same as above, but handles three non-numeric exit codes
! in run_exit_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Equal\((\S+)\)\)/Expect($1).Should(Exit($2))/' *_test.go
! negative assertions. Difference is the spelling of 'To(Not)',
! 'ToNot', and 'NotTo'. I assume those are all the same.
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Not\(Equal\((0)\)\)\)/Expect($1).To(ExitWithError())/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.ToNot\(Equal\((0)\)\)/Expect($1).To(ExitWithError())/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.NotTo\(Equal\((0)\)\)/Expect($1).To(ExitWithError())/' *_test.go
! negative, old use of BeZero()
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.ToNot\(BeZero\(\)\)/Expect($1).Should(ExitWithError())/' *_test.go
Run those on a clean copy of main branch (at the same branch
point as my PR, of course), then diff against a checked-out
copy of my PR. There should be no differences. Then all you
have to review is that my replacements above are sane.
UPDATE: nope, that's not enough, you also need to add gomega/gexec
to the files that don't have it:
perl -pi -e '$_ .= "$1/gexec\"\n" if m!^(.*/onsi/gomega)"!' $(grep -L gomega/gexec $(git log -1 --stat | awk '$1 ~ /test\/e2e\// { print $1}'))
UPDATE 2: hand-edit run_volume_test.go
UPDATE 3: sigh, add WaitWithDefaultTimeout() to a couple of places
UPDATE 4: skip a test due to bug #10935 (race condition)
Signed-off-by: Ed Santiago <santiago@redhat.com>
Podman uses the volume option map to check if it has to mount the volume
or not when the container is started. Commit 28138dafcc39 added to uid
and gid options to this map, however when only uid/gid is set we cannot
mount this volume because there is no filesystem or device specified.
Make sure we do not try to mount the volume when only the uid/gid option
is set since this is a simple chown operation.
Also when a uid/gid is explicity set, do not chown the volume based on
the container user when the volume is used for the first time.
Fixes#10620
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Currently pull policy is set incorrectly when users set --pull-never.
Also pull-policy is not being translated correctly when using
podman-remote.
Fixes: #9573
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
As part of a fix for an earlier bug (#5698) we added the ability
for Podman to chown volumes to correctly match the user running
in the container, even in adverse circumstances (where we don't
know the right UID/GID until very late in the process). However,
we only did this for volumes created automatically by a
`podman run` or `podman create`. Volumes made by
`podman volume create` do not get this chown, so their
permissions may not be correct. I've looked, and I don't think
there's a good reason not to do this chwon for all volumes the
first time the container is started.
I would prefer to do this as part of volume copy-up, but I don't
think that's really possible (copy-up happens earlier in the
process and we don't have a spec). There is a small chance, as
things stand, that a copy-up happens for one container and then
a chown for a second, unrelated container, but the odds of this
are astronomically small (we'd need a very close race between two
starting containers).
Fixes#9608
Signed-off-by: Matthew Heon <mheon@redhat.com>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Instead of using the container's mountpoint as the base of the
chroot and indexing from there by the volume directory, instead
use the full path of what we want to copy as the base of the
chroot and copy everything in it. This resolves the bug, ends up
being a bit simpler code-wise (no string concatenation, as we
already have the full path calculated for other checks), and
seems more understandable than trying to resolve things on the
destination side of the copy-up.
Fixes#9354
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This one is rather bizarre because it triggers only on some
systems. I've included a CI test, for example, but I'm 99% sure
we use images in CI that have volumes over empty directories, and
the earlier patch to change copy-up implementation passed CI
without complaint.
I can reproduce this on a stock F33 VM, but that's the only place
I have been able to see it.
Regardless, the issue: under certain as-yet-unidentified
environmental conditions, the copier.Get method will return an
ENOENT attempting to stream a directory that is empty. Work
around this by avoiding the copy altogether in this case.
Signed-off-by: Matthew Heon <mheon@redhat.com>
The old copy-up implementation was very unhappy with symlinks,
which could cause containers to fail to start for unclear reasons
when a directory we wanted to copy-up contained one. Rewrite to
use the Buildah Copier, which is more recent and should be both
safer and less likely to blow up over links.
At the same time, fix a deadlock in copy-up for volumes requiring
mounting - the Mountpoint() function tried to take the
already-acquired volume lock.
Fixes#6003
Signed-off-by: Matthew Heon <mheon@redhat.com>
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1915332
```
According to the Docker docs, the consistency option should be ignored on Linux.
the possible values are 'cached', 'delegated', and 'consistent', but they should be ignored equally.
This is a widely used option in scripts run by developer machines, as this makes file I/O less horribly slow on MacOS.
```
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When making containers, we want to lock all named volumes we are
adding the container to, to ensure they aren't removed from under
us while we are working. Unfortunately, this code did not account
for a container having the same volume mounted in multiple places
so it could deadlock. Add a map to ensure that we don't lock the
same name more than once to resolve this.
Fixes#8221
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Detached containers and detach keys are only created with the podman run, i
exec, and start commands. We do not store the detach key sequence or the
detach flags in the database, nor does Docker. The current code was ignoreing
these fields but documenting that they can be used.
Fix podman create man page and --help output to no longer indicate that
--detach and --detach-keys works.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Now that Dan has added helpful comments to each SkipIfRemote,
let's take the next step and include those messages in the
Skip() output so someone viewing test results can easily
see if a remote test is skipped for a real reason or for
a FIXME.
This commit is the result of a simple:
perl -pi -e 's;(SkipIfRemote)\(\)(\s+//\s+(.*))?;$1("$3");' *.go
in the test/e2e directory, with a few minor (manual) changes
in wording.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Remove ones that are not needed.
Document those that should be there.
Document those that should be fixed.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules. While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.
Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`. The renaming of the imports
was done via `gomove` [1].
[1] https://github.com/KSubedi/gomove
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This is just an alias to the `ro` option, but it's already in the
manpages (and Docker) so we might as well add support for it.
Fixes#6379
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
We previously enforced this for security reasons, but as Dan has
explained on several occasions, it's not very valuable there
(it's trivially easy to bypass) and it does seriously annoy folks
trying to use named volumes. Flip the default from 'on' to 'off'.
Signed-off-by: Matthew Heon <mheon@redhat.com>
enabled integration tests for volumes. there are two exceptions that still need work because of something not yet implemented.
also, add code to deal with the fact that containers conf appears to set a local volume driver where it used to be simply blank.
Signed-off-by: Brent Baude <bbaude@redhat.com>
This should complete Podmanv2's support for volume-related flags.
Most code was sourced from the old pkg/spec implementation with
modifications to account for the split between frontend flags
(volume, mount, tmpfs) and the backend flags implemented here.
Also enables tests for podman run with volumes
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
For volume and bind mount tests, use the in-container mount point path
that has no common ancestor with any host path (except for root).
This might help to uncover bugs like [1]. Even if not, it seems
lile a good cleanup regardless.
[1] https://github.com/containers/libpod/pull/5676
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Move declaration of a dockerfile closer to its use.
Since it is used only once, there's no sense in having it declared
globally.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Remove repeated mountPath directory creation.
* For the first two hunks it is the same dir ("secrets") that was
already created before.
* For the last hunk ("scratchpad") it is not used at all.
Add an empty line after Mkdir for cases where dir is used more than once.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
We need to consistently use --time rather then --timeout throughout the code.
Fix locations where timeout defaults are not set correctly as well.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We use filepath.Clean() to remove trailing slashes to ensure that
when we supercede image mounts with mounts from --volume and
--mount, paths are consistent when we compare. Unfortunately,
while we used the cleaned path for the destination in the mount,
it was accidentally not used to index the maps that we use to
identify what to supercede, so our comparisons might be thrown
off by trailing slashes and similar.
Fixes#5219
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
When Docker performs a copy up, it first verifies that the volume
being copied into is empty; thus, for volumes that have been
modified elsewhere (e.g. manually copying into then), the copy up
will not be performed at all. Duplicate this behavior in Podman
by checking if the volume is empty before copying.
Furthermore, move setting copyup to false further up. This will
prevent a potential race where copy up could happen more than
once if Podman was killed after some files had been copied but
before the DB was updated.
This resolves CVE-2020-1726.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
command.Start() just starts the command. That catches some
errors, but the nasty ones - bad options and similar - happen
when the command runs. Use CombinedOutput() instead - it waits
for the command to exit, and thus catches non-0 exit of the
`mount` command (invalid options, for example).
STDERR from the `mount` command is directly used, which isn't
necessarily the best, but we can't really get much more info on
what went wrong.
Fixes#4303
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
