Address a number of issues in the streaming logic in remote build, most
importantly an error in using buffered channels on the server side.
The pattern below does not guarantee that the channel is entirely read
before the context fires.
for {
select {
case <- bufferedChannel:
...
case <- ctx.Done():
...
}
}
Fixes: #10154
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The multi-image load test was failing on non-x86_64 arch,
because the images used by the test (:00000000, :20200902)
did not have manifests for the given arch.
Solution: all we need are two nonlocal images. Use the
predefined NONLOCAL_IMAGE for one, and a new :multiimage
tag (manually created, currently == :20210610) for the other.
Document, so if/when RHEL adds new supported arches, the
test will fail but a maintainer will have a clue what to do.
Also, as long as I'm in here: add 'image prune -f' to teardown
in build.bats, to avoid seeing lots of red "stray image"
warnings in test logs.
Also: skip a broken/flaky python test
Signed-off-by: Ed Santiago <santiago@redhat.com>
$ mkdir zzz;cd zzz
$ printf "FROM quay.io/libpod/testimage:20210610\nCOPY ./ ./\nCOPY subdir ./\n" >Dockerfile
$ printf "*\nsubdir\n\!*/sub1*\n" >.dockerignore
$ mkdir subdir; touch subdir/sub1.txt
$ ../bin/podman-remote build .
Should fail, but succeeds because we are not sending over the
.dockerignore file to the server side. This PR will send the
.dockerignore so the server side and use it.
Fixes: #10907
Also if both .containerignore and .dockerignore in the context
directory, podman-remote should prefer .containerignore and not use
.dockerignore.
Fixes: #10908
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
If the user specifies "*" in a .dockerignore or a .containerignore
then podman-remote build should not exclude the Dockerfile or
Containerfile or any content pointed to by `-f` in the context
directory.
We still need these files on the server side to complete the build.
Fixes: https://github.com/containers/podman/issues/9867
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
podman-remote build has to handle multiple different locations
for the Containerfile. Currently this works in local mode but not
when using podman-remote.
Fixes: https://github.com/containers/podman/issues/9871
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The initial version of libimage changed the order of layers which has
now been restored to remain backwards compatible.
Further changes:
* Fix a bug in the journald logging which requires to strip trailing
new lines from the message. The system tests did not pass due to
empty new lines. Triggered by changing the default logger to
journald in containers/common.
* Fix another bug in the journald logging which embedded the container
ID inside the message rather than the specifid field. That surfaced
in a preceeding whitespace of each log line which broke the system
tests.
* Alter the system tests to make sure that the k8s-file and the
journald logging drivers are executed.
* A number of e2e tests have been changed to force the k8s-file driver
to make them pass when running inside a root container.
* Increase the timeout in a kill test which seems to take longer now.
Reasons are unknown. Tests passed earlier and no signal-related
changes happend. It may be CI VM flake since some system tests but
other flaked.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Migrate the Podman code base over to `common/libimage` which replaces
`libpod/image` and a lot of glue code entirely.
Note that I tried to leave bread crumbs for changed tests.
Miscellaneous changes:
* Some errors yield different messages which required to alter some
tests.
* I fixed some pre-existing issues in the code. Others were marked as
`//TODO`s to prevent the PR from exploding.
* The `NamesHistory` of an image is returned as is from the storage.
Previously, we did some filtering which I think is undesirable.
Instead we should return the data as stored in the storage.
* Touched handlers use the ABI interfaces where possible.
* Local image resolution: previously Podman would match "foo" on
"myfoo". This behaviour has been changed and Podman will now
only match on repository boundaries such that "foo" would match
"my/foo" but not "myfoo". I consider the old behaviour to be a
bug, at the very least an exotic corner case.
* Futhermore, "foo:none" does *not* resolve to a local image "foo"
without tag anymore. It's a hill I am (almost) willing to die on.
* `image prune` prints the IDs of pruned images. Previously, in some
cases, the names were printed instead. The API clearly states ID,
so we should stick to it.
* Compat endpoint image removal with _force_ deletes the entire not
only the specified tag.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This test continues to flake on podman-remote (especially Ubuntu)
even after #10030 and #10034. I give up. Stop checking the error
message in podman-remote tests.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Currently pull policy is set incorrectly when users set --pull-never.
Also pull-policy is not being translated correctly when using
podman-remote.
Fixes: #9573
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
- cp test: clean up stray image
- build test: add workaround for #9567 (ultra-slow ubuntu).
We're seeing CI flakes (timeouts) due to ubuntu 2004 being
absurdly slow. Workaround: double our timeout on one specific
test when ubuntu + remote.
- build test: clean up new copy-from test (from #9275).
The test was copy-pasted from buildah system tests, without
really adapting for podman environment (e.g. it was using
images that we don't use here, and would cause pulls, which
will cause flakes). Rewrite test so it references only $IMAGE,
remove some confusing/unnecessary stuff, selectively run
parts of it even when rootless or remote, and add a
test to confirm that copy-from succeeded.
- load test: add error-message test to new load-invalid (#9672).
Basically, make sure the command fails for the right reason.
- play test (kube): use $IMAGE, not alpine; and add pause-image
cleanup to teardown()
- apiv2 mounts test: add a maintainability comment in a tricky
section of code; and tighten up the mount point test.
Signed-off-by: Ed Santiago <santiago@redhat.com>
buildah now supports running under emulation[1] as well as
creating manifests. Use those features to create a multiarch
testimage that can be used to test podman on other arches.
[1] on Fedora 33, this requires the qemu-user-static package
We also build a new :00000001 image, replacing :00000000,
because (sigh) some tests try to run 'true' in the container.
Include instructions on building said image.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Make sure that Podman's default OCI runtime is passed to Buildah in
`podman build`. In theory, Podman and Buildah should use the same
defaults but the projects move at different speeds and it turns out
we caused a regression in v3.0.
Fixes: #9365
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Fixes: https://github.com/containers/podman/issues/9290
Currently we still have hard coded --isolation=chroot for podman-remote build.
Implement missing arguments for podman build
Implements
--jobs, --disable-compression, --excludes
Fixes:
MaxPullPushRetries
RetryDuration
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This reverts commit 15caebfe56.
The previous behaviour of `--pull` to *always* attempt to pull the image
and error out if the pull failed aligns with Docker. Since Podman aims
at feature parity with Docker, the `--pull` behaviour must match.
Fixes: #9134
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The `--pull` flag should be using the "pull if newer" pull policy rather
than "pull always". This aligns with what the help message states, what
Buildah does and, according to #9111, what was done before,
Also add a test to prevent future regressions.
Fixes: #9111
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
A container's workdir can be specified via the CLI via `--workdir` and
via an image config with the CLI having precedence.
Since images have a tendency to specify workdirs without necessarily
shipping the paths with the root FS, make sure that Podman creates the
workdir. When specified via the CLI, do not create the path, but check
for its existence and return a human-friendly error.
NOTE: `crun` is performing a similar check that would yield exit code
127. With this change, however, Podman performs the check and yields
exit code 126. Since this is specific to `crun`, I do not consider it
to be a breaking change of Podman.
Fixes: #9040
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The --default-mounts-file path was not being handled in
podman build. This will enable it to use for testing.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
- stop: test --all and --ignore (#9051)
- build: test /run/secrets (#8679, but see below)
- sensitive mount points: deal with 'stat' failures
- selinux: confirm useful diagnostics on unknown labels (#8946)
The 'build' test is intended as a fix for #8679, in which
'podman build' does not mount secrets from mounts.conf.
Unfortunately, as of this writing, 'podman build' does
not pass the --default-mounts-file option to buildah,
so there's no reasonable way to test this path. Still,
we can at least confirm /run/secrets on 'podman run'.
The /sys thing is related to #8949: RHEL8, rootless, cgroups v1.
It's just a workaround to get gating tests to pass on RHEL.
Signed-off-by: Ed Santiago <santiago@redhat.com>
A opened file object of a logfile gets lost because the variable
`logfile` is redefined in a `if` block. This fix stops redefining
the variable.
Signed-off-by: Hironori Shiina <Hironori.Shiina@fujitsu.com>
- run test : tweaks to recently-added network-conflict test:
* remove "-d" in run
* confirm exact warning text, and also that container
runs successfully
* test multiple --net options (regression #8057)
- images, run, build, exec tests: add multiple-flag
testing for various flags, confirming as appropriate
whether options are overridden or accumulated.
- ps test : add --filter and --sort tests
- pod test: run 'ping' inside container (confirms that
container gets PING capability)
Signed-off-by: Ed Santiago <santiago@redhat.com>
- images: confirm that 'podman images' emits headings
even if there are no images present. Intended to
replace e2e test which is difficult to get working
under podman-remote.
- build: add test for #8092, podman-build gobbling stdin.
Workaround needed for issues #8342 and #8343, in which
podman-remote output differs from podman local.
- volumes: add test for #8307, double-lock on same volume.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Check to see if we are recording the version of buildah
used to build the image as a label in the image.
Also we should make sure the filter "since" works.
We are only testing "after", which we don't document.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
- run --userns=keep-id: confirm that $HOME gets set (#8013)
- inspect: confirm that JSON output is a sane number of
lines (10 or more), not an unreadable one-liner (#8011
and #8021). Do so with image, pod, network, volume
because the code paths might be different.
- cgroups: confirm that 'run' preserves cgroup manager (#7970)
- sdnotify: reenable tests, and hope CI doesn't hang. This
test was disabled on August 18 because CI jobs were hanging
and timing out. My suspicion was that it was #7316, which
in turn seems to have hinged on conmon #182. The latter
was merged on Sep 16, so let's cross our fingers and see
what happens.
Also: remove inaccurate warning from a networking test.
And, wow, fix is_cgroupsv2(), it has never actually worked.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- pause test: enable when rootless + cgroups v2
(was previously disabled for all rootless)
- run --pull: now works with podman-remote
(in #7647, thank you @jwhonce)
- various other run/volumes tests: try reenabling
It looks like #7195 was fixed (by #7451? I'm not
sure if I'm reading the conversation correctly).
Anyway, remove all the skip()s on 7195. Only time
will tell if it's really fixed)
Also:
- new test for podman image tree --whatrequires
(because TIL). Doesn't work with podman-remote.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Refactor/Rename channel.WriteCloser() to encapsulate the channel
* Refactor build endpoint to "live" stream buildah output channels
over API rather then buffering output
* Refactor bindings/tunnel build because endpoint changes
* building tar file now in bindings rather then depending on
caller
* Cleanup initiating extra image engine
* Remove setting fields to zero values (less noise in code)
* Update tests to support remote builds
Fixes#7136Fixes#7137
Signed-off-by: Jhon Honce <jhonce@redhat.com>
The test that does 'adduser' in a keep-id container had a
really dumb bug: if the user running the test has UID 1000,
then podman itself (via keep-id) will add the "1000" passwd
entry, and the in-container "adduser" will allocate 1001,
making our test fail. This triggered in f31/f32 podman gating
tests, but (?!?) never in rawhide gating tests.
Solution: explicitly feed a UID to adduser. Make sure that
it's not the same as the UID of the current user.
Also (unrelated): fix a ridiculous "run mkdir || die". At
the time I wrote that I probably had no idea how BATS works.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- new sanity checks for podman-remote:
- first, confirm that when PODMAN is "-remote",
we actually talk to a server (validated by
presence of "Server:" string in "podman version").
- second, add test for #7212, in which we run
"podman --remote" (podman with --remote flag,
not podman-remote command) and make sure --remote
is allowed both as the first option and also
with other flag options preceding.
- new test for "podman image tree" (piggybacking on
top of a "podman build" test, because that gives
us lots of layers).
- skip "podman exec - basic test" when remote. It is consistently
causing CI failures, breaking all of CI, due to #7241.
Signed-off-by: Ed Santiago <santiago@redhat.com>
We need to remove the container after it has exited for
podman-remote run --rm commands. If we don't remove this
container at this step, we open ourselves up to race conditions.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
podman-remote is in better shape now. Let's see what needs
to be done to reenable remote system tests.
- logs test: skip multilog, it doesn't work remote
- diff test: use -l only when local, not with remote
- many other tests: skip_if_remote, with 'FIXME: pending #xxxx'
where xxxx is a filed issue.
Unrelated: added new helper to skip_if_remote and _if_rootless,
where we check if the source message includes "remote"/"rootless"
and insert it if missing. This is a minor usability enhancement
to make it easier to understand at-a-glance why a skip triggers.
Signed-off-by: Ed Santiago <santiago@redhat.com>
