Significant bitrot results in almost immediate test failure. This
commit adds only the very basic, bare-minimum needed to get them
started.
***TESTING RESULTS ARE IGNORED***
Signed-off-by: Chris Evich <cevich@redhat.com>
Currently podman only works with --isolation chroot. This PR
fixes this by allowing the isolation mode to default to OCI and to
also allow users to pass the isolation mode into the containers.
The current tests for --isolation should cause this code to be tested.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Set of scripts to run buildah's bud.bats test using
podman build in podman CI.
podman build is not 100% compatible with buildah bud.
In particular:
* podman defaults to --layers=true; buildah to false
* podman defaults to --force-rm=true; buildah to false
* podman error exit status is 125; buildah is 2
* differences in error messages, command-line arguments
Some of the above can be dealt with programmatically,
by tweaking the buildah helpers.bash (BATS helpers).
Some need to be tweaked by patching bud.bats itself.
This PR includes a patch that will, I fear, need to
be periodically maintained over time.
There will likely be failures when vendoring in a
new buildah, possibly because new tests were added
for new features that don't exist in podman, possibly
(I hope unlikely) if existing tests are changed in
ways that make the patch file fail to apply. I've
tried to write good instructions and to write the run
script in such a way that it will offer helpful hints
on failure. My instructions and code will be imperfect;
I hope they will be good enough to merit continued use
of this test (possibly with improvements to the instructions
as we learn more about real-world failures).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Initial validation of using podman-in-podman to create an
old-podman root, then use new-podman to play with the
containers created therein.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Followup to dbb9943
Despite skipping the `Smoke` check, it was observed on a *new* branch,
the `validate` task (specifically `git-validation`) will fail. This
is because:
* `$CIRRUS_LAST_GREEN_CHANGE` will be empty on a new branch.
* `$CIRRUS_BASE_SHA` is always empty for runs triggered by branch-push
* `$EPOCH_TEST_COMMIT` will be set to `YOU_FOUND_A_BUG`.
Fix this by eliminating the `Smoke` task entirely, simplifying all
the `make validate` operations into the `validate` cirrus task. Ensure
this task does not run when a new branch or tag is pushed.
Also, eliminate the `$CIRRUS_BUILD_ID` value as it's confusing and not
actually used anywhere. It was formerly used for building VM images,
but this has moved to another repo entirely.
Signed-off-by: Chris Evich <cevich@redhat.com>
The podman documentation site uses javascript to display
API documentation at:
http://docs.podman.io/en/latest/Reference.html
As input, the javascript sources from a CORS-enabled Google Cloud
Storage object. This commit ensures the storage object is present and
updated for every Cirrus-CI execution context: Tags, Branches, and PRs.
As of this commit, the documentation site only utilizes the object
uploaded by the Cirrus-CI run on the `master` branch:
`swagger-master.yaml`. The file produced and uploaded due to a PR is
intended for testing purposes: Confirm it's generation and uploading are
both functional.
Signed-off-by: Chris Evich <cevich@redhat.com>
Make sure that bindings are in sync with the code. The check is similar
to what's already being done with `make vendor`, so integrate the two.
[NO TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Previously automation always dropped the minor version number for
distributions. This was intended for presentation and conditional
simplicity. Bash does not support non-integer comparison natively.
With the release of version 20.10, supporting testing with it and
the LTS release (20.04) requires scripts to consider minor version
numbers for Ubuntu VMs. This is necessary because many times in
the past, some behaviors needed to be conditional on the release
version number.
With this commit, the images and embedded scripts/tooling uses an
altered format of `$UBUNTU_NAME', `$PRIOR_UBUNTU_NAME`, and (crucially)
`$OS_RELEASE_VER` and `$OS_REL_VER`. Any `.` characters appearing
in the official version (from `/etc/os-release`) are dropped, and
the result is concatenated.
For example the current Ubuntu LTS version is `20.04`. Prior to
this commit, `$OS_RELEASE_VER` would have been `20`. With this
change, `$OS_RELEASE_VER` will now show `2004`. Similarly `20.10`
is shown as `2010`.
Signed-off-by: Chris Evich <cevich@redhat.com>
Includes disk-space increase for all Fedora images to accommodate
the static-build job disk space requirements. This job substantially
leverages task-cache, which was previously failing to restore early on
in the Cirrus-CI task setup, due to disk-space limitations.
Also simplify .cirrus.yml slightly by removing an unncessary setup
and run directory change step.
Signed-off-by: Chris Evich <cevich@redhat.com>
Test VMs by design are to be single-purpose, single-use, and
readily disposable. Therefore it's unnecessary to overcomplicate
storage of runtime environment variables. This commit makes these
points clear, and reorganizes all CI-related env. vars on the system
into a single location, `/etc/ci_environment`. This file is then
automatically loaded, and variables exported, (by `lib.sh`) from
`runner.sh` prior to executing all forms of testing.
Signed-off-by: Chris Evich <cevich@redhat.com>
My patches to fix `--runtime /usr/bin/crun` being allowed to use
a different version of the crun runtime revealed a problem: we
were actually relying on that exact behavior in our E2E tests. We
specified the runtime path as `/usr/bin/runc` for the Ubuntu
tests, but that didn't exist, so Podman was actively looking for
a different, usable runc binary and using that, instead of the
path we explicitly hardcoded. Fixing the bug broke this, and thus
broke the tests.
Instead of hard-coding OCI runtime paths, swap to just using the
runtime name, `runc` or `crun`, and letting Podman figure out
where the runtime lives - it's quite good at that. This should
un-break the tests and make them more durable.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Also removed automatic exection of setup_environment.sh since most
people using this script are podman developers (not automation/CI
folks). If executing the automation scripts is necessary, manual
attendance to required variables like `$TEST_FLAVOR` is mandatory.
Signed-off-by: Chris Evich <cevich@redhat.com>
Reimplement CI-automation to remove accumulated technical-debt and
optimize workflow. The task-dependency graph designed goal was to
shorten it's depth and increase width (i.e. more parallelism). A
reduction in redundant building (and 3rd party module download) was
also realized by caching `$GOPATH` and `$GOCACHE` early on. This
cache is then reused in favor of a fresh clone of the repository
(when possible).
Note: The system tests typically execute MUCH faster than the
integration tests. However, contrary to a fail-fast/fail-early
principal, they are executed last. This was implemented due to
debug-ability related concerns/preferences of the primary
(golang-centric) project developers.
Signed-off-by: Chris Evich <cevich@redhat.com>
All VM-building functionality has been migrated to
https://github.com/containers/automation_images
Some container-build functions are still maintained here
but are on a very-short list to also be migrated to
the repository linked above.
Signed-off-by: Chris Evich <cevich@redhat.com>
(This is an adoption of #7533 because Brent is on PTO).
Pick up new crun and crio-runc.
Also: renames from useful fedora-32 and -31 to less-useful
names; presumably this is needed by something-something in
the new VM setup.
Also: tweak two e2e tests to more properly handle a kernel
(5.8.4) with a greater set of capabilities than what we
or crun can yet handle.
Signed-off-by: Ed Santiago <santiago@redhat.com>
when running CICD on Ubuntu where no cgroups v2, we need to use a newer runc for things like seccomp and the default ubuntu runc
is not new enough.
Signed-off-by: Brent Baude <bbaude@redhat.com>
It's not possible to run any of the scripts on distributions which do
have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env
bash` instead.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
We know these are TEST_, hoping this makes the display in
cirrus easier for users to see true|false, since this is the
valuable information is.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
- New test for #6991 - passwd file is writable even when
run with --userns=keep-id
- Enable another keep-id test, commented out due to #6593
- New test for podman system df
Also, independently, removed this line:
apt-get -y upgrade conmon
...because it's causing CI failures, probably because of the
boothole CVE, probably because the Ubuntu grub update was
rushed out. I believe it is safe to remove this, because
both Ubuntu 19 and 20 report:
conmon is already the newest version (2.0.18~1).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Primary purpose: upgrade crun to 0.14 on f31, in hopes of
eliminating the 'cgroups.freeze' flake that is plaguing CI.
While I'm at it:
- remove a no-longer-needed dnf upgrade that was running in CI
itself (not image building, in each actual CI run). The purpose
was to upgrade conmon, but that was added a long time ago and
the required conmon is now in stable. The effect of this
dnf upgrade today was simply to cause flakes when fedora
repos were offline.
- remove a no-longer-needed check for varlink.
- networking.sh : add a timeout! 'openssl s_client' will happily
hang forever if a host is unreachable, which means we waste
two hours waiting for Cirrus to time out.
- timestamp.awk : include date (not just time) in START/END msgs.
There are times when I'm looking at a CI log and it is ultra
important to know if it is from yesterday or today.
- add progress messages in some places where I've previously
struggled to understand context in logs; and improve some
unlikely error messages to include script name.
...then, after all that, wrote a new README about how to to
all this. Hope it helps someone.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Currently you can not apply an ApparmorProfile if you specify
--privileged. This patch will allow both to be specified
simultaniosly.
By default Apparmor should be disabled if the user
specifies --privileged, but if the user specifies --security apparmor:PROFILE,
with --privileged, we should do both.
Added e2e run_apparmor_test.go
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When the container uses journald logging, we don't want to
automatically use the same driver for its exec sessions. If we do
we will pollute the journal (particularly in the case of
healthchecks) with large amounts of undesired logs. Instead,
force exec sessions logs to file for now; we can add a log-driver
flag later (we'll probably want to add a `podman logs` command
that reads exec session logs at the same time).
As part of this, add support for the new 'none' logs driver in
Conmon. It will be the default log driver for exec sessions, and
can be optionally selected for containers.
Great thanks to Joe Gooch (mrwizard@dok.org) for adding support
to Conmon for a null log driver, and wiring it in here.
Fixes#6555
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This allows the containers/conmon repository to share the same VM
images produced by containers/libpod. Included are several packages
which are downloaded only since they might otherwise interfere with
testing for some repos. This allows stable versions to be at the ready
at testing runtime, avoiding any version updates surprising developers.
Also, re-enable running the VM-image check test which was not working
due to a logic problem in Cirrus-CI configuration. Update the neglected
tests so that they pass on all distros.
Signed-off-by: Chris Evich <cevich@redhat.com>
The biggest obstacle here was cleanup - we needed a way to remove
detached exec sessions after they exited, but there's no way to
tell if an exec session will be attached or detached when it's
created, and that's when we must add the exit command that would
do the removal. The solution was adding a delay to the exit
command (5 minutes), which gives sufficient time for attached
exec sessions to retrieve the exit code of the session after it
exits, but still guarantees that they will be removed, even for
detached sessions. This requires Conmon 2.0.17, which has the new
`--exit-delay` flag.
As part of the exit command rework, we can drop the hack we were
using to clean up exec sessions (remove them as part of inspect).
This is a lot cleaner, and I'm a lot happier about it.
Otherwise, this is just plumbing - we need a bindings call for
detached exec, and that needed to be added to the tunnel mode
backend for entities.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Now that we're shipping containers.conf, we don't want to provide
a libpod.conf anymore. This removes libpod.conf from the repo and
as many direct uses as I can find.
There are a few more mentions in the documentation, but someone
more familiar with containers.conf should make those edits.
Signed-off-by: Matthew Heon <mheon@redhat.com>
some small fix ups for binding tests and then make them required.
update containers-common
V2 bindings tests were failing because of changes introduced in commit
a2ad5bb.
Fix some typos.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
in the case where the specgen attribute for Env and Labels are nil, we should should then make the map IF we have labels and envs that need to be added.
Signed-off-by: Brent Baude <bbaude@redhat.com>
this is a temporary fix for the flake that has been troubling us. once conmon is in fedora 30 and 31 stable, we can remove this fix. the images will just need to be rebuilt.
Signed-off-by: Brent Baude <bbaude@redhat.com>
during container creation, if no network is provided, we need to add a default value so the container can be later started.
use apiv2 container creation for RunTopContainer instead of an exec to the system podman. RunTopContainer now also returns the container id and an error.
added a libpod commit endpoint.
also, changed the use of the connections and bindings slightly to make it more convenient to write tests.
Fixes: 5366
Signed-off-by: Brent Baude <bbaude@redhat.com>
Suspect crun might be sneaking in during VM image build via podman RPM
dependency. Add it to the removal list when building, then also force
use of runc at runtime in F30.
Also quote all true/false vars to force them as strings instead of
booleans (which will become capitalized)
Signed-off-by: Chris Evich <cevich@redhat.com>
There are a number of env. vars set during the setup script. Therefore
displaying them at end of the script is more helpful for debugging.
Signed-off-by: Chris Evich <cevich@redhat.com>
In some distributions it's possible to have both runc and crun
installed and/or for podman to be confused about which to use. In these
instances, force the decision by adding `OCI_RUNTIME=/usr/bin/crun` into
`/etc/environment`. Also in-place modify libpod.conf to use 'crun'
instead of 'runc'
Signed-off-by: Chris Evich <cevich@redhat.com>
