Test VMs by design are to be single-purpose, single-use, and
readily disposable. Therefore it's unnecessary to overcomplicate
storage of runtime environment variables. This commit makes these
points clear, and reorganizes all CI-related env. vars on the system
into a single location, `/etc/ci_environment`. This file is then
automatically loaded, and variables exported, (by `lib.sh`) from
`runner.sh` prior to executing all forms of testing.
Signed-off-by: Chris Evich <cevich@redhat.com>
My patches to fix `--runtime /usr/bin/crun` being allowed to use
a different version of the crun runtime revealed a problem: we
were actually relying on that exact behavior in our E2E tests. We
specified the runtime path as `/usr/bin/runc` for the Ubuntu
tests, but that didn't exist, so Podman was actively looking for
a different, usable runc binary and using that, instead of the
path we explicitly hardcoded. Fixing the bug broke this, and thus
broke the tests.
Instead of hard-coding OCI runtime paths, swap to just using the
runtime name, `runc` or `crun`, and letting Podman figure out
where the runtime lives - it's quite good at that. This should
un-break the tests and make them more durable.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Also removed automatic exection of setup_environment.sh since most
people using this script are podman developers (not automation/CI
folks). If executing the automation scripts is necessary, manual
attendance to required variables like `$TEST_FLAVOR` is mandatory.
Signed-off-by: Chris Evich <cevich@redhat.com>
While reviewing #6784 I found myself having a lot of trouble
with this script: it was a complicated mix of case statement
and helper functions, requiring a reader to jump back and
forth between the two.
This PR defines a convention such that a given TEST_FLAVOR=foo
must have a corresponding _run_foo() handler function. The
goal is to have all TEST_FLAVOR-related code in one place,
or at least less scattered (integration and system tests
still rely on other helper functions).
Signed-off-by: Ed Santiago <santiago@redhat.com>
(that got accidentally dropped in the new Cirrus makeover).
Note that 'dotest' does not actually 'do tests', it's
only used for a small subset of tests.
Also, make logformatter work better in the new Cirrus setup.
Remove duplicate test/subtest, remove no-longer-used SPECIALMODE,
and make the Cirrus build/task display a little cleaner.
Signed-off-by: Ed Santiago <santiago@redhat.com>
When validating code on a branch, determining a starting commit to check
from isn't as straightforward as it would seem. Default to using the
SHA from last time CI was green. If for some reason that isn't
available, use an obviously wrong value to cause an intentional
failure. Entirely skip this check on tag-push, since determining a
starting point is incredibly difficult to do automatically.
Signed-off-by: Chris Evich <cevich@redhat.com>
Reimplement CI-automation to remove accumulated technical-debt and
optimize workflow. The task-dependency graph designed goal was to
shorten it's depth and increase width (i.e. more parallelism). A
reduction in redundant building (and 3rd party module download) was
also realized by caching `$GOPATH` and `$GOCACHE` early on. This
cache is then reused in favor of a fresh clone of the repository
(when possible).
Note: The system tests typically execute MUCH faster than the
integration tests. However, contrary to a fail-fast/fail-early
principal, they are executed last. This was implemented due to
debug-ability related concerns/preferences of the primary
(golang-centric) project developers.
Signed-off-by: Chris Evich <cevich@redhat.com>
At the top of each generated page, add a Synopsis table with:
PR number/name, and link to github
Author name(s)
Test name (fedora/ubuntu, rootless, etc)
Cirrus build ID (usually uninteresting)
Cirrus task ID (usu. important), with link to Cirrus
The value of $SPECIALMODE
This is all we can get from the Cirrus environment in
which logformatter runs; we can't get things like
cgroup manager or username that the test runs under.
Note that the table is at the top, which is usually
unseen because we autoscroll to the bottom on
page load. I tentatively think that top is a more
natural place for this info than bottom, but am
willing to listen to arguments against.
Also, one minor tweak: highlight podman commands in
the BATS output. The idea is to make it easier for the eye
to spot those, then copy/paste them to find a reproducer.
And, sigh, disable the new 'podman network create'
system test. It is flaking much too much.
Signed-off-by: Ed Santiago <santiago@redhat.com>
dependabot seems to submit PRs without running 'make vendor'.
This script automates (with some safety checks) the manual
process for pulling the PR, running 'make vendor-in-container',
and force-pushing the PR.
Usage: ./contrib/dependabot-dance
It should take care of identifying your github repo, finding
all active dependabot branches, running the make, git-add,
and commit, then git-pushing.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Move the `rootless-cni-infra` image to `quay.io/libpod/rootless-cni-image:$tag`
where $tag has the format `$version-$architecture`. Whenever we upload
a new image (e.g., after changing the Containerfile), we need to make
sure to increase the version number (an ordinary integer for simplicity) so we
have a notion of support.
Thanks to @AkihiroSuda for working on rootless CNI!
Fixes: #7617
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
All VM-building functionality has been migrated to
https://github.com/containers/automation_images
Some container-build functions are still maintained here
but are on a very-short list to also be migrated to
the repository linked above.
Signed-off-by: Chris Evich <cevich@redhat.com>
We've recently had a number of issues reported against our
pre-fabricated images on quay.io and a couple of rhel repositories
throwing a fuse error when run:
```
fuse: device not found, try 'modprobe fuse' first
```
The tip on modprobe fuse is not always seen by or displayed to
the end user. Adding a couple of doc pointers to hopefully help.
Arises from this BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1867892
and several others.
Replaces: 7453 where I was going crazy with whitespace and merge issues.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
(This is an adoption of #7533 because Brent is on PTO).
Pick up new crun and crio-runc.
Also: renames from useful fedora-32 and -31 to less-useful
names; presumably this is needed by something-something in
the new VM setup.
Also: tweak two e2e tests to more properly handle a kernel
(5.8.4) with a greater set of capabilities than what we
or crun can yet handle.
Signed-off-by: Ed Santiago <santiago@redhat.com>
when running CICD on Ubuntu where no cgroups v2, we need to use a newer runc for things like seccomp and the default ubuntu runc
is not new enough.
Signed-off-by: Brent Baude <bbaude@redhat.com>
It's not possible to run any of the scripts on distributions which do
have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env
bash` instead.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Commit 2b6dd3fb4384 set the killmode of the podman.service to the
systemd default which ultimately lead to the problem that systemd
will kill *all* processes inside the unit's cgroup and hence kill
all containers whenever the service is stopped.
Fix it by setting the type to sdnotify and the killmode to process.
`podman system service` will send the necessary notify messages
when the NOTIFY_SOCKET is set and unset it right after to prevent
the backend and container runtimes from jumping in between and send
messages as well.
Fixes: #7294
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
correct small typo that sets the path on windows via the msi xml.
in the remote client, prompt for SSH password when no identity or alternate means of authentication are provided.
Signed-off-by: Brent Baude <bbaude@redhat.com>
xz package is required by buildah and podman when building a
image and ADD a tar.xz file archive is used
See https://github.com/containers/buildah/issues/2525
Signed-off-by: Job Cespedes Ortiz <jobcespedes@gmail.com>
We know these are TEST_, hoping this makes the display in
cirrus easier for users to see true|false, since this is the
valuable information is.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Updates to the README.md for the contrib/podmanimages directory.
This completes the changes to answer this Buildah issue: https://github.com/containers/buildah/issues/1693
and then also adds the quay.io/conatiners/podman images to the list of images.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
podman.service is socket activated through podman.socket. It should not
have its own [Install] section, it does not make sense to systemctl
enable podman.service.
This leads to podman.service always running on a Debian system, as
Debian's policy is to enable/start running services by default.
We don't want a daemon :^)
Fixes: #7190
Reported-by: @martinpitt
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
They are needed in support of future testing additions.
Also reduce unnecessary output by not printing the downloaded package
list. The set can be examined using other tooling if/when necessary.
Signed-off-by: Chris Evich <cevich@redhat.com>
- New test for #6991 - passwd file is writable even when
run with --userns=keep-id
- Enable another keep-id test, commented out due to #6593
- New test for podman system df
Also, independently, removed this line:
apt-get -y upgrade conmon
...because it's causing CI failures, probably because of the
boothole CVE, probably because the Ubuntu grub update was
rushed out. I believe it is safe to remove this, because
both Ubuntu 19 and 20 report:
conmon is already the newest version (2.0.18~1).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Problem: formatted logs no longer have live links to sources
in error-report lines.
Cause: script was searching for '/libpod'.
Solution: make it more flexible.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Primary purpose: upgrade crun to 0.14 on f31, in hopes of
eliminating the 'cgroups.freeze' flake that is plaguing CI.
While I'm at it:
- remove a no-longer-needed dnf upgrade that was running in CI
itself (not image building, in each actual CI run). The purpose
was to upgrade conmon, but that was added a long time ago and
the required conmon is now in stable. The effect of this
dnf upgrade today was simply to cause flakes when fedora
repos were offline.
- remove a no-longer-needed check for varlink.
- networking.sh : add a timeout! 'openssl s_client' will happily
hang forever if a host is unreachable, which means we waste
two hours waiting for Cirrus to time out.
- timestamp.awk : include date (not just time) in START/END msgs.
There are times when I'm looking at a CI log and it is ultra
important to know if it is from yesterday or today.
- add progress messages in some places where I've previously
struggled to understand context in logs; and improve some
unlikely error messages to include script name.
...then, after all that, wrote a new README about how to to
all this. Hope it helps someone.
Signed-off-by: Ed Santiago <santiago@redhat.com>
