EL8 builds are failing because hack/markdown-preprocess needs python3
which AFAICT isn't included by default in EL8 build environments.
This commit also includes an additional `[CI:COPR]` mode which is
currently runs the same tests as `[CI:DOCS]` but could differ in future.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
we were not using the correct GOARCH to build the podman remote
and podman-mac-helper binaries, this uses the ARCH value passed
to the make invocation to set the GORACH
[NO NEW TESTS NEEDED]
Signed-off-by: Anjan Nath <kaludios@gmail.com>
this updates downloading of gvproxy and qemu using a standard
makefile rule which will avoid downloading them again if its
already downloaded
[NO NEW TESTS NEEDED]
Signed-off-by: Anjan Nath <kaludios@gmail.com>
add file hvf.entitlements which has the com.apple.security.hypervisor
entitlement needed for qemu
[NO NEW TESTS NEEDED]
Signed-off-by: Anjan Nath <kaludios@gmail.com>
new file: test/e2e/config_arm64.go
Tests that fail on aarch64 have been skipped with
`skip_if_aarch64`.
Co-authored-by: Chris Evich <cevich@redhat.com>
Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
it installs podman and supporting binaries along with
qemu to have a functioning podman install using a pkg
podman and podman-mac-helper is compiled from source
gvproxy binary is downloaded from its github releases
and qemu from github release of containers/podman-machine-qemu
[NO NEW TESTS NEEDED]
Signed-off-by: Anjan Nath <kaludios@gmail.com>
...and enable the at-test-time confirmation, the one that
double-checks that if CI requests runc we actually use runc.
This exposed a nasty surprise in our setup: there are steps to
define $OCI_RUNTIME, but that's actually a total fakeout!
OCI_RUNTIME is used only in e2e tests, it has no effect
whatsoever on actual podman itself as invoked via command
line such as in system tests. Solution: use containers.conf
Given how fragile all this runtime stuff is, I've also added
new tests (e2e and system) that will check $CI_DESIRED_RUNTIME.
Image source: https://github.com/containers/automation_images/pull/146
Since we haven't actually been testing with runc, we need
to fix a few tests:
- handle an error-message change (make it work in both crun and runc)
- skip one system test, "survive service stop", that doesn't
work with runc and I don't think we care.
...and skip a bunch, filing issues for each:
- #15013 pod create --share-parent
- #15014 timeout in dd
- #15015 checkpoint tests time out under $CONTAINER
- #15017 networking timeout with registry
- #15018 restore --pod gripes about missing --pod
- #15025 run --uidmap broken
- #15027 pod inspect cgrouppath broken
- ...and a bunch more ("podman pause") that probably don't
even merit filing an issue.
Also, use /dev/urandom in one test (was: /dev/random) because
the test is timing out and /dev/urandom does not block. (But
the test is still timing out anyway, even with this change)
Also, as part of the VM switch we are now using go 1.18 (up
from 1.17) and this broke the gitlab tests. Thanks to @Luap99
for a quick fix.
Also, slight tweak to #15021: include the timeout value, and
reword message so command string is at end.
Also, fixed a misspelling in a test name.
Fixes: #14833
Signed-off-by: Ed Santiago <santiago@redhat.com>
It was not obvious enough in the scripts how much of a snowflake this
environment is. Fix that with lots of capitalized words and asterisks.
Signed-off-by: Chris Evich <cevich@redhat.com>
We're still not testing runc in CI (#14833), and it may be weeks
or months before we can, due to criu/glibc nightmare, but one day
we'll be back on track, then later on we'll update VMs again,
and screw it up, and lose runc, and not notice, and RHEL will
break, and oh noes headless chicken again, repeat repeat.
We can do better. Use .cirrus.yml to explicitly define which
VMs should use which runtimes, and enforce it early in the
CI build step. This should never fail (uh huh) in a PR,
only in one of the update-VM PRs.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Correct spelling and typos.
* Improve language.
Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
Reason: task IDs are unique and permanent; linking by
build ID and task name is non-unique, because Re-run.
Fixes: #14863
Signed-off-by: Ed Santiago <santiago@redhat.com>
The podman-machine integration tests are designed to execute on
bare-metal, since they perform significant work with virtual-machines.
This test is costly to run at scale, so it is limited to being manually
triggered by developers (for now). A 'trigger' button will appear in the
task status page of the Github WebUI once all test dependencies are met.
In the Cirrus-CI WebUI, there is also a 'pre-trigger' button that may be
pressed if a developer doesn't wish to wait. Also:
* Add a `localmachine` target in the `Makefile` on the off-chance
developers wish to execute locally. Update the `ginkgo-run` target
to accommodate re-use by the new `localmachine` target.
* Exclude `podman_machine` task from `success` dependency verification.
This also involves adding an exception to `cirrus_yaml_test.py`
otherwise it will complain loudly.
* ***NOTE*** Inclusion of `ec2_instance` in *any* task will cause
`hack/get_ci_vm.sh` to barf and be non-functional. Future updates will
be made to restore functionality. Before then, simply comment out
the `ec2_instance` section as a temporarily workaround.
Signed-off-by: Chris Evich <cevich@redhat.com>
In order to support execution on various non-GCP cloud environments, the
BFQ scheduler workaround needs updating. Previously it assumed the root
disk was always `/dev/sda`. With the addition of new clouds (AWS) and
different environment types, the assumption is not always valid. Update
the workaround to take care in looking up the block device where '/'
comes from.
Also update the scheduler to 'none', as all modern clouds already have
highly optimized underlying storage configurations. There's no reason
to complicate I/O paths further by hard-coding specific scheduler(s) for
all environment types.
Signed-off-by: Chris Evich <cevich@redhat.com>
With the upcoming plans of introducing a podman-kube command with
various subcommands, rename the podman-play-kube systemd template
to podman-kube before releasing it.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Drop a reference as to why the `rpm --setcaps...` line is needed, along
with a `TODO` reminder to check if it's still needed.
Signed-off-by: Chris Evich <cevich@redhat.com>
add support for podman-remote image scp as well as direct access via the API. This entailed
a full rework of the layering of image scp functions as well as the usual API plugging and type creation
also, implemented podman image scp tagging. which makes the syntax much more readable and allows users t tag the new image
they are loading to the local/remote machine:
allow users to pass a "new name" for the image they are transferring
`podman tag` as implemented creates a new image im `image list` when tagging, so this does the same
meaning that when transferring images with tags, podman on the remote machine/user will load two images
ex: `podman image scp computer1::alpine computer2::foobar` creates alpine:latest and localhost/foobar on the remote host
implementing tags means removal of the flexible syntax. In the currently released podman image scp, the user can either specify
`podman image scp source::img dest::` or `podman image scp dest:: source::img`. However, with tags this task becomes really hard to check
which is the image (src) and which is the new tag (dst). Removal of that streamlines the arg parsing process
Signed-off-by: Charlie Doern <cdoern@redhat.com>
One day we may use AWS for part of CI. Do you want to maintain
two separate code paths in this script for linking to artifacts
in multiple cloud providers? Can you say no? I knew you could.
Cirrus already knows the location of the artifacts and provides
a transparent mechanism for accessing them. Use it.
This PR exposed a nasty bug in our environment-variable handling:
envariables passed through to the containerized environment were
being double-space-escaped, so "FOO=a b" ended up as "FOO=a\ b"
(with a backslash), with one consequence being invalid URLs.
The solution is simple: run 'podman -e FOO', not '-e FOO=value'.
Finally, reinstate the environment-variable dump (in comments).
I had removed this in a moment of panic over leaking secrets,
but no, that doesn't happen. Exclude scary-sounding vars anyway.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Replace "setup", "lookup", "cleanup", "backup" with
"set up", "look up", "clean up", "back up"
when used as verbs. Replace also variations of those.
* Improve language in a few places.
Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
The Containerfiles were built with sed -i, which is leading to empty
storage.conf files. This will cause Podman in a container to print
warning information about storage.driver not being set to something.
[NO NEW TESTS REQUIRED]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Using both the 'skip' and 'only_if' features at the same time may be
hard for maintainers to decipher. Consolidate them into `only_if` since
that bypasses creation of the task all together - meaning there are
potentially fewer tasks for a developer to scroll through.
Since the `multiarch` Cirrus-Cron build no-longer depends on the direct
"build-ability" from the current repo. state, it can be further
optimized. When operating in this context, avoid running many/most
other tasks, depending instead only on `ext_svc_check`.
Finally, add a simple document describing the various runtime contexts
along with the list of expected tasks. Reference this prominently right
in front of every `only_if` so it's impossible for a maintainer to miss.
Signed-off-by: Chris Evich <cevich@redhat.com>
This was an accidental leftover from an in-development implementation.
The `sed` command further down entirely replaces the file in the image.
Strip out the unnecessary 'storage.conf' ADD instruction.
Signed-off-by: Chris Evich <cevich@redhat.com>
@cevich recently renamed all the files named Dockerfile to Containerfile
in this directory. Touching up the README.md to reflect that.
Also, as I was doing the submit, I noticed a couple of nits in the PR
request template and cleaned those up.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Fixes#14021
Substitution values built from `$(shell ...)` output can easily be empty
due to the shell's default `pipefail` behavior. This can also hide
non-zero exit codes, similarly resulting in empty values being set.
While not a perfect fix, the situation is improved by using the
`err_if_empty` function in all cases where empty values would be
unexpected. Remove the definitions for `GIT_BRANCH` and
`GIT_BRANCH_CLEAN` which don't seem to actually be used anywhere
(including in code).
Add a simple release-test to verify `podman info` outputs a non-empty
value for "GitCommit".
Signed-off-by: Chris Evich <cevich@redhat.com>
As of Fedora 36, `/etc/containers/storage.conf` with defaults is
installed under `/usr/share/containers/`. This was causing builds
to fail in the necessary `sed` command that enables fuse-overlayfs.
Fix this by using sed on the new location with an output redirect
into the `etc` location.
Also, perform a mass-cleanup of the three files to make them easier
to read/maintain. Including renaming them to `Containerfile`,
since all native build tooling is now used to produce them.
Lastly, take advantage of the `podman-next` copr repository to install
the latest/greatest podman from `main`, rather than building it from
scratch. This will greatly speed up the image build speed.
Signed-off-by: Chris Evich <cevich@redhat.com>
Most were simply deleted, the main one addressed is in the
"pre-testing" `ext_svc_check.sh` script. It will now verify
accessibility of several key test images we maintain in `quay.io`.
Signed-off-by: Chris Evich <cevich@redhat.com>
Support running `podman play kube` in systemd by exploiting the
previously added "service containers". During `play kube`, a service
container is started before all the pods and containers, and is stopped
last. The service container communicates its conmon PID via sdnotify.
Add a new systemd template to dispatch such k8s workloads. The argument
of the template is the path to the k8s file. Note that the path must be
escaped for systemd not to bark:
Let's assume we have a `top.yaml` file in the home directory:
```
$ escaped=$(systemd-escape ~/top.yaml)
$ systemctl --user start podman-play-kube@$escaped.service
```
Closes: https://issues.redhat.com/browse/RUN-1287
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Followup to #14178: I broke CI. "--fork-point" does not do
what I thought it did, so remove it.
Also, add 'set -x' to help us debug the next time something
like this happens.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add support for new automation library version w/ `$DEBUG` fix
(ref: containers/automation_images#128) and added definitions
for commonly used Distro/version variables.
Signed-off-by: Chris Evich <cevich@redhat.com>
With the increasingly complex `.cirrus.yml` task relationships, build
cache wasn't always working as intended. Recently, non-build tasks were
observed assuming authority over `gopath_cache`. Ref.:
https://github.com/containers/podman/pull/13998#issuecomment-1108834538
Address this by an overall simplification using artifacts instead of
cache. Using artifacts allows establishing concrete
authorship/authority over cached repo. content. In this way, dependent
tasks may simply consume the artifact with `curl` instead of relying on
complex caching algorithms.
Also/Minor: Add YAML checking to the pre-commit configuration.
Signed-off-by: Chris Evich <cevich@redhat.com>
The sshd service is guaranteed to be running by the VM image build
process - it's required by the packer tool for access. Remove the
startup and check on the sshd service.
For many tests, man ssh connections to/from $ROOTLESS_USER on the
host are needed. To facilitate this, the localhost key is added to
`known_hosts` for root and `$ROOTLESS_USER`. Simplify this setup using
the `ssh-keyscan` tool.
Signed-off-by: Chris Evich <cevich@redhat.com>
There are several runtime contexts (rootless and container) where
`$DEST_BRANCH` is needed but was not supplied. A prior commit
(c4865767171b) removed the default value, `main` which was being
set incorrectly when CI ran on release branches. Fix this by ensuring
the variable is non-empty upon entry to `setup_environment.sh`, then
ensure it gets passed through to child environments by way of the
`/etc/ci_environment` file. This will maintain compatibility with
both CI and `hack/get_ci_vm.sh` use.
Signed-off-by: Chris Evich <cevich@redhat.com>
