WIP: run buildah bud tests using podman

Set of scripts to run buildah's bud.bats test using
podman build in podman CI.

podman build is not 100% compatible with buildah bud.
In particular:

  * podman defaults to --layers=true; buildah to false
  * podman defaults to --force-rm=true; buildah to false
  * podman error exit status is 125; buildah is 2
  * differences in error messages, command-line arguments

Some of the above can be dealt with programmatically,
by tweaking the buildah helpers.bash (BATS helpers).
Some need to be tweaked by patching bud.bats itself.
This PR includes a patch that will, I fear, need to
be periodically maintained over time.

There will likely be failures when vendoring in a
new buildah, possibly because new tests were added
for new features that don't exist in podman, possibly
(I hope unlikely) if existing tests are changed in
ways that make the patch file fail to apply. I've
tried to write good instructions and to write the run
script in such a way that it will offer helpful hints
on failure. My instructions and code will be imperfect;
I hope they will be good enough to merit continued use
of this test (possibly with improvements to the instructions
as we learn more about real-world failures).

Signed-off-by: Ed Santiago <santiago@redhat.com>
This commit is contained in:
Ed Santiago
2021-02-09 14:33:19 -07:00
parent ebc9871c93
commit 3baa9da4ed
7 changed files with 532 additions and 0 deletions

View File

@ -579,6 +579,29 @@ remote_system_test_task:
TEST_FLAVOR: sys
PODBIN_NAME: remote
buildah_bud_test_task:
name: *std_name_fmt
alias: buildah_bud_test
skip: *tags
only_if: *not_docs
depends_on:
- local_integration_test
env:
TEST_FLAVOR: bud
DISTRO_NV: ${FEDORA_NAME}
# Not used here, is used in other tasks
VM_IMAGE_NAME: ${FEDORA_CACHE_IMAGE_NAME}
CTR_FQIN: ${FEDORA_CONTAINER_FQIN}
# ID for re-use of build output
_BUILD_CACHE_HANDLE: ${FEDORA_NAME}-build-${CIRRUS_BUILD_ID}
PODBIN_NAME: podman
gce_instance: *standardvm
timeout_in: 45m
clone_script: *noop
gopath_cache: *ro_gopath_cache
setup_script: *setup
main_script: *main
always: *int_logs_artifacts
rootless_system_test_task:
name: *std_name_fmt
@ -687,6 +710,7 @@ success_task:
- remote_system_test
- rootless_system_test
- upgrade_test
- buildah_bud_test
- meta
container: *smallcontainer
env: