System tests: the continuing multiarch saga

TL;DR podman needs "arm64" as arch, not "arm64v8".

Unexpurgated version: docker.io publishes ${ARCH}/alpine for
several values of ARCH. Unfortunately, the arm64 one is
called "arm64v8", which is sensible, but podman needs the
--arch value of the manifest to be exactly "arm64". So we
need to special-case this value in our loop. Do so, and
build/publish a new 20210610 testimage. Use that in tests
moving forward.

And, since we need to jump through the same hoops to build
the nonlocal image, include it in the build loop instead
of as a tacked-on comment. Try to be helpful by determining
the next-available numeric tag.

And: don't push anything by default. Instead, just tell
the user what buildah-push commands to run.

And: refactor $PODMAN_NONLOCAL_IMAGE_TAG, to make it easier
for the RHEL-arch-testing folx to override using envariables
instead of inplace-sed. (Not that they should ever need to
override again, because this is the final multiarch commit
that should be forevermore perfect and need no further commits
ever again).

And, finally, bump up to latest alpine/busybox images.

Signed-off-by: Ed Santiago <santiago@redhat.com>

test/system/build-testimage

@@ -61,8 +61,8 @@ chmod 755 pause
 #    - check for updates @ https://hub.docker.com/_/alpine
 # busybox-extras provides httpd needed in 500-networking.bats
 cat >Containerfile <<EOF
-ARG ARCH=please-override-arch
+ARG REPO=please-override-repo
-FROM docker.io/\${ARCH}/alpine:3.12.0
+FROM docker.io/\${REPO}/alpine:3.13.5
 RUN apk add busybox-extras
 ADD testimage-id pause /home/podman/
 LABEL created_by=$create_script
@@ -74,17 +74,46 @@ EOF
 # --squash-all    : needed by 'tree' test in 070-build.bats
 podman rmi -f testimage &> /dev/null || true
 
+# There should always be a testimage tagged ':0000000<X>' (eight digits,
+# zero-padded sequence ID) in the same location; this is used by tests
+# which need to pull a non-locally-cached image. This image will rarely
+# if ever need to change, nor in fact does it even have to be a copy of
+# this testimage since all we use it for is 'true'.
+# However, it does need to be multiarch :-(
+zerotag_latest=$(skopeo list-tags docker://quay.io/libpod/testimage |\
+                     jq -r '.Tags[]' |\
+                     sort --version-sort |\
+                     grep '^000' |\
+                     tail -n 1)
+zerotag_next=$(printf "%08d" $((zerotag_latest + 1)))
+
+# We don't always need to push the :00xx image, but build it anyway.
+zeroimg=quay.io/libpod/testimage:${zerotag_next}
+buildah manifest create $zeroimg
+
 # We need to use buildah because (as of 2021-02-23) only buildah has --manifest
 # and because Dan says arch emulation is not currently working on podman
 # (no further details).
 # Arch emulation on Fedora requires the qemu-user-static package.
-for arch in amd64 arm64v8 ppc64le s390x;do
+for arch in amd64 arm64 ppc64le s390x;do
+    # docker.io repo is usually the same name as the desired arch; except
+    # for arm64, where podman needs to have the arch be 'arm64' but the
+    # image lives in 'arm64v8'.
+    repo=$arch
+    if [[ $repo = "arm64" ]]; then
+        repo="${repo}v8"
+    fi
+
     ${BUILDAH} bud \
                --arch=$arch \
-               --build-arg ARCH=$arch \
+               --build-arg REPO=$repo \
                --manifest=testimage \
                --squash \
                .
+
+    # The zero-tag image
+    ${BUILDAH} pull --arch $arch     docker.io/$repo/busybox:1.33.1
+    ${BUILDAH} manifest add $zeroimg docker.io/$repo/busybox:1.33.1
 done
 
 # Clean up
@@ -94,23 +123,13 @@ rm -rf $tmpdir
 # Tag image and push (all arches) to quay.
 remote_tag=quay.io/libpod/testimage:$YMD
 podman tag testimage ${remote_tag}
-${BUILDAH} manifest push --all  ${remote_tag} docker://${remote_tag}
+cat <<EOF
 
-# Side note: there should always be a testimage tagged ':0000000<X>'
+If you're happy with these images, run:
-# (eight digits, zero-padded sequence ID) in the same location; this is
+
-# used by tests which need to pull a non-locally-cached image. This
+   ${BUILDAH} manifest push --all  ${remote_tag} docker://${remote_tag}
-# image will rarely if ever need to change, nor in fact does it even
+   ${BUILDAH} manifest push --all  ${zeroimg} docker://${zeroimg}
-# have to be a copy of this testimage since all we use it for is 'true'.
+
-# However, it does need to be multiarch :-(
+(You do not always need to push the :0000 image)
-#
+
-# As of 2021-02-24 it is simply busybox, because it is super small,
+EOF
-# but it's complicated because of multiarch:
-#
-#    img=quay.io/libpod/testimage:0000000<current+1>
-#    buildah manifest create $img
-#    for arch in amd64 arm64v8 ppc64le s390x;do
-#        buildah pull --arch $arch docker.io/$arch/busybox:1.32.0
-#        buildah manifest add $img docker.io/$arch/busybox:1.32.0
-#    done
-#    buildah manifest push --all $img docker://$img
-#

test/system/helpers.bash

@@ -7,14 +7,15 @@ PODMAN=${PODMAN:-podman}
 PODMAN_TEST_IMAGE_REGISTRY=${PODMAN_TEST_IMAGE_REGISTRY:-"quay.io"}
 PODMAN_TEST_IMAGE_USER=${PODMAN_TEST_IMAGE_USER:-"libpod"}
 PODMAN_TEST_IMAGE_NAME=${PODMAN_TEST_IMAGE_NAME:-"testimage"}
-PODMAN_TEST_IMAGE_TAG=${PODMAN_TEST_IMAGE_TAG:-"20210427"}
+PODMAN_TEST_IMAGE_TAG=${PODMAN_TEST_IMAGE_TAG:-"20210610"}
 PODMAN_TEST_IMAGE_FQN="$PODMAN_TEST_IMAGE_REGISTRY/$PODMAN_TEST_IMAGE_USER/$PODMAN_TEST_IMAGE_NAME:$PODMAN_TEST_IMAGE_TAG"
 PODMAN_TEST_IMAGE_ID=
 
 # Remote image that we *DO NOT* fetch or keep by default; used for testing pull
-# This changed from 0 to 1 on 2021-02-24 due to multiarch considerations; it
+# This has changed in 2021, from 0 through 3, various iterations of getting
-# should change only very rarely.
+# multiarch to work. It should change only very rarely.
-PODMAN_NONLOCAL_IMAGE_FQN="$PODMAN_TEST_IMAGE_REGISTRY/$PODMAN_TEST_IMAGE_USER/$PODMAN_TEST_IMAGE_NAME:00000002"
+PODMAN_NONLOCAL_IMAGE_TAG=${PODMAN_NONLOCAL_IMAGE_TAG:-"00000003"}
+PODMAN_NONLOCAL_IMAGE_FQN="$PODMAN_TEST_IMAGE_REGISTRY/$PODMAN_TEST_IMAGE_USER/$PODMAN_TEST_IMAGE_NAME:$PODMAN_NONLOCAL_IMAGE_TAG"
 
 # Because who wants to spell that out each time?
 IMAGE=$PODMAN_TEST_IMAGE_FQN