454 Commits

Author SHA1 Message Date
d36c62d88d Revert "Cirrus: Temp. workaround missing imgprune image"
This reverts commit 9b2e98f1e872354f0708a86b59e16b8b86e9f8b2.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-08-12 15:04:30 -04:00
5f99703594 Cirrus: Add verification for cgroupv2 image
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-08-12 12:01:55 -04:00
7508179ed9 Cirrus: Add experimental fedora VM image & test
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-08-12 12:01:55 -04:00
fd312ae30a Cirrus: Minor, use newer Ubuntu base image
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-08-12 11:02:02 -04:00
0798fc5fc4 cirrus: install crun
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2019-08-12 14:11:59 +02:00
926901d78d Merge pull request #3594 from cevich/second_partition
Add Second partition for storage-drive testing
2019-08-10 12:10:43 +02:00
b6e2bc3cc9 Merge pull request #3770 from cevich/updates_testing
Cirrus: Enable updates-testing repo for Fedora
2019-08-10 11:43:44 +02:00
10834c525f Bump to v1.5.1-dev
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2019-08-09 16:00:38 -04:00
54b394bd3e Bump to v1.5.0
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2019-08-09 16:00:35 -04:00
decfea65be add --pull flag for podman create&run
Requirement from https://github.com/containers/libpod/issues/3575#issuecomment-512238393

Added --pull for podman create and pull to match the newly added flag in docker CLI.
`missing`: default value, podman will pull the image if it does not exist in the local.
`always`: podman will always pull the image.
`never`: podman will never pull the image.

Signed-off-by: Qi Wang <qiwan@redhat.com>
2019-08-09 15:21:34 -04:00
ec2ea0278e Cirrus: Enable updates-testing repo for Fedora
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-08-09 10:25:50 -04:00
096af85278 add make to make installs
as issue #2702 describes, we want to make podman and podman-remote as
part of make install.

Fixes: #2702

Signed-off-by: baude <bbaude@redhat.com>

avoid `make` in `make install` in the rpmbuild process.

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2019-08-07 14:52:49 -04:00
0a05af1dd6 Cirrus: Add Second partition for storage testing
This is mainly/initially to support use of Cirrus-CI
in https://github.com/containers/buildah since that setup
re-uses the VM images from this project. However, it also
opens doors here, if libpod ever needs/wants to do things
with a dedicated storage device and/or storage-drivers.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-08-01 14:07:55 -04:00
3e3afb942a Cirrus: Fix release dependencies
The release-task ***must*** always execute last, in order to guarantee a
consistent cache of release archives from dependent tasks.  It
accomplishes this by verifying it's task-number matches one-less than
the total number of tasks.  Previous to this commit, a YAML anchor/alias
was used to avoid duplication of the dependency list between 'success'
and 'release'

However, it's been observed that this opens the possibility for
'release' and 'success' tasks to race when running on a PR.  Because
YAML anchor/aliases cannot be used to modify lists, duplication is
required to make 'release' actually depend upon 'success'.

This duplication will introduce an additional maintenance burden.
Though when adding a new task, it's already very easy to forget to
update the 'depends_on' list.  Assist both cases by the addition
unit-tests to verify ``.cirrus.yml`` dependency contents and structure.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-31 11:49:53 -04:00
cb2ea1a27b Cirrus: Fix re-run of release task into no-op.
This task depends upon other tasks caching their binaries.  If for
whatever reason the `release` task is re-run and/or is out-of-order
with it's dependents, the state of cache will be undefined. Previously
this would result in an error, and failing of the release task.
This commit alters this behavior to issue a warning instead.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-31 09:42:52 -04:00
07b1e331c2 Cirrus: Ubuntu: Set + Test for $RUNC_BINARY
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-25 14:02:12 -04:00
f55288c96f Cirrus: Simplify evil-unit check in image
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-25 13:51:33 -04:00
ceb3d76298 Cirrus: Silence systemd-banish noise
It's somewhat hard to predict which units are certinly present
for any given base-image.  Therefore, at image-build time, it's
distracting and unhelpful to see all the errors about units that
don't exist, on every platform.  Simply ignore them and rely on
the `check_image.sh` test to confirm none are enabled.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-25 13:51:33 -04:00
e3082762fe Cirrus: Fix image build metadata update
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-25 13:51:33 -04:00
6942d3275d Cirrus: Fix missing -n on CentOS
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-25 13:51:33 -04:00
6dc0f1bec3 Change wait to sleep in podmanimage readme
Change wait to sleep 100000 as we've had some reports of problems
with wait:  https://github.com/containers/buildah/issues/1665

Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
2019-07-22 16:23:39 -04:00
0671dca90d bump conmon to 1.0.0-rc2
Signed-off-by: Peter Hunt <pehunt@redhat.com>
2019-07-22 13:20:51 -04:00
67098029bc Cirrus: Minor scripting typo fix
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-18 10:40:26 -04:00
04a9cb01fe Merge pull request #3335 from cevich/imgprune
Cirrus: Print images that should be pruned
2019-07-17 11:47:42 +02:00
f58b754c8d Cirrus: Add image-test for locked dpkg
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-16 14:30:42 -04:00
1abb217493 Cirrus: Disable most periodic services/timers
For CI testing, it's important to remove as much variability from the
overall system as possible.  This permits focusing just on problems
closely related to code-changes.  To this end, and because VMs are very
short-lived (2 hours at most), disable all systemd services and timers
which perform periodic activities.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-16 14:30:42 -04:00
35c174f723 Cirrus: Abstract destination branch refs.
Various tasks and scripts behave differently depending on whether or not
the build is running against a PR or on a branch, post-merge.  However,
a great number of them are hard-coded to the string 'master' as the
destination.  Since this is not always the case (there are other
relevant branches), it makes sense to abstract the references with a
single definition.

Add a top-level `$DEST_BRANCH` variable to CI, and otherwise
default to 'master' when unset.  This enables running CI builds on
additional branches without the overhead of updating all the static
references to 'master'.  Simply update `$DEST_BRANCH` at the top-level
and all branch-conditional logic will function as intended.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-15 16:42:41 -04:00
e8564dc44f Cirrus: Print images that should be pruned
Over time unless they're removed, the project could grow quite a large
collection of VM images.  While generally cheap (less than a penny each,
per month), these will become a significant cost item if not kept
in-check.

Add a specialized container for handling image-pruning, but limit
it to only finding and printing (not actually deleting) images.

Also update the image-building workflow so that base-images used to
compose cache-images are also labeled with metadata.

N/B: As an additional safeguard, the service account which
     executes the new container in production *DOES NOT*
     have access to delete images.  This can be enabled
     by adding the GCE IAM role: CustomComputeImagePrune

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-15 15:26:26 -04:00
9043b816d4 Cirrus: Fix missing removal of packaged podman
This was originally intended, but somehow omitted from #1936

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-12 08:44:27 -04:00
81e722d086 Merge pull request #3106 from cevich/cirrus_release
Cirrus: Automate releasing of tested binaries
2019-07-10 15:08:03 +02:00
7633bd3b1b Merge pull request #1936 from cevich/cirrus_packaged_deps
Cirrus: Use packaged-based dependencies
2019-07-09 00:16:45 +02:00
1ef8637ae3 Cirrus: Automate releasing of tested binaries
It's desirable to make archives available of builds containing actual
tested content.  While not official distro-releases, these will enable
third-party testing, experimentation, and development for both branches
(e.g. "master") and pull requests (e.g. "pr3106").

* Add a Makefile targets for archiving both regular podman binaries
  and the remote-client.  Encode release metadata within these
  archives so that their exact source can be identified.

* Fix bug with cross-compiling remote clients for the Windows and Darwin
  platforms.

* Add unit-testing of cross-compiles for Windows and Darwin platforms.

* A few small CI-script typo-fixes

* Add a script which operates in two modes:

  1. Call Makefile targets which produce release archives.
     Upload the archive to Cirrus-CI's built-in caching system
     using reproducible cache keys.

  2. Utilize reproduced cache keys to attempt download of cache
     from each tasks.  When successful, parse the file's
     release metadata, using it to name the archive file.  Upload
     all recovered archives to a publicly accessible storage bucket
     for future reference.

* Update the main testing task to call the script in mode #1 for
  all primary platforms.

* Add a new `$SPECIALMODE` task to call the script in mode #1 for
  Windows and Darwin targets.

* Add a new 'release' task to the CI system, dependent upon all other
  tasks.  This new tasks executes the script in mode #2.

* Update CI documentation

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-03 16:51:21 -04:00
4ced285857 Bump to v1.4.5-dev
Signed-off-by: Matthew Heon <mheon@redhat.com>
2019-07-02 14:58:54 -04:00
3d559df540 Cirrus: Use packaged dependencies
Building/installing dependencies from fixed source-version ensures
testing is reliable, but introduces a maintenance burden and
risks testing far outside of a real-world environment.  The
sensible alternative is to install dependencies from distro-packaging
systems.

Install all development and testing dependencies at VM cache-image build
time, to help ensure testing remains stable.  The existing cache-image
build workflow can be utilized at any future time to build/test
with updated packages.

***N/B***: This does not update any dockerfiles used by testing, that is
left up to future efforts.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-06-27 09:05:26 -04:00
fccf4ad503 Merge pull request #3193 from cevich/check_image
Cirrus: More tests to verify cache_images
2019-06-27 03:31:35 +02:00
5f21858ae3 Bump to v1.4.4-dev
Signed-off-by: Matthew Heon <mheon@redhat.com>
2019-06-25 15:46:02 -04:00
cd78825be7 Cirrus: More tests to verify cache_images
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-06-25 15:45:12 -04:00
284208a238 Update conmon to include attach socket unlink
Signed-off-by: Peter Hunt <pehunt@redhat.com>
2019-06-21 10:55:24 -04:00
7be87f5551 Merge pull request #3078 from cevich/cirrus_f30
Cirrus F30
2019-06-19 16:13:47 +02:00
c225934726 Bump to v1.4.3-dev
Signed-off-by: Matthew Heon <mheon@redhat.com>
2019-06-18 19:27:19 -04:00
d6f436204f run BATS tests in Cirrus
I'm running the BATS tests manually once in a while, and
catching several problems each week that make it past
the rest of CI. Since the BATS tests run at RPM gating
time, we need to catch problems earlier. Try running
the tests from Cirrus.

Tests will be skipped on Ubuntu due to a too-ancient
version of coreutils (8.28; the 'timeout -v' we use
requires 8.29).

Tests are run *after* integration tests, even though
these take three minutes and would be nice to have
fail quickly, because running before causes bizarre
CI failures. Shrug.

UPDATE: also fix run test, broken by #3311.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2019-06-17 13:41:14 -06:00
058c93f850 Merge pull request #3308 from cevich/always_collect_logs
Cirrus: Simplify log collection commands
2019-06-17 21:38:25 +02:00
e0db4690aa Bump to v1.4.2-dev
Signed-off-by: Matthew Heon <mheon@redhat.com>
2019-06-14 18:19:30 -04:00
ab3105a46d Cirrus: Fix F30 ssh guarantee
The original solution using --wait does not function on F30, waiting
forever.  Replace it with a simple 5-minute timeout loop.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-06-14 15:47:07 -04:00
11484580d3 Cirrus: Add support for testing F30
Remove disused `build_cache_images` task, and
update relevant dockerfiles for F30.

Fix problem of cloud-init failing to expand root-device on boot
(/var/lib/cloud/instance left in improper state).

Fix problem of cloud-init racing with google-network-daemon.service on
boot (looking for cloudconfig metadata too early).  Causing
root-device to _sometimes_ fail to expand.

Fix problem of hack/get_ci_vm.sh argument passing.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-06-14 13:41:58 -04:00
60a857db62 Bump gate-container up to F30
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-06-14 10:27:06 -04:00
1322b41aa0 Merge pull request #3320 from cevich/fix_ssh_race
Cirrus: Guarantee ssh is running for rootless
2019-06-14 15:44:58 +02:00
4e9f5e5f2b Cirrus: Simplify log collection commands
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-06-14 08:49:18 -04:00
a3ed16416f Bump conmon to 0.3.0
This enables cgroup v2 OOM

Signed-off-by: Peter Hunt <pehunt@redhat.com>
2019-06-13 12:03:57 -04:00
8440656fd5 Cirrus: Guarantee ssh is running for rootless
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-06-13 11:32:06 -04:00