Clarify that system test specifc configuration must be done via config
files and not via ad hoc environment variables like in test/e2e.
Also not that we only run the tests with crun so other runtimes may not
work but we accept patches to make them work with runc, e.g. Suse folks
currently run them with runc and contribute patches for them.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
As discussed in Aug 13 Cabal, we are almost at a point where
e2e tests are reliably passing on the first try. Let's try to
keep things that way, and not hide future flakes.
Closes: #17967
Signed-off-by: Ed Santiago <santiago@redhat.com>
First, point users to hack/bats for running them locally. Second, remove
TODO.md as it doesn't contain any helpful information. Basically all the
missing tests there have been added so this does not serve any purpose
and is missleading.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
...and an optional error-message string, to be checked
against stderr.
This is a starting point and baby-steps progress toward #18188.
There are 249 ExitWithError() checks in test/e2e. It will take
weeks to fix them all. This commit enables new functionality:
Expect(ExitWithError(125, "expected substring"))
...while also allowing the current empty-args form. Once
all 249 empty-args uses are modernized, the matcher code
will be cleaned up.
I expect it will take several months of light effort to get
all e2e tests transitioned to the new form. I am choosing to
do so in pieces, for (relative) ease of review. This PR:
1) makes the initial changes described above; and
2) updates a small subset of e2e _test.go files such that:
a) ExitWithError() is given an exit code and error string; and
b) Exit(Nonzero) is changed to ExitWithError(Nonzero, "string")
(when possible)
Signed-off-by: Ed Santiago <santiago@redhat.com>
Followup to #20318: now that sqlite is the podman default,
enforce that in CI as well. Test boltdb only in Prior Fedora.
In the process, discovered & cleaned up some duplication
and unused YAML anchors.
Signed-off-by: Ed Santiago <santiago@redhat.com>
When you run e2e tests locally they use CNI unless the NETWORK_BACKEND
env was set to netavark. Because our main focus is on netavark we should
test it by default.
For local tests this should help to prevent CNI/netavark conflicts as I
assume most systems where people run tests on are on netavark by now.
For CI testing we hardcode NETWORK_BACKEND there to test both netavark
(on current fedora) and CNI (prior fedora). MAke sure to switch the
logic in the CI setup to reflect that.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When running ginkgo tests locally we often only want to test a small
subset. I think most people just add the `FIt` block but then you need
to remember to undo that before pushing the changes.
With this change you can just run:
```
make localintegration FOCUS="test name here"
make localintegration FOCUS_FILE="some_test.go"
```
I updated the test Readme to use this new syntax.
The options just map to the ginkgo options, see the upstream docs
linked in the readme for more information about syntax.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
- document env vars that can be used
- list up to date dependencies
- remove unnecessary GOPATH mention, no longer needed with gomodules
- use make targets to tests everything (much faster due `-p` option)
- remove tests in container section as make shell is not a valid target
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Basically, acknowledge the need for dependencies and link
to Fedora specfile which is the only sane place to find
such a list.
Closes: #16365
Signed-off-by: Ed Santiago <santiago@redhat.com>
Followup to #10932: add a validation check to prevent introduction
of new 'Expect(foo.ExitCode()).To(...)' patterns. If such use is
absolutely necessary -- there is one such instance in the code
already -- require that the assertion include a description.
Also: clean up instances that were introduced since the merging
of #10932.
Also: fix one remaining instance in run_exit_test.go: it had
a FIXME comment mentioning a race condition, but unfortunately
there was no issue or bug ID, hence no way to know if the race
is fixed or not. We will assume it is.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Update the CNI configuration instructions to line up with
the changes introduced in #3868. Also do a bit less documentation
of the configuration and point to the GitHub project so we won't
get out of sync in the future.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Add e2e/test/common_test.go to the single integration test
instructions. Without it the documented process fails.
* Make sure that all vendored dependencies are in sync with the code and
the vendor.conf by running `make vendor` with a follow-up status check
of the git tree.
* Vendor ginkgo and gomega to include the test dependencies.
Signed-off-by: Chris Evic <cevich@redhat.com>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The tests can be filter by --focus and --skip to fit different test
target. Also be able to set global options and cmd options by export
it to ENV to fit different test matrix.
Signed-off-by: Yiqiao Pu <ypu@redhat.com>
Some of the paths in the e2e tests are hard-coded, which complicates
testing a bit on systems with different paths for runc, conmon, etc.
Add a make shell target to the Makefile, which will build and run the
libpod containers, giving a shell to the user in which podman can be
built, run, tested etc.
Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
Closes: #481
Approved by: rhatdan
Completion of the migration from bats to ginkgo. This includes:
* load
* mount
* pause
* port
* run_networking
* search
Note: build will be done within a different PR
Signed-off-by: baude <bbaude@redhat.com>
