Specifically, Darwin's bash is very old so it doesn't support newer
features like `declare -A`. Reduce the complexity of the script
so that it can be used for all platforms. Comment heavily regarding the
scripts various execution contexts to prevent developers relying on
advanced features for any future modifications.
Signed-off-by: Chris Evich <cevich@redhat.com>
There are darwin-specific code paths which were not being linted prior
to this commit. Fix this with a new, darwin-specific section of the lint
runner script.
Signed-off-by: Chris Evich <cevich@redhat.com>
New CI validation check: all keys in quadlet.go must be
documented at least once in podman-systemd.unit.5.md.
Adding '// deprecated' next to an enum definition will
exclude said key from the documentation cross-checks.
And, because the md file lists keys in both table and block
form, make sure those all match.
And make sure everything is sorted in lexical order, in
both .go source and in man page.
And add a validation check to make sure it stays that way.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Initial impetus was #20958 (ps --format .Label abc). This is
a complicated solution to a simple-seeming problem.
The problem: .Label is a cobra *function*, something I did not
know about nor handle.
Solution: recognize cobra functions. Switch to __complete,
not __completeNoDesc, so we can see the number of arguments
required. Invent new man-page format for documenting functions.
And, finally, start enforcing how functions (and cobra structs)
are documented.
This discovered a never-used completion function, .Recycle(),
in podman-events. Remove it.
[NO NEW TESTS NEEDED] - the .go change is an excision of dead code.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Followup to #21055: regression tests for the code that
reads man pages. These are not xref-related at all, just
simple consistency checks on the man page content.
In the process of writing these tests, I also fixed a
longstanding bug where warning messages could be emitted
multiple times, once for each time we read a man page file
(as happens with command aliases).
Signed-off-by: Ed Santiago <santiago@redhat.com>
In the process of adding new functionality to the xref script,
I realized it is much too fragile. It's too easy to make some
minor change that could break the crossrefs, giving us the
illusion of testing.
Solution: add a test suite for the script. Still incomplete,
but an important step toward building confidence.
Requires minor surgery to the script itself
Signed-off-by: Ed Santiago <santiago@redhat.com>
There's a stanza in .cirrus.yml that only "runs" in
the treadmill cron job ... but that job is long gone.
The task actually runs in the buildah treadmill PR, #13808,
but that's not obvious to someone reading .cirrus.yml.
This is a maintenance burden. Remove it.
Because rootless bud tests are still important, and we
still want to run them in the treadmill PR, modify the
treadmill script itself so it (ugh) injects rootless jobs
into the buildah_bud test matrix. This is super fragile
but acceptable because I am the only one who ever runs
the treadmill script. I will notice if this breaks.
Signed-off-by: Ed Santiago <santiago@redhat.com>
I'm not sure about apparmor tag. Atleast runc isn't using it anymore.
"apparmor (since runc v1.0.0-rc93 the feature is always enabled)" from https://github.com/opencontainers/runc
containers-common still seems to check for apparmor, so not touching it for now.
Signed-off-by: Rahil Bhimjiani <rahil3108@gmail.com>
Some --filter descriptions listed the filters with asterisks,
i.e. markdown italics. There were 60+ of those, 250+ without
asterisks, so I choose to de-asterisk them all. Update the
xref script to remove the allow-asterisk exception. (Except
for the column title, which is sometimes written with two
asterisks--boldface--and sometimes plain).
Signed-off-by: Ed Santiago <santiago@redhat.com>
For all commands with a --filter option, cross-reference
against man pages, and vice-versa.
I'm sorry. I know this script has gone off the deep end.
[NO NEW TESTS NEEDED] although actually I would like to test some broken completions
Signed-off-by: Ed Santiago <santiago@redhat.com>
First do not lint pkg/domain/infra/abi with the remote tag as this is
only local code.
Then mark the cacheLibImage field as unused, this should be an unused
stub for the remote client so that we do not leak libimage.
The linter sees that with the remote tag so we need to silence that
warning.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Shortcuts like unix:path and unix:/path do not work everywhere,
so make sure to use unix://path when quoting the url (or address)
Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
The performance issue in #19467 drove me to add a benchmark for
system-df to avoid regressing on it in the future.
Comparing current HEAD to v4.6.0 yields
```
/home/vrothberg/containers/podman/bin/podman system df ran
201.47 times faster than /usr/bin/podman system df
```
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Check for duplicate subcommands, flags, and format specifiers.
I assumed this would never be necessary, that code review would
catch dups, but it happened (#19462). Prevent future ones.
Also, make it a fatal error for a --format to be undocumented,
except for 'podman inspect'. So many exceptions ... :(
Signed-off-by: Ed Santiago <santiago@redhat.com>
BATS 1.8.0 introduces tags: metadata that can be applied to
a single test or one entire file, then used for filtering
in a test run.
Issue #19299 introduces the possibility of using OpenQA
for podman reverse dependency testing: continuous CI on
all packages that can affect podman, so we don't go two
months with no bodhi builds then get caught by surprise
when systemd or kernel or crun change in ways that break us.
This PR introduces one bats tag, "distro-integration".
The intention is for OpenQA (or other) tests to install
the podman-tests package and run:
bats --filter-tags distro-integration /usr/share/podman/test/system
Goal is to keep the test list short and sweet: we do not
need to test command-line option parsing. We *DO* need to
test interactions with systemd, kernel, nethack, and other
critical components.
Signed-off-by: Ed Santiago <santiago@redhat.com>
**podman compose** is a thin wrapper around an external compose provider
such as docker-compose or podman-compose. This means that `podman
compose` is executing another tool that implements the compose
functionality but sets up the environment in a way to let the compose
provider communicate transparently with the local Podman socket. The
specified options as well the command and argument are passed directly
to the compose provider.
The default compose providers are `docker-compose` and `podman-compose`.
If installed, `docker-compose` takes precedence since it is the original
implementation of the Compose specification and is widely used on the
supported platforms (i.e., Linux, Mac OS, Windows).
If you want to change the default behavior or have a custom installation
path for your provider of choice, please change the `compose_provider`
field in `containers.conf(5)`. You may also set the
`PODMAN_COMPOSE_PROVIDER` environment variable.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The podman-login tests have accumulated much cruft over the
years, because that's the only place where we run a local
registry, and the process was crufty: we actually start/stopped
the registry as the first & last tests of the file. Meaning,
you couldn't do 'hack/bats 150:just-one-test' because that
would skip the registry start. And just now, a completely
unrelated test has had to be shoved into the login file.
This PR revamps the whole thing, by adding a new registry helper
module that can be used anywhere. And, once the registry is
started, it just stays running until the end of tests. (This
requires BATS 1.7 or greater).
Signed-off-by: Ed Santiago <santiago@redhat.com>
First: fix podman-registry script so it preserves the initial $PODMAN,
so all subsequent invocations of ps, logs, and stop will use the
same binary and arguments. Until now we've handled this by requiring
that our caller manage $PODMAN (and keep it the same), but that's
just wrong.
Next, simplify the golang interface: move the $PODMAN setting into
registry.go, instead of requiring e2e callers to set it. (This
could use some work: the local/remote conditional is icky).
IMPORTANT: To prevent registry.go from using the wrong podman binary,
the Start() call is gone. Only StartWithOptions() is valid now.
And, minor cleanup: comments, and add an actual error-message check
Reason for this PR is a recurring flake, #18355, whose multiple
failure modes I truly can't understand. I don't think this PR
is going to fix it, but this is still necessary work.
Signed-off-by: Ed Santiago <santiago@redhat.com>
hack/podman-registry --help option does not exist.
We need to use -h option when we want to see the usage message.
[NO NEW TESTS NEEDED]
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
Work around a go-md2man bug, and add a check script to make sure
this doesn't hit us again.
Background: go-md2man can't deal with a left-hand column > 31 chars.
It produces man pages that look like:
| Something With >31 Character | |
| | ..description |
(should be all on one row). It also has trouble when the vertical
bars are misaligned: it completely removes the right-hand side.
There's almost certainly a better solution: fix go-md2man, or
use a different conversion tool, or maybe even pre/postprocess.
But this is a quick interim solution.
Sorry for the perl. This could be done in bash/sed/awk/grep,
but not with any sort of sane error messages.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- treadmill script: run root & rootless in parallel, not
sequentially. It's only four jobs, and it seems dumb
to fix root tests, repush, then discover a rootless failure.
- apply-podman-deltas: implement skip_if_rootless(), and
use it to skip a nasty longstanding flake
- bud-tests-in-podman diffs: ugly code to fix a rootless hang.
background: rootless remote tests hang
cause: stray podman server process
root cause: no idea. No clue at all. I just gave up
workaround: seek out and kill stray server processes
Rootless buildah-bud tests are not run in regular CI,
only in the buildah treadmill.
Signed-off-by: Ed Santiago <santiago@redhat.com>
There are days when I really, really, really hate GNU. Remember
when someone decided that 'head -1' would no longer work, and
that it was OK to break an infinite number of legacy production
scripts? Someone now decided that egrep/fgrep are deprecated,
and our CI logs (especially pr-should-include-tests) are now
filled with hundreds of warning lines, making it difficult
to find actual errors.
I expect that those warnings will be removed quickly after
furious community backlash, just like the 'head -1' fiasco
was quietly reverted, but ITM the warnings are annoying
so I capitulate.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Porting them over to v2 requires a full rewrite.
IT is not clear who actually uses these benchmarks, Valentin who wrote
them originally is in favor of removing them. He recommends to use
script from hack/perf instead.
This commit also drop the CI integration, it is not clear who actually
uses this data. If it is needed for something please speak up.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
As long as podman uses a fork/exec model this eBPF program is able to trace the performance of each podman command and the resulting child processes from start to finish. This is an improvement to the already existing podmansnoop eBPF program which only looks at sched_process_exit and enter/exit sys_execve tracepoints.
Signed-off-by: Paul Wallrabe <54737071+raballew@users.noreply.github.com>
It flakes once or twice a day:
VERSION=1.51.1 ./hack/install_golangci.sh
Installing golangci-lint v1.51.1 into ./bin/golangci-lint
golangci/golangci-lint info checking GitHub for tag 'v1.51.1'
golangci/golangci-lint crit unable to find 'v1.51.1' - use 'latest'
or see https://github.com/golangci/golangci-lint/releases for details
No visibility into why, and no special reason to believe that
retrying five seconds later will work, but it seems worth a try.
Signed-off-by: Ed Santiago <santiago@redhat.com>
In February we started running rootless bud tests in cron (#17608).
That's nice, but nobody ever looks at cron results. The idea behind
adding a rootless task was to run it in the manual treadmill, too.
This PR enables that, and more clearly documents the how and why.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Followup to #17486: stricter checks on --format.
* If a subcommand offers autocompletion for templates,
it must also offer a '--format json' option.
* If a subcommand has a --format option that DOES NOT
offer autocompletion for templates, it must be listed
in a hardcoded grandparented-in table of commands
where that's not applicable. (Mostly commands
like build, commit, save, where "format" is used
in the context of "oci/docker").
Only likely to trigger on PRs which add new subcommands,
and is intended to catch oversights.
Also, test for alphanumeric order in man page tables.
Sort all existing tables.
Signed-off-by: Ed Santiago <santiago@redhat.com>
command tables are chock full of duplication, hence they break.
Look for inconsistencies between the displayed man page name
and the actual man page name:
| foo | [podman-foo(1)](podman-cmd-foo.1.md) | ...
^^^
Inspired by #17474.
We can't actually check the subcommand name (the plain "foo")
because there are many existing subcommands whose name does
not match the man page: rmi vs image-rm, list vs podman-ps.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Very belated successor to #14046.
I don't know why this is so important to me. Probably because we're
doing a halfhearted sloppy job of documenting, and new options get
added, and not documented, and that's just wrong.
I've given up on documenting internal structs. This iteration
has a $Format_Exceptions table defined at the top of the xref
script, enumerating a hardcoded defined set of podman commands
and fields that should remain undocumented.
This iteration also forgives completely-undocumented formats.
If podman-foo has a --format, but podman-foo.1.md does not
list *any* valid fields, the script warns but does not fail.
This at least is better than documenting a random mix of fields.
This version of the xref script is much slower: 10s vs 4. I
think we can live with that in a CI-only script.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Create the containers with a custom network. It has an impact on the
speed of container creation.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
One of the tmpfiles was not uniquely named, resulting in
conflicts with parallel 'make docs'. Fix that.
A better solution would be to run only one instance of the
preprocess script, but the Makefile incantation for that
is beyond me. This approach is CPU-wasteful but good enough.
Fixes: #17322
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a script with reproducers for bugzilla.redhat.com/show_bug.cgi?id=2162111.
The results are not much different compared to the already existing
scripts. Podman is faster despite for container removal.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Updated build scripts and installer build scripts to include gvproxy.exe.
Includes tutorial on setting up a Podman VM with QEMU and gvproxy on Windows.
Signed-off-by: Arthur Sengileyev <arthur.sengileyev@gmail.com>
Add a set of scripts using hyperfine for comparing two container
engines. I am currently using the scripts for comparing Podman
and Docker, and with older versions of Podman.
These scripts are not meant for production usage but to aid in tracking
down performance regressions and bottlenecks.
Run the scripts via `sudo sh $script.sh`.
Use the following environment variables to change the default behavior:
* `ENGINE_A` to set container engine A (default `/usr/bin/podman`)
* `ENGINE_B` to set container engine B (default `/usr/bin/docker`)
* `RUNS` to change the runs/repetitions of each benchmarks (default `100`)
* `NUM_CONTAINERS` to change the number of created containers for some benchmarks (e.g., `ps`) (default `100`)
* `IMAGE` to change the default container image (default `docker.io/library/alpine:latest`)
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Some (?) Python versions assume that text files are encoded as 7-bit ASCII and abort when encountering other encoding. Some of podman's markdown documentation files are encoded as UTF-8, and this needs to be specified explicitly when opening files.
Closes https://github.com/containers/podman/issues/16996.
[NO NEW TESTS NEEDED]
Signed-off-by: Erik Schnetter <schnetter@gmail.com>
Even though we still rely on the user to start the podman
system service, enable the option anyway. It's much, much
friendlier than requiring 'env PODMAN=etc-etc'.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The ROOT{,LESS}_ONLY logic is confusing and unmaintainable.
Change it to an easier-to-read positive check.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The man-page cross-reference script checks the SEE ALSO section
to confirm that all references are to existing man pages (#12258).
However, it's a little too forgiving: it allows aliases, the
short '.so' files under the 'links/' subdirectory. That means
we could link to non-default command names, and were doing so.
As of this PR, we no longer allow that. Any podman command
referenced in SEE ALSO must be the canonical command name
(and man page). Fix existing non-canonical names, and
remove the exception so we don't allow this again.
See #16848 for discussion of context.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This adds basic container and volume system tests for quadlet. These
install and run actual systemd units and ensure they work.
Signed-off-by: Alexander Larsson <alexl@redhat.com>
