13799 Commits

Author SHA1 Message Date
33643f4b09 Merge pull request #12159 from jwhonce/issues/12115
Implement top streaming for containers and pods
2021-11-02 22:28:54 +01:00
6e6388eac4 Make stop message more similar to start
[NO TESTS NEEDED]

Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
2021-11-02 22:26:04 +01:00
82dba97547 Merge pull request #12158 from edsantiago/more_bats
System tests: enhance volume test, add debug prints
2021-11-02 18:51:24 +01:00
e63e90999c Merge pull request #12156 from matejvasek/docker-api-zero-value-fixes
Fix libpod API conformance to swagger
2021-11-02 18:19:24 +01:00
0d5aef47d3 Merge pull request #12051 from machacekondra/fix_http409_errorhandling
Handle HTTP 409 error messages properly
2021-11-02 16:11:28 +01:00
449cc7a5c2 Implement top streaming for containers and pods
* Implement API query parameter stream and delay for containers and
  pods top endpoints
* Update swagger with breaking changes
* Add python API tests for endpoints

Fixes #12115

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-11-02 08:11:14 -07:00
0686f0bb2f Merge pull request #12118 from hshiina/log-f-journald
Set flags to test 'logs -f' with journald driver
2021-11-02 13:18:26 +01:00
24e5cbd9a5 Merge pull request #12160 from matejvasek/fix-swagger-api
Fix swagger definitions
2021-11-02 12:46:25 +01:00
f2115471dd Handle HTTP 409 error messages properly for Pod actions
This PR fixes the case when the API return HTTP 409 response. Where the
API return the body format different then for other HTTP error codes.

Signed-off-by: Ondra Machacek <omachace@redhat.com>
2021-11-02 12:28:52 +01:00
d0dfc5e223 Add tests
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 23:01:17 +01:00
48d0d2b4a3 Fix swagger definitions
[NO TESTS NEEDED]

Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 22:37:35 +01:00
3147ff829b Merge pull request #12139 from cevich/add_rootless_sshkey
Cirrus: Authorize rootless user self-ssh
2021-11-01 22:23:32 +01:00
48e1cca9f8 More conforming libpod API and swagger types
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 20:25:58 +01:00
62ee24bb7c More conforming libpod API and swagger types
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 20:19:06 +01:00
a845613d03 Better emptiness test for custom JSON serializer
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 20:12:35 +01:00
7b2531c135 System tests: enhance volume test, add debug prints
Volume test: add a sequence of stat()s to confirm that volumes
are mounted as a different device than root.

Network test: add debugging code for #11825 (dnsmasq inotify
failure in bodhi only).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-11-01 13:03:05 -06:00
15eb016017 add unit test to containers_test
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-11-01 19:53:20 +02:00
120ad2d3ca Use correct swagger type in doc-comment
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 17:09:35 +01:00
9c34bd52fd Cirrus: Authorize rootless user self-ssh
Future testing needs dictate rootless (in addition to root) users are
able to ssh to localhost.  Add ssh-key generation commands for the
rootless user, and authorize their public key.

Minor: Also remove update of `/etc/sub{uid,gid}` files, since this is
now done automatically by `{user,group}add` commands.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-11-01 12:05:05 -04:00
218d91d76d Fix libpod API conformance to swagger
* Return empty array when nothing has been pruned.
* Use correct return type swagger doc-comment.

Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 16:53:49 +01:00
85bad0cc7c Merge pull request #12119 from ashley-cui/updates
[CI:DOCS] Add information on how podman machine is updated
2021-10-30 14:44:43 +00:00
7cd317e9c3 Merge pull request #12141 from kprav33n/help-doc
Fix help message case for `podman version`
2021-10-30 08:56:45 +00:00
09efcd48a8 Merge pull request #12090 from afbjorklund/image-stream
Record the image stream along with the path
2021-10-29 20:22:29 +00:00
e3c45ab9cc Merge pull request #12128 from containers/dependabot/go_modules/k8s.io/api-0.22.3
Bump k8s.io/api from 0.22.2 to 0.22.3
2021-10-29 20:21:29 +00:00
e69eae6458 Fix help message case for podman version
This is a cosmetic change. The help message for `podman version` is in
title case whereas all other command help messages are not in title
case. This stands out as inconsistent when looking at the output of
`podman help`.

Signed-off-by: Praveen Kumar <praveen+git@kumar.in>
2021-10-29 11:49:49 -07:00
1305902ff4 Merge pull request #12127 from vrothberg/bz-2014149
volumes: be more tolerant and fix infinite loop
2021-10-29 13:30:29 +00:00
584049325b Merge pull request #12138 from gsanchietti/doc_pause_typo
[CI:DOCS] Fix pause usage example
2021-10-29 13:28:28 +00:00
9fc98f265a Fix pause usage example
The page contains a wrong 'stop' command example.

Signed-off-by: Giacomo Sanchietti <giacomo.sanchietti@nethesis.it>
2021-10-29 14:24:10 +02:00
197152b02b Merge pull request #12133 from jwhonce/issues/12102
Allow label and labels when creating volumes
2021-10-29 09:36:28 +00:00
7494876000 Use systemctl in local system test
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-10-29 09:37:15 +03:00
98506c961b Allow label and labels when creating volumes
JSON payload may have either key. Labels will override any values set
via Label.

Fixes #12102

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-28 16:02:22 -07:00
f7ca045737 Merge pull request #12124 from giuseppe/allow-devpts-options
volumes: allow more options for devpts
2021-10-28 18:52:24 +00:00
d2147bada6 Merge pull request #12117 from adrianreber/2021-10-27-set-checkpointed-false-after-restore
Set Checkpointed state to false after restore
2021-10-28 18:06:25 +00:00
c5f0a5d788 volumes: be more tolerant and fix infinite loop
Make Podman more tolerant when parsing image volumes during container
creation and further fix an infinite loop when checking them.

Consider `VOLUME ['/etc/foo', '/etc/bar']` in a Containerfile.  While
it looks correct to the human eye, the single quotes are wrong and yield
the two volumes to be `[/etc/foo,` and `/etc/bar]` in Podman and Docker.

When running the container, it'll create a directory `bar]` in `/etc`
and a directory `[` in `/` with two subdirectories `etc/foo,`.  This
behavior is surprising to me but how Docker behaves.  We may improve on
that in the future.  Note that the correct way to syntax for volumes in
a Containerfile is `VOLUME /A /B /C` or `VOLUME ["/A", "/B", "/C"]`;
single quotes are not supported.

This change restores this behavior without breaking container creation
or ending up in an infinite loop.

BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2014149
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-28 16:37:33 +02:00
3c79202fb9 Add information on how podman machine is updated
Update documentation on how the default podman machine distribution,
FCOS, is updated.

Signed-off-by: Ashley Cui <acui@redhat.com>
2021-10-28 09:43:00 -04:00
4e9e6f21ff volumes: allow more options for devpts
allow to pass down more options that are supported by the kernel.

Discussion here: https://github.com/containers/toolbox/issues/568

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 15:30:06 +02:00
e6286fbace volumes: do not pass mount opt as formatter string
otherwise passing a formatter string as an option causes a weird
error message:

$ podman run --mount type=devpts,destination=/dev/pts,%sfoo ...
Error: %!s(MISSING)foo: invalid mount option

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 15:30:05 +02:00
feebf1bd81 Bump k8s.io/api from 0.22.2 to 0.22.3
Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.22.2 to 0.22.3.
- [Release notes](https://github.com/kubernetes/api/releases)
- [Commits](https://github.com/kubernetes/api/compare/v0.22.2...v0.22.3)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-28 12:27:17 +00:00
3bc449371c Merge pull request #12126 from giuseppe/fix-race-warning-message
runtime: change PID existence check
2021-10-28 12:16:24 +00:00
f16b133664 Merge pull request #12120 from giuseppe/rename-cgroup-subtree
oci: rename sub-cgroup to runtime instead of supervisor
2021-10-28 12:15:26 +00:00
960831f9c8 runtime: change PID existence check
commit 6b3b0a17c625bdf71b0ec8b783b288886d8e48d7 introduced a check for
the PID file before attempting to move the PID to a new scope.

This is still vulnerable to TOCTOU race condition though, since the
PID file or the PID can be removed/killed after the check was
successful but before it was used.

Closes: https://github.com/containers/podman/issues/12065

[NO NEW TESTS NEEDED] it fixes a CI flake

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 11:18:48 +02:00
9e5cd32056 oci: rename sub-cgroup to runtime instead of supervisor
we are having a hard time figuring out a failure in the CI:

https://github.com/containers/podman/issues/11191

Rename the sub-cgroup created here, so we can be certain the error is
caused by this part.

[NO NEW TESTS NEEDED] we need this for the CI.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 09:18:08 +02:00
0136a66a83 libpod: deduplicate ports in db
The OCICNI port format has one big problem: It does not support ranges.
So if a users forwards a range of 1k ports with podman run -p 1001-2000
we have to store each of the thousand ports individually as array element.
This bloats the db and makes the JSON encoding and decoding much slower.
In many places we already use a better port struct type which supports
ranges, e.g. `pkg/specgen` or the new network interface.

Because of this we have to do many runtime conversions between the two
port formats. If everything uses the new format we can skip the runtime
conversions.

This commit adds logic to replace all occurrences of the old format
with the new one. The database will automatically migrate the ports
to new format when the container config is read for the first time
after the update.

The `ParsePortMapping` function is `pkg/specgen/generate` has been
reworked to better work with the new format. The new logic is able
to deduplicate the given ports. This is necessary the ensure we
store them efficiently in the DB. The new code should also be more
performant than the old one.

To prove that the code is fast enough I added go benchmarks. Parsing
1 million ports took less than 0.5 seconds on my laptop.

Benchmark normalize PortMappings in specgen:
Please note that the 1 million ports are actually 20x 50k ranges
because we cannot have bigger ranges than 65535 ports.
```
$ go test -bench=. -benchmem  ./pkg/specgen/generate/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/pkg/specgen/generate
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
BenchmarkParsePortMappingNoPorts-12             480821532                2.230 ns/op           0 B/op          0 allocs/op
BenchmarkParsePortMapping1-12                      38972             30183 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMapping100-12                    18752             60688 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMapping1k-12                      3104            331719 ns/op          223840 B/op       3018 allocs/op
BenchmarkParsePortMapping10k-12                      376           3122930 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMapping1m-12                         3         390869926 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingReverse100-12             18940             63414 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMappingReverse1k-12               3015            362500 ns/op          223841 B/op       3018 allocs/op
BenchmarkParsePortMappingReverse10k-12               343           3318135 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMappingReverse1m-12                  3         403392469 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingRange1-12                 37635             28756 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange100-12               39604             28935 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1k-12                38384             29921 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange10k-12               29479             40381 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1m-12                  927           1279369 ns/op          143022 B/op        164 allocs/op
PASS
ok      github.com/containers/podman/v3/pkg/specgen/generate    25.492s
```

Benchmark convert old port format to new one:
```
go test -bench=. -benchmem  ./libpod/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/libpod
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Benchmark_ocicniPortsToNetTypesPortsNoPorts-12          663526126                1.663 ns/op           0 B/op          0 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1-12                 7858082               141.9 ns/op            72 B/op          2 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10-12                2065347               571.0 ns/op           536 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts100-12                138478              8641 ns/op            4216 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1k-12                   9414            120964 ns/op           41080 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10k-12                   781           1490526 ns/op          401528 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1m-12                      4         250579010 ns/op        40001656 B/op          4 allocs/op
PASS
ok      github.com/containers/podman/v3/libpod  11.727s
```

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-27 18:59:56 +02:00
a05a445f48 Merge pull request #12066 from matejvasek/set-docker-host
Set DOCKER_HOST in the VM
2021-10-27 15:25:21 +00:00
d908da51fb Merge pull request #12064 from vrothberg/fix-11933
container create: fix --tls-verify parsing
2021-10-27 15:24:23 +00:00
6caf5e3b7c Merge pull request #12111 from giuseppe/fix-warning-move-pause-process
runtime: check for pause pid existence
2021-10-27 15:07:59 +00:00
b29dc1bde3 Merge pull request #12110 from cevich/fix_systemd_pid1
Fix systemd PID1 test
2021-10-27 14:34:58 +00:00
e68fbf03aa Set flags to test 'logs -f' with journald driver
`logs -f` with `journald` is supported only when `journald` events
backend is used. To pass system tests using `logs -f` in an environment
where `events_logger` is not set to `journald` in `containers.conf`,
this fix sets `--events-backend` or `--log-driver` temporally.

Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2021-10-27 10:28:16 -04:00
dcbf5cae12 Set Checkpointed state to false after restore
A restored container still had the state set to 'Checkpointed: true'
which seems wrong if it running again.

[NO NEW TESTS NEEDED]

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-10-27 13:40:54 +00:00
ff31f2264d container create: fix --tls-verify parsing
Make sure that the value is only set if specified on the CLI.  c/image
already defaults to true but if set in the system context, we'd skip
settings in the registries.conf.

Fixes: #11933
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-27 14:36:25 +02:00