278 Commits

Author SHA1 Message Date
2aeb690d37 Don't return a header name from auth.GetCredentials
Almost every caller is using it only to wrap an error
in exactly the same way, so move that error context into GetCredentials
and simplify the users.

(The one other caller, build, was even wrapping the error incorrectly
talking about query parameters; so let it use the same text as the others.)

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-12-10 18:16:18 +01:00
2130d18539 Update vendor or containers/common moving pkg/cgroups there
[NO NEW TESTS NEEDED] This is just moving pkg/cgroups out so
existing tests should be fine.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-12-07 06:17:11 -05:00
c9ad1da51c Add restart-sec option to systemd generate
Signed-off-by: Ondra Machacek <omachace@redhat.com>
2021-12-03 12:11:28 +01:00
566b78dd02 generate systemd: add --start-timeout flag
Add a new flag to set the start timeout for a generated systemd unit.
To make naming consistent, add a new --stop-timeout flag as well and let
the previous --time map to it.

Fixes: #11618
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-23 09:38:51 +01:00
b2623ceb4b api/handlers: Add checkpoint/restore FileLocks
Signed-off-by: Radostin Stoyanov <radostin@redhat.com>
2021-11-19 13:11:21 +00:00
fceecc3a5b remote checkpoint/restore: more fixes
* Support `checkpoint --pre-checkpoint`

* Support `checkpoint --with-previous`

* Disable `restore --import-previous` for the remote client since we had
  to send two files which in turn would require to tar them up and hence
  be a breaking change.  Podman 4.0 would be the chance and I hope we'll
  find time before that to remote-restore prettier.

Note that I did not run over swagger yet to check whether all parameters
are actually documented due to time constraints.

Fixes: #12334
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-18 16:52:15 +01:00
33ec8c6698 fix remote checkpoint/restore
Nothing was working before, and it's too much to summarize.  To make
sure we're not regressing in the future again, enable the remote e2e
tests.

Fixes: #12007
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-16 14:41:18 +01:00
e9d8ca22b8 Merge pull request #11076 from boaz0/closes_10275
Support template unit files in podman generate systemd
2021-11-15 16:30:38 +01:00
80e56fa12b Added optional container restore statistics
This adds the parameter '--print-stats' to 'podman container restore'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to restore a checkpoint and print out these
information. CRIU already creates process restore statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the restored container, Podman will now print
out JSON:

 # podman container restore --latest --print-stats
 {
     "podman_restore_duration": 305871,
     "container_statistics": [
         {
             "Id": "47b02e1d474b5d5fe917825e91ac653efa757c91e5a81a368d771a78f6b5ed20",
             "runtime_restore_duration": 140614,
             "criu_statistics": {
                 "forking_time": 5,
                 "restore_time": 67672,
                 "pages_restored": 14
             }
         }
     ]
 }

The output contains 'podman_restore_duration' which contains the
number of microseconds Podman required to restore the checkpoint. The
output also includes 'runtime_restore_duration' which is the time
the runtime needed to restore that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
6202e8102b Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:

 # podman container checkpoint --latest --print-stats
 {
     "podman_checkpoint_duration": 360749,
     "container_statistics": [
         {
             "Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
             "runtime_checkpoint_duration": 177222,
             "criu_statistics": {
                 "freezing_time": 100657,
                 "frozen_time": 60700,
                 "memdump_time": 8162,
                 "memwrite_time": 4224,
                 "pages_scanned": 20561,
                 "pages_written": 2129
             }
         }
     ]
 }

The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:24 +00:00
9226ccb59f Enable 'podman run --memory-swappiness=0'
'--memory-swappiness=0' used to work. This patch fixes the regression
issue, which was caused by the change of infra container creation
process.

Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2021-11-12 13:20:20 -05:00
449cc7a5c2 Implement top streaming for containers and pods
* Implement API query parameter stream and delay for containers and
  pods top endpoints
* Update swagger with breaking changes
* Add python API tests for endpoints

Fixes #12115

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-11-02 08:11:14 -07:00
98506c961b Allow label and labels when creating volumes
JSON payload may have either key. Labels will override any values set
via Label.

Fixes #12102

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-28 16:02:22 -07:00
75f478c08b pod create: remove need for pause image
So far, the infra containers of pods required pulling down an image
rendering pods not usable in disconnected environments.  Instead, build
an image locally which uses local pause binary.

Fixes: #10354
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-26 13:51:45 +02:00
acd8b49000 Add support to play kube for --log-opt
Fixes: https://github.com/containers/podman/issues/11727

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-25 11:40:42 -04:00
ece0c7e5d3 Support template unit files in podman generate systemd
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-10-22 04:19:18 +03:00
fad14dafe1 faster image inspection
Vendor the latest HEAD in c/common to pull in changes for a faster
inspection of images.  Previously, only the size computation was
optional, now the one for the parent image is as well.

In many cases, the parent image is not needed but it takes around 10ms
on my local machine.  With this change, we cut off 10ms from many code
paths, most importantly, container creation.

[NO NEW TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-08 14:47:33 +02:00
21c9dc3c40 Add --time out for podman * rm -f commands
Add --time flag to podman container rm
Add --time flag to podman pod rm
Add --time flag to podman volume rm
Add --time flag to podman network rm

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-04 07:07:56 -04:00
641f0ccc4e Add podman play kube --no-hosts options
This option will setup the containers to not modify their /etc/hosts
file and just use the one from the image.

Fixes: https://github.com/containers/podman/issues/9500

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-01 10:53:50 -04:00
a9a54eefab image prune: support removing external containers
Support removing external containers (e.g., build containers) during
image prune.

Fixes: #11472
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-09-28 10:24:16 +02:00
01bf8a6687 podman save: enforce signature removal
Enforce the removal of signatures in `podman save` to restore behavior
prior to the migration to libimage.  We may consider improving on that
in the future.  For details, please refer to the excellent summary by
@mtrmac [1].

[NO TESTS NEEDED] - manually verified but exisiting tests need some
further investigation (see [1]).

[1] https://github.com/containers/podman/pull/11669#issuecomment-925250264

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-09-23 13:50:49 +02:00
1c4e6d8624 standardize logrus messages to upper case
Remove ERROR: Error stutter from logrus messages also.

[ NO TESTS NEEDED] This is just code cleanup.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-09-22 15:29:34 -04:00
49c5688a30 podman save: add --uncompressed
Add an option to `podman save` to allow uncompressed layers when
copying OCI images.  Do the neccessary plumbing for the remote client,
add tests and vendor in the latest commit from c/common to fetch
the neccessary changes in libimage.

Closes: #11613
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-09-22 12:38:07 +02:00
8fac34b8ff Pod Device Support
added support for pod devices. The device gets added to the infra container and
recreated in all containers that join the pod.

This required a new container config item to keep track of the original device passed in by the user before
the path was parsed into the container device.

Signed-off-by: cdoern <cdoern@redhat.com>
2021-09-20 23:22:43 -04:00
5c7935057c Do not allow network modes to be used as network names
`podman network create` should not allow users to create networks with a
name which is already used for a network mode in `podman run --network`.

Fixes #11448

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-09-16 11:01:52 +02:00
85e8fbf7f3 Wire network interface into libpod
Make use of the new network interface in libpod.

This commit contains several breaking changes:
- podman network create only outputs the new network name and not file
  path.
- podman network ls shows the network driver instead of the cni version
  and plugins.
- podman network inspect outputs the new network struct and not the cni
  conflist.
- The bindings and libpod api endpoints have been changed to use the new
  network structure.

The container network status is stored in a new field in the state. The
status should be received with the new `c.getNetworkStatus`. This will
migrate the old status to the new format. Therefore old containers should
contine to work correctly in all cases even when network connect/
disconnect is used.

New features:
- podman network reload keeps the ip and mac for more than one network.
- podman container restore keeps the ip and mac for more than one
  network.
- The network create compat endpoint can now use more than one ipam
  config.

The man pages and the swagger doc are updated to reflect the latest
changes.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-09-15 20:00:20 +02:00
4b6ffda31c Merge pull request #11409 from cdoern/podVolumes
Pod Volumes Support
2021-09-15 09:10:12 -04:00
9b04e17893 api: handle nil pointer dereference in rest endpoints
When `?all=garbage` is passed to an API endpoint schema validation fails
and err is nil. Wrapf uses err to create an error message causing a nil
pointer dereference.

Signed-off-by: Jelle van der Waa <jvanderwaa@redhat.com>
2021-09-15 09:14:34 +02:00
84005330aa Pod Volumes Support
added support for the --volume flag in pods using the new infra container design.
users can specify all volume options they can with regular containers

resolves #10379

Signed-off-by: cdoern <cdoern@redhat.com>
2021-09-14 08:32:07 -04:00
deaf969243 Refacter API server emphasis on logging
* To aid in debugging log API request and response bodies at trace
  level. Events can be correlated using the X-Reference-Id.
* Server now echos X-Reference-Id from client if set, otherwise
  generates an unique id.
* Move logic for X-Reference-Id into middleware
* Change uses of Header.Add() to Set() when setting Content-Type
* Log API operations in Apache format using gorilla middleware
* Port server code to use BaseContext and ConnContext

Fixes #10053

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-09-10 15:07:25 -07:00
d1573b95e3 generate systemd: handle --restart
Handle custom restart policies of containers when generating the unit
files; those should be set on the unit level and removed from ExecStart
flags.

Fixes: #11438
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-09-07 13:58:11 +02:00
d28e85741f InfraContainer Rework
InfraContainer should go through the same creation process as regular containers. This change was from the cmd level
down, involving new container CLI opts and specgen creating functions. What now happens is that both container and pod
cli options are populated in cmd and used to create a podSpecgen and a containerSpecgen. The process then goes as follows

FillOutSpecGen (infra) -> MapSpec (podOpts -> infraOpts) -> PodCreate -> MakePod -> createPodOptions -> NewPod -> CompleteSpec (infra) -> MakeContainer -> NewContainer -> newContainer -> AddInfra (to pod state)

Signed-off-by: cdoern <cdoern@redhat.com>
2021-08-26 16:05:16 -04:00
1e176923b1 teardown play kube
add the ability for play kube to tear down based on the yaml used to
play it.  it is indicated by --down in the play kube command.  volumes
are NOT deleted during the teardown.  pods and their containers are
stopped and removed.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-24 14:26:14 -05:00
8c228bdbd0 Merge pull request #11154 from cdoern/imagesPull
Libpod images pull changes
2021-08-16 15:46:33 +00:00
221b1add74 Add support for pod inside of user namespace.
Add the --userns flag to podman pod create and keep
track of the userns setting that pod was created with
so that all containers created within the pod will inherit
that userns setting.

Specifically we need to be able to launch a pod with
--userns=keep-id

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-08-09 15:17:22 -04:00
12ac4198ca Libpod images pull changes
Added quiet param to docs to limit stream output. Formatted JSON.

fixes #10612

Signed-off-by: cdoern <cbdoer23@g.holycross.edu>
Signed-off-by: cdoern <cdoern@redhat.com>
2021-08-09 14:17:08 -04:00
3a922cbc99 Merge pull request #11003 from pascomnet/f_stats
stats: add a interval parameter to cli and api stats streaming
2021-08-04 09:56:57 +00:00
9cc974c963 Only support containers stats using cgroups v2
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1988252

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-08-03 15:31:59 -07:00
cdbbd79155 stats: add a interval parameter to cli and api stream mode
podman stats polled by default in a 1 sec period.
This can put quite some load on a machine if you run many containers.

The default value is now 5 seconds.
You can change this interval with a new, optional, --interval, -i cli flag.
The api request got also a interval query parameter for the same purpose.

Additionally a unused const was removed.
Api and cli will fail the request if a 0 or negative value is passed in.

Signed-off-by: Thomas Weber <towe75@googlemail.com>
2021-07-27 07:20:47 +02:00
db2f474284 Drop podman create --storage-opt container flag
The global flag will work in either location, and this flag just breaks
users expectations, and is basically a noop.

Also fix global storage-opt so that podman-remote can use it.

[NO TESTS NEEDED] Since it would be difficult to test in ci/cd.

Fixes: https://github.com/containers/podman/issues/10264

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-07-20 16:36:10 -04:00
e1ac0c3033 vendor containers/common@main
The `IgnorePlatform` options has been removed from the
`LookupImageOptions` in libimage to properly support multi-arch images.

Skip one buildah-bud test which requires updated CI images.  This is
currently being done in github.com/containers/podman/pull/10829 but
we need to unblock merging common and buildah into podman.

[NO TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-16 14:19:56 +02:00
e7507fe7cf make DriverOpts name consistent.
Signed-off-by: Tino Rusch <tino.rusch@gmail.com>
2021-06-25 08:33:15 +02:00
b56b4b5374 read secret config from config file if no user data.
feat: read secret config from config file if the user hasn't entered
explicit config values

feat: allow to specify `--driver-opts opt1=val1,opt2=val2` in the secret
create command to allow overriding the default values

fix: show driver options in `podman secret inspect`

Signed-off-by: Tino Rusch <tino.rusch@gmail.com>
2021-06-24 12:31:14 +02:00
5fc622f945 create: support images with invalid platform
Much to my regret, there is a number of images in the wild with invalid
platforms breaking the platform checks in libimage that want to make
sure that a local image is matching the expected platform.

Imagine a `podman run --arch=arm64 fedora` with a local amd64 fedora
image.  We really shouldn't use the local one in this case and pull down
the arm64 one.

The strict platform checks in libimage in combination with invalid
platforms in images surfaced in Podman being able to pull an image but
failing to look it up in subsequent presence checks.  A `podman run`
would hence pull such an image but fail to create the container.

Support images with invalid platforms by vendoring the latest HEAD from
containers/common.  Also remove the partially implemented pull-policy
logic from Podman and let libimage handle that entirely.  However,
whenever --arch, --os or --platform are specified, the pull policy will
be forced to "newer".  This way, we pessimistically assume that the
local image has an invalid platform and we reach out to the registry.
If there's a newer image (i.e., one with a different digest), we'll pull
it down.

Please note that most of the logic has either already been implemented
in libimage or been moved down which allows for removing some clutter
from Podman.

[NO TESTS NEEDED] since c/common has new tests.  Podman can rely on the
existing tests.

Fixes: #10648
Fixes: #10682
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-06-23 15:42:13 +02:00
8e89d70713 [CI:DOCS] Update swagger for inspect network
struct for swagger was pointing to wrong internal type

Fixes #10559

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-06-08 15:23:46 -07:00
3d961acc22 Merge pull request #10597 from Luap99/prune
Fix network prune api docs
2021-06-08 17:48:01 +02:00
c605482791 remote pull: cancel pull when connection is closed
If a client closes the http connection during image pull, the
service should cancel the pull operation.

[NO TESTS NEEDED] I have no idea how we could test this reliable.

Fixes: #7558

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-08 13:36:07 +02:00
8378a9c4df Fix network prune api docs
The api doc used wrong response examples for both the compat and libpod
network prune endpoints. Change the doc so that it matches the actual
return values. Also fix the endpoints to return an empty array instead
of null when no networks are removed.

[NO TESTS NEEDED]

Fixes: #10564

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-08 11:38:26 +02:00
df7c3a7039 [CI:DOCS] fix incorrect network remove api doc
The endpoint returns an array and not a single entry.

Fixes #10494

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-06-04 14:22:52 +02:00
fb4a0c572e support tag@digest notation
Vendor in the latest HEAd of containers/common to implicitly support the
tag@digest notation for images.  To remain compatible with Docker, the
tag will be stripped off the image reference and is entirely ignored.

Fixes: #6721
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-05-31 14:38:43 +02:00