55 Commits

Author SHA1 Message Date
bd09b7aa79 bump go module to version 4
Automated for .go files via gomove [1]:
`gomove github.com/containers/podman/v3 github.com/containers/podman/v4`

Remaining files via vgrep [2]:
`vgrep github.com/containers/podman/v3`

[1] https://github.com/KSubedi/gomove
[2] https://github.com/vrothberg/vgrep

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2022-01-18 12:47:07 +01:00
8a7e70919f Refactor manifest list operations
* Update method/function signatures use the manifest list name and
  images associated with the operation explicitly, in general

  func f(ctx context.Context, manifestListName string,
         ImageNames []string, options *fOptions)

* Leverage gorilla/mux Subrouters to support API v3.x and v4.x for
  manifests
* Make manifest API endpoints more RESTful
* Add PUT /manifest/{id} to update existing manifests
* Add manifests.Annotate to go bindings, uncommented unit test
* Add DELETE /manifest/{Id} to remove existing manifest list, use
  PUT /manifest/{id} to remove images from a list
* Deprecated POST /manifest/{id}/add and /manifest/{id}/remove, use
  PUT /manifest/{id} instead
* Corrected swagger godoc and updated to cover API changes
* Update podman manifest commands to use registry.Context()
* Expose utils.GetVar() to obtain query parameters by name
* Unexpose server.registerSwaggerHandlers, not sure why this was ever
  exposed.
* Refactored code to use http.Header instead of map[string]string when
  operating on HTTP headers.
* Add API-Version header support in bindings to allow calling explicate
  versions of the API. Header is _NOT_ forwarded to the API service.

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2022-01-14 16:13:35 -07:00
0459484bdf Fix CI flake on time of shutdown for API service
* Increase timeout for tests to 10s
* To aid in debugging add PID to shutdown package logging
* Added new message for forced service shutdown
* Always wait for HTTP server to shutdown, duration of 0 not friendly
  to clients

Note: The log event

"IdleTracker: StateClosed transition by connection marked un-managed"

denotes a TCP connection has been initiated but no HTTP request was sent.
And is expected during these tests.

Fixes #11921

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-12 09:53:19 -07:00
8d3aec9d08 Enable /debug/pprof API service endpoints
* Refactor sidecar HTTP service for /debug/pprof endpoints to use a TCP
  address given via new podman system service --pprof-address flag

* Allow same URL parsing in "system service" as bindings/connection.go

* Refactor NewServerWithSettings() to use entities.ServiceOptions
  in place of deleted server.Options

* Updated godoc for impacted functions and types

* Fixed API service Shutdown() to do an orderly shutdown when
  terminated and running with --time=0

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-08 13:57:20 -07:00
1c4e6d8624 standardize logrus messages to upper case
Remove ERROR: Error stutter from logrus messages also.

[ NO TESTS NEEDED] This is just code cleanup.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-09-22 15:29:34 -04:00
deaf969243 Refacter API server emphasis on logging
* To aid in debugging log API request and response bodies at trace
  level. Events can be correlated using the X-Reference-Id.
* Server now echos X-Reference-Id from client if set, otherwise
  generates an unique id.
* Move logic for X-Reference-Id into middleware
* Change uses of Header.Add() to Set() when setting Content-Type
* Log API operations in Apache format using gorilla middleware
* Port server code to use BaseContext and ConnContext

Fixes #10053

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-09-10 15:07:25 -07:00
872c442e67 Remove ReadHeaderTimeout
Effectively sets timeout to infinity.
This is needed in order to make `podman` work with `pack`.
The `pack` CLI is keeping one connection for prolonged time.
Closing the connection breaks `pack`'s functionality.

[NO TESTS NEEDED]

Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-07-29 14:07:47 +02:00
2810c478a7 Add CORS support
[NO TESTS NEEDED]

Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-06-04 16:14:52 +03:00
7bcfae44b9 Downgrade API service routing table logging
[NO TESTS NEEDED]

* Log the routing table output at Trace vs. Debug level. Reduce noise
  in debugging output.
* Tweak SDNotify message to report Warn when it fails. Previously
  failures were silent.

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-05-20 09:10:45 -07:00
5dded6fae7 bump go module to v3
We missed bumping the go module, so let's do it now :)

* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-02-22 09:03:51 +01:00
832a69b0be Implement Secrets
Implement podman secret create, inspect, ls, rm
Implement podman run/create --secret
Secrets are blobs of data that are sensitive.
Currently, the only secret driver supported is filedriver, which means creating a secret stores it in base64 unencrypted in a file.
After creating a secret, a user can use the --secret flag to expose the secret inside the container at /run/secrets/[secretname]
This secret will not be commited to an image on a podman commit

Signed-off-by: Ashley Cui <acui@redhat.com>
2021-02-09 09:13:21 -05:00
5350254f05 Ensure shutdown handler access is syncronized
There was a potential race where two handlers could be added at
the same time. Go Maps are not thread-safe, so that could do
unpleasant things. Add a mutex to keep things safe.

Also, swap the order or Register and Start for the handlers in
Libpod runtime created. As written, there was a small gap between
Start and Register where SIGTERM/SIGINT would be completely
ignored, instead of stopping Podman. Swapping the two closes this
gap.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-01-25 15:44:42 -05:00
4fa1fce930 Spelling
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-12-22 13:34:31 -05:00
d8795a36b2 Make podman service log events
* Log endpoint calls at level Info
* Ensure API server started at level Info

Fixes #8390

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-11-19 14:42:56 -07:00
f58d2f5e75 Enforce LIFO ordering for shutdown handlers
This allows us to run both the Libpod and Server handlers at the
same time without unregistering one.

Also, pass the signal that killed us into the handlers, in case
they want to use it to determine what to do (e.g. what exit code
to set).

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-10-13 14:21:19 -04:00
83e6e4ccdd Enable masking stop signals within container creation
Expand the use of the Shutdown package such that we now use it
to handle signals any time we run Libpod. From there, add code to
container creation to use the Inhibit function to prevent a
shutdown from occuring during the critical parts of container
creation.

We also need to turn off signal handling when --sig-proxy is
invoked - we don't want to catch the signals ourselves then, but
instead to forward them into the container via the existing
sig-proxy handler.

Fixes #7941

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-10-12 17:08:26 -04:00
8381f3feee Add a shutdown handler package
We need a unified package for handling signals that shut down
Libpod and Podman. We need to be able to do different things on
receiving such a signal (`system service` wants to shut down the
service gracefully, while most other commands just want to exit)
and we need to be able to inhibit this shutdown signal while we
are waiting for some critical operations (e.g. creating a
container) to finish. This takes the first step by defining the
package that will handle this.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-10-12 14:32:27 -04:00
2fc50f8b13 Fixes remote attach and exec to signal IdleTracker
- Fixes issue where remote attach and exec only signaled the IdleTracker
   on errors. Needs to done anytime after connection has been hijacked
 - Fixes trying to send multiple http status codes to client
 - Changes pprof and API server shutdowns to run in parallel
 - Changes shutdown to run in sync.Once block

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-10-05 10:40:23 -07:00
f03d470349 Refactor IdleTracker to handle StateIdle transitions
* Remove stutter naming for package and types
* Stop treating StateIdle the same as StateClosed, rather transitions to
  StateIdle will keep API timeout window open
* Remove redundate code

Fixes #7826

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-09-29 11:32:47 -07:00
0f4e2be073 podman.service: use sdnotiy
Commit 2b6dd3fb4384 set the killmode of the podman.service to the
systemd default which ultimately lead to the problem that systemd
will kill *all* processes inside the unit's cgroup and hence kill
all containers whenever the service is stopped.

Fix it by setting the type to sdnotify and the killmode to process.
`podman system service` will send the necessary notify messages
when the NOTIFY_SOCKET is set and unset it right after to prevent
the backend and container runtimes from jumping in between and send
messages as well.

Fixes: #7294
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-08-13 17:53:59 +02:00
a5e37ad280 Switch all references to github.com/containers/libpod -> podman
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-07-28 08:23:45 -04:00
324a02ec60 Fix: Correct connection counters for hijacked connections
This patch fixes connection counters for v2 endpoints

Idletracker was moved to a new package to prevent package cycle.
Hijacking code still remains in wrong place and should be moved
later to isolated package

Signed-off-by: Sami Korhonen <skorhone@gmail.com>
2020-07-09 22:17:27 +03:00
cf64e3cad8 Remove hijacked connections from active connections list
StateHijacked is a terminal state. If hijacked connection
is registered as an active connection, connection will
never be unregistered. This causes two issues

First issue is that active connection counters are off.

Second issue is a resource leak caused by connection
object that is stored to a map.

After this patch hijacked connections are no longer
visible in counters. If a counter for hijacked
connections is required, podman must track
connections returned by Hijacker.Hijack()

It might make sense to develop abstraction layer for
hijacking - and move all hijacking related code to a
separate package. Hijacking code is prone to resource
leaks and it should be thoroughly tested.

Signed-off-by: Sami Korhonen <skorhone@gmail.com>
2020-07-09 22:16:07 +03:00
8489dc4345 move go module to v2
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules.  While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.

Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`.  The renaming of the imports
was done via `gomove` [1].

[1] https://github.com/KSubedi/gomove

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-07-06 15:50:12 +02:00
f0ca17650e Ensure umask is set appropriately for 'system service'
We need a umask of 0022 to ensure containers are created
correctly, but we set a different one prior to starting the
server (to ensure the unix socket has the right permissions).
Thus, we need to set the umask after the socket has been bound,
but before the server begins accepting requests.

Fixes #6787

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-26 10:07:20 -04:00
4bb43b898d Fixup issues found by golint
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-06-10 05:49:41 -04:00
5dfddf04ee v2 copy endpoints
add copy endpoint inputs and outputs.  these endpoints are not implemented yet, nor are any bindings.  this allows us to update this later without having to change our api version.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-05-26 12:04:20 -05:00
6f5bc712de Add MethodNotAllowedHandler() to add in debugging
Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-05-22 11:48:13 -07:00
cf502b99fe enable podman v2 networking for remote client
Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-05-12 13:23:43 -05:00
f269be3a31 add {generate,play} kube
Add the `podman generate kube` and `podman play kube` command.  The code
has largely been copied from Podman v1 but restructured to not leak the
K8s core API into the (remote) client.

Both commands are added in the same commit to allow for enabling the
tests at the same time.

Move some exports from `cmd/podman/common` to the appropriate places in
the backend to avoid circular dependencies.

Move definitions of label annotations to `libpod/define` and set the
security-opt labels in the frontend to make kube tests pass.

Implement rest endpoints, bindings and the tunnel interface.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-05-06 17:08:22 +02:00
d7a27b8e84 Instrumentation to answer #5765
* currently wired to localhost:8888 to prevent access from off machine

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-22 11:07:28 -07:00
4e0326241b V2 Fix support for tcp://[::]<port> connections
* Fix support for socket activation, on remote and service

$ systemd-socket-activate -l 8083 --fdname=podman bin/podman system service --log-level=debug --time=30
$ bin/podman-remote --remote=tcp://[::]:8083 image ls

Or, use the podman.{socket,service} unit files

$ bin/podman-remote --remote=unix:///run/podman/podman.sock image ls

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-17 16:39:59 -07:00
f8892e7c6b Refactor service idle support
* Move connection tracking into APIServer using ConnState()
* Remove Connection counters from CLI code
* Update events handler to support client not closing connection
* Improve logging messages

Fixes #5599

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-13 08:11:36 -07:00
e0847f5457 V2 podman system service
* Added support for system service
* Enabled linting on the varlinkapi source, needed to support V2
  service command
* Added support for PODMAN_SOCKET

Skip linting deprecated code

Rather than introduce bugs by correcting deprecated code, linting the
code is being skipped. Code that is being ported into V2 is being
checked.

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-07 19:22:10 -07:00
464a1cd485 Merge pull request #5540 from jwhonce/issues/5531
Reduce CPU usage when --timeout=0
2020-03-18 21:55:09 +01:00
52c8350530 serve swagger when present
register the swagger endpoint and add some error handling for when the swagger file does not exist

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-18 14:34:14 -05:00
651ddd3560 Reduce CPU usage when --timeout=0
* Add second go routine for when a Timer is not needed.
* goimports updated some project files

Fixes #5531

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-03-18 10:27:25 -07:00
abbbeacd68 apiv2 addition of manifests
add endpoints for create, add, remove, inspect, and push.  this allows manifests to be managed through the restful interfaces.

also added go-bindings and tests

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-16 10:03:55 -05:00
450361fc64 update systemd & dbus dependencies
Update the outdated systemd and dbus dependencies which are now provided
as go modules.  This will further tighten our dependencies and releases
and pave the way for the upcoming auto-update feature.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-03-10 18:34:55 +01:00
997e3c8863 Remove 1 sec delay
* Stop closing net.Listener() twice on interrupt
 * Do not report error if closing server twice

Fixes #5311

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-02-25 11:04:20 -07:00
c0c44ae8a3 Fix handler and systemd activation errors
On panic from handler: log warning and stack trace, report
InternalServerError to client

When using `podman system service` make determining the listening endpoint deterministic.

  // When determining _*THE*_ listening endpoint --
  // 1) User input wins always
  // 2) systemd socket activation
  // 3) rootless honors XDG_RUNTIME_DIR
  // 4) if varlink -- adapter.DefaultVarlinkAddress
  // 5) lastly adapter.DefaultAPIAddress

Fixes #5150
Fixes #5151

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-02-17 10:29:32 -07:00
1b290f995b Merge pull request #5074 from jwhonce/wip/swagger_endpoint
Add /swagger/ endpoint to serve swagger yaml to clients
2020-02-14 00:32:26 +01:00
5b830cca90 apiv2 stream events
the events endpoint should be stream-based.  it also needed to be registered to answer and not produce 404s.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-02-11 15:00:41 -06:00
4bb55161e9 Add /swagger/ endpoint to serve swagger yaml to clients
The provided yaml file will describe the current Podman REST API.

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-02-04 16:39:54 -07:00
81296cba82 [CI:DOCS]add apiv2 endpoints for exec
add the openapi/swagger documentation for exec.  The subcommands added are create, inspect, resize, and start.

at the time of this writing, no structure is declared for the inspect response.  once the libpod work for this is complete, we can inherit and swaggerize it.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-02-03 12:40:01 -06:00
f1eaccedfa fix longname handling for bindings
the api needs to account for image input where the image is encoded as a fqd image name.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-01-31 08:38:25 -06:00
9634e7eef7 Add query parameter converters for complex types
* Add converter for URL query parameters of type map[string][]string
* Add converter for URL query parameters of type time.Time
* Added function to allocate and configure schema.Decoder for API use
* Updated API handlers to leverage new converters, and correct handler
  code for filter type

An encoding example for a client using filters:

  v := map[string][]string{
      "dangling": {"true"},
  }
  payload, err := jsoniter.MarshalToString(v)
  if err != nil {
    panic(err)
  }
  payload = "?filters=" + url.QueryEscape(payload)

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-01-23 16:32:00 -07:00
da1186ae63 Enable swagger validation for each PR
* Update swagger:operation's to pass validation
* 'name' path parameter now used throughout API
* Added #/response/ok for 200 returns, TBD values have been replaced
  with legal values.

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-01-22 15:17:14 -07:00
c40664d698 Merge pull request #4912 from jwhonce/wip/swagger
[CI:DOCS] Update build images
2020-01-22 12:53:57 -05:00
d4c2aaf38a Add service endpoint
add service endpoint for the new API.  Also supports the varlink
implementation.

Signed-off-by: baude <bbaude@redhat.com>

Refactor to allow developer more control of API server

* Add api.NewServerWithSettings() to create an API server with custom
  settings
* Add api.ListenUnix() to create a UDS net.Listener and setup UDS

Signed-off-by: Jhon Honce <jhonce@redhat.com>
Signed-off-by: baude <bbaude@redhat.com>

More service completion

Add podman service command that allows users to run either a RESTful or
varlink protocol API service.

Addition of docs and RESTful listening.

Signed-off-by: baude <bbaude@redhat.com>
Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-01-21 16:35:45 -06:00