39 Commits

Author SHA1 Message Date
84381df810 swagger: v2: libpod/images/{import,load,pull}
Note: this commit is merely adding swagger documentation and the golang
stubs and types for the proposed endpoints.  The implementation will
follow in separate individual changes in the future.

The ultimate goal is to prevent the libpod API from exposing the rather
complex /images/create endpoint from Docker and split it into easier to
implement, use and comprehend endpoints with a more narrow focus.

 # Import

Add the v2 swagger documentation for the libpod/images/import endpoint.
Note that we have intend to have separate backend and not mix it up with
load since import allows for specifying a URL instead of a local
tarball.

 # Load

Complete the v2 swagger documentation for the libpod/images/load
endpoint.  Note that we are accounting for future plans to be able to
load multiple images from one oci/docker archive by returning an array
of image-load responses.

Also move the (incomplete) implementation of the generic endpoint to the
corresponding package and create a stub for the libpod handler, which
will be implemented once there's an agreement on the proposed API.

 # Pull

Add the v2 swagger documentation for the libpod/images/pull endpoint.
Similar to the load endpoint, we return an array since more than one
image can be pulled when the `all-tags` parameter is set.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-02-04 10:45:54 +01:00
54926d1b03 Merge pull request #5061 from baude/apiv2swaggerexec
[CI:DOCS]add apiv2 endpoints for exec
2020-02-04 01:39:45 -08:00
4d301c8c4f Update /_ping support
* Add swagger support and documentation
* Promote handler as there can only ever be one
* Update swagger tags

Between verbs and paths there are four endpoints for /_ping.  I've
attempted to document them well without four copies of the details.
Time will tell.

Clients will use the headers `Libpod-API-Version` and
`Libpod-Buildha-Version` to determine if they are connecting to a Podman
service or some other compatible engine.

Client calls GET(/_ping):
 * VARLINK engines will fail with error message
 * Compatible engines will return 200 without Libpod headers
 * Podman engines will return 200 and Libpod headers

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-02-03 11:57:54 -07:00
81296cba82 [CI:DOCS]add apiv2 endpoints for exec
add the openapi/swagger documentation for exec.  The subcommands added are create, inspect, resize, and start.

at the time of this writing, no structure is declared for the inspect response.  once the libpod work for this is complete, we can inherit and swaggerize it.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-02-03 12:40:01 -06:00
f1eaccedfa fix longname handling for bindings
the api needs to account for image input where the image is encoded as a fqd image name.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-01-31 08:38:25 -06:00
616acf181d compat container names begin with /
when using the apiv2, containers names are preceeded with a /.  fixing this for inspect and listcontainers.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-01-30 13:21:40 -06:00
54587335be [CI:DOCS]Binding overhauls
Add binding for networks and begin documentation for binding methods for godoc.  Also, add major functions to their own subpackages so reduce the amount of of method confusion.  So instead of: bindings.ListImages(), we now do a [bindings].images.List().

Also, the connection is passed to each binding method via a context to allow for future growth.

Lastly, add first set of tests.  There are a couple of things to work out for rootless tests yet.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-01-28 08:42:18 -06:00
d07f611885 Merge pull request #4965 from baude/reviewcorrections3
APIv2 review corrections #3
2020-01-25 23:23:39 -08:00
5da70b04dd APIv2 review corrections #3
The third pass of corrections for the APIv2.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-01-25 15:09:09 -06:00
9a2e275abd Refactor time parsing to be more liberal in accepted values
* Added helper function to allow parsing from filters

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-01-24 10:11:05 -07:00
81e59a742b Merge pull request #4958 from jwhonce/wip/filters
[CI:DOCS] Add query parameter converters for complex types
2020-01-24 06:56:55 -08:00
9634e7eef7 Add query parameter converters for complex types
* Add converter for URL query parameters of type map[string][]string
* Add converter for URL query parameters of type time.Time
* Added function to allocate and configure schema.Decoder for API use
* Updated API handlers to leverage new converters, and correct handler
  code for filter type

An encoding example for a client using filters:

  v := map[string][]string{
      "dangling": {"true"},
  }
  payload, err := jsoniter.MarshalToString(v)
  if err != nil {
    panic(err)
  }
  payload = "?filters=" + url.QueryEscape(payload)

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-01-23 16:32:00 -07:00
cf7be58b2c Review corrections pass #2
Add API review comments to correct documentation and endpoints.  Also, add a libpode prune method to reduce code duplication.  Only used right now for the API but when the remote client is wired, we will switch over there too.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-01-23 11:58:26 -06:00
da1186ae63 Enable swagger validation for each PR
* Update swagger:operation's to pass validation
* 'name' path parameter now used throughout API
* Added #/response/ok for 200 returns, TBD values have been replaced
  with legal values.

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-01-22 15:17:14 -07:00
c40664d698 Merge pull request #4912 from jwhonce/wip/swagger
[CI:DOCS] Update build images
2020-01-22 12:53:57 -05:00
68896b18e5 Update build images
* Add swagger annotations for all the query and response parameters
  for buildimages
* Improve populating the BuildOptions struct
* Improve swagger.json generation, removing tags.xml and move tag
  definiation into the swagger:meta block
* Update Makefile to be more robust, added target for validation

* TODO once validation passes add that step to the generation step

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-01-21 14:19:42 -07:00
78e4e83e92 [CI:DOCS]First pass at review comments
Tackling the first comments in the review pass.  More to come.

Signed-off-by: baude <bbaude@redhat.com>
Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-01-21 12:50:41 -06:00
ac47e80b07 Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.

To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.

At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.

I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-16 13:49:21 -05:00
741e29caf3 Merge pull request #4860 from vrothberg/v2-top
v2 api: top improvements
2020-01-15 23:33:10 +01:00
a6ea17455c [CI:DOCS]swagger cleanup and left-hand nav
add a static tags file so we can dictate the left-hand navigation. in
doing so we now override the tag in the swagger:operation. we now have
images and images (compat) as a way to differentiate.

Signed-off-by: baude <bbaude@redhat.com>
2020-01-15 14:22:25 -06:00
974373f2b3 Merge pull request #4868 from jwhonce/wip/sshd
[CI:DOCS] Add APIv2 CLI example POC
2020-01-15 18:14:11 +01:00
89678ab0ed Add APIv2 CLI example POC
* Add ReadMe, CLI and unit files to support socket activation, both for
  system and rootless

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-01-15 09:13:45 -07:00
69bd57f518 api: stats: fix typo
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 17:08:22 +01:00
a65068aff8 api: utils: add an IsLibpodRequest handler
Add a hanlder to figure out if the specified http request came through
a libpod endpoint.  A first user is the top endpoint which has a
different default value for `ps_args` depending if the request came
through the docker or libpod endpoint.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 17:08:22 +01:00
9d54815c26 refactor top code
Move the top logic from pkg/adapter into the (*libpod.Container).Top().
This way, we drop the dependency from pkg/api on pkg/adapters and have
a clearer separation of concerns.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 17:08:22 +01:00
63d8ba64e4 v2 api: top improvements
* Use `pkg/adapter` to increase code reuse and reduce code redundancy.
* Extend swagger docs to mention AIX descriptors.
* Document the libpod endpoint which shares the same handler.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 17:08:16 +01:00
d8d42512c1 v2: stats: drop redundant sleep when streaming
Also remove the redundant stats handler in libpod.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 13:42:21 +01:00
fb5e597302 v2: stats: rigorous error checks
Also expect the container to be running.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 13:14:56 +01:00
0f6955c30e v2: stats: fix errors
Also add some comments.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 12:49:47 +01:00
3c7b7761ce v2: stats: do not ignore errors
We must check all errors and handle them properly. Otherwise, we can run
into nil dereferences ultimately killing the service.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 11:41:38 +01:00
a1ff93bdfe v2: stats: remove windows-specific fields
`NumProcs` and `StorageStats` are windows specific and are not
popoulated on Linux.  Hence, we can safely remove them.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 11:11:30 +01:00
f2f53acbec [CI:DOCS]swagger corrections
Signed-off-by: baude <bbaude@redhat.com>
2020-01-14 15:29:56 -06:00
3961882ffe Merge pull request #4843 from baude/apiv2swagger4
swagger documentation updates
2020-01-14 15:06:22 +01:00
da4b8d3c64 swagger documentation updates
adhere closer to the spec by using description and summary fields and
also ensuring that the id is unique to avoid collision between generic
and libpod endpoints.

also, make swagger output work with redoc which seems to display our
information better for our needs.

Signed-off-by: baude <bbaude@redhat.com>
2020-01-13 14:33:47 -06:00
67165b7675 make lint: enable gocritic
`gocritic` is a powerful linter that helps in preventing certain kinds
of errors as well as enforcing a coding style.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-13 14:27:02 +01:00
986feef2e8 fix e2e test failure
it is possible for layers.names to be nil and we must account for that.

Signed-off-by: baude <bbaude@redhat.com>
2020-01-10 13:06:26 -06:00
c3956b1974 address review comments before merge
Signed-off-by: baude <bbaude@redhat.com>
2020-01-10 09:47:19 -06:00
25b34972f4 [CI:DOCS]update apiv2 documentation with swagger goods
Signed-off-by: baude <bbaude@redhat.com>
2020-01-10 09:41:54 -06:00
d924494f56 Initial commit on compatible API
Signed-off-by: Jhon Honce <jhonce@redhat.com>

Create service command

Use cd cmd/service && go build .

$ systemd-socket-activate -l 8081 cmd/service/service &
$ curl http://localhost:8081/v1.24/images/json

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Correct Makefile

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Two more stragglers

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Report errors back as http headers

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Split out handlers, updated output

Output aligned to docker structures

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Refactored routing, added more endpoints and types

* Encapsulated all the routing information in the handler_* files.
* Added more serviceapi/types, including podman additions. See Info

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Cleaned up code, implemented info content

* Move Content-Type check into serviceHandler
* Custom 404 handler showing the url, mostly for debugging
* Refactored images: better method names and explicit http codes
* Added content to /info
* Added podman fields to Info struct
* Added Container struct

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Add a bunch of endpoints

containers: stop, pause, unpause, wait, rm
images: tag, rmi, create (pull only)

Signed-off-by: baude <bbaude@redhat.com>

Add even more handlers

* Add serviceapi/Error() to improve error handling
* Better support for API return payloads
* Renamed unimplemented to unsupported these are generic endpoints
  we don't intend to ever support.  Swarm broken out since it uses
  different HTTP codes to signal that the node is not in a swarm.
* Added more types
* API Version broken out so it can be validated in the future

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Refactor to introduce ServiceWriter

Signed-off-by: Jhon Honce <jhonce@redhat.com>

populate pods endpoints

/libpod/pods/..

exists, kill, pause, prune, restart, remove, start, stop, unpause

Signed-off-by: baude <bbaude@redhat.com>

Add components to Version, fix Error body

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Add images pull output, fix swarm routes

* docker-py tests/integration/api_client_test.py pass 100%
* docker-py tests/integration/api_image_test.py pass 4/16
+ Test failures include services podman does not support

Signed-off-by: Jhon Honce <jhonce@redhat.com>

pods endpoint submission 2

add create and others; only top and stats is left.

Signed-off-by: baude <bbaude@redhat.com>

Update pull image to work from empty registry

Signed-off-by: Jhon Honce <jhonce@redhat.com>

pod create and container create

first pass at pod and container create.  the container create does not
quite work yet but it is very close.  pod create needs a partial
rewrite.  also broken off the DELETE (rm/rmi) to specific handler funcs.

Signed-off-by: baude <bbaude@redhat.com>

Add docker-py demos, GET .../containers/json

* Update serviceapi/types to reflect libpod not podman
* Refactored removeImage() to provide non-streaming return

Signed-off-by: Jhon Honce <jhonce@redhat.com>

create container part2

finished minimal config needed for create container.  started demo.py
for upcoming talk

Signed-off-by: baude <bbaude@redhat.com>

Stop server after honoring request

* Remove casting for method calls
* Improve WriteResponse()
* Update Container API type to match docker API

Signed-off-by: Jhon Honce <jhonce@redhat.com>

fix namespace assumptions

cleaned up namespace issues with libpod.

Signed-off-by: baude <bbaude@redhat.com>

wip

Signed-off-by: baude <bbaude@redhat.com>

Add sliding window when shutting down server

* Added a Timeout rather than closing down service on each call
* Added gorilla/schema dependency for Decode'ing query parameters
* Improved error handling
* Container logs returned and multiplexed for stdout and stderr
  * .../containers/{name}/logs?stdout=True&stderr=True
* Container stats
  * .../containers/{name}/stats

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Improve error handling

* Add check for at least one std stream required for /containers/{id}/logs
* Add check for state in /containers/{id}/top
* Fill in more fields for /info
* Fixed error checking in service start code

Signed-off-by: Jhon Honce <jhonce@redhat.com>

get rest  of image tests for pass

Signed-off-by: baude <bbaude@redhat.com>

linting our content

Signed-off-by: baude <bbaude@redhat.com>

more linting

Signed-off-by: baude <bbaude@redhat.com>

more linting

Signed-off-by: baude <bbaude@redhat.com>

pruning

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]apiv2 pods

migrate from using args in the url to using a json struct in body for
pod create.

Signed-off-by: baude <bbaude@redhat.com>

fix handler_images prune

prune's api changed slightly to deal with filters.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]enabled base container create tests

enabling the base container create tests which allow us to get more into
the stop, kill, etc tests. many new tests now pass.

Signed-off-by: baude <bbaude@redhat.com>

serviceapi errors: append error message to API message

I dearly hope this is not breaking any other tests but debugging
"Internal Server Error" is not helpful to any user.  In case, it
breaks tests, we can rever the commit - that's why it's a small one.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

serviceAPI: add containers/prune endpoint

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

add `service` make target

Also remove the non-functional sub-Makefile.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

add make targets for testing the service

 * `sudo make run-service` for running the service.

 * `DOCKERPY_TEST="tests/integration/api_container_test.py::ListContainersTest" \
 	make run-docker-py-tests`
   for running a specific tests.  Run all tests by leaving the env
   variable empty.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

Split handlers and server packages

The files were split to help contain bloat. The api/server package will
contain all code related to the functioning of the server while
api/handlers will have all the code related to implementing the end
points.

api/server/register_* will contain the methods for registering
endpoints.  Additionally, they will have the comments for generating the
swagger spec file.

See api/handlers/version.go for a small example handler,
api/handlers/containers.go contains much more complex handlers.

Signed-off-by: Jhon Honce <jhonce@redhat.com>

[CI:DOCS]enabled more tests

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]libpod endpoints

small refactor for libpod inclusion and began adding endpoints.

Signed-off-by: baude <bbaude@redhat.com>

Implement /build and /events

* Include crypto libraries for future ssh work

Signed-off-by: Jhon Honce <jhonce@redhat.com>

[CI:DOCS]more image implementations

convert from using for to query structs among other changes including
new endpoints.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]add bindings for golang

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]add volume endpoints for libpod

create, inspect, ls, prune, and rm

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]apiv2 healthcheck enablement

wire up container healthchecks for the api.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]Add mount endpoints

via the api, allow ability to mount a container and list container
mounts.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]Add search endpoint

add search endpoint with golang bindings

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]more apiv2 development

misc population of methods, etc

Signed-off-by: baude <bbaude@redhat.com>

rebase cleanup and epoch reset

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]add more network endpoints

also, add some initial error handling and convenience functions for
standard endpoints.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]use helper funcs for bindings

use the methods developed to make writing bindings less duplicative and
easier to use.

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]add return info for prereview

begin to add return info and status codes for errors so that we can
review the apiv2

Signed-off-by: baude <bbaude@redhat.com>

[CI:DOCS]first pass at adding swagger docs for api

Signed-off-by: baude <bbaude@redhat.com>
2020-01-10 09:41:39 -06:00