Previously, Make would execute these shell commands even if we didn't
need the resulting variable. With ?='s recursive expansion [1], we
only expand the variable when it's consumed. For example, the ISODATE
variable is only needed in the recipe for the changelog target, so
most Make invocations won't need the value, and the computation is
just making whatever Make actually is doing slower.
I've shifted the GIT_COMMIT and BUILD_INFO values over to
LDFLAGS_PODMAN, because the test/*/* targets don't care about those.
I've also moved the Go-specific -ldflags from the variables into the
recipes themselves, because callers probably expect C semantics for
LDFLAGS and not Go's wrapper. That means that there's no longer a
need for the LDFLAGS/BASE_LDFLAGS separation, so I'm just using
LDFLAGS (and LDFLAGS_PODMAN) now. That reduces the declared variables
to just LDFLAGS_PODMAN, so I've shifted that declaration up to get it
closer to its GIT_COMMIT and BUILD_INFO precursors.
[1]: https://www.gnu.org/software/make/manual/html_node/Setting.html
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #777
Approved by: rhatdan
If the user uses the image ID when saving to either docker-archive
or oci-archive, then do not save a reference in the manifest/index.json.
If the user chooses to push without an image reference, i.e <transport>:<path>
it should be valid and succeed.
Signed-off-by: umohnani8 <umohnani@redhat.com>
Closes: #782
Approved by: rhatdan
Changes include saving image to docker-archive and oci-archive
eithout an image reference.
Signed-off-by: umohnani8 <umohnani@redhat.com>
Closes: #782
Approved by: rhatdan
And use 'go env GOBIN' to detect the user's existing preference. From
[1]:
> The bin directory holds compiled commands. Each command is named
> for its source directory, but only the final element, not the entire
> path. That is, the command with source in DIR/src/foo/quux is
> installed into DIR/bin/quux, not DIR/bin/foo/quux. The "foo/"
> prefix is stripped so that you can add DIR/bin to your PATH to get
> at the installed commands. If the GOBIN environment variable is
> set, commands are installed to the directory it names instead of
> DIR/bin. GOBIN must be an absolute path.
> ...
> Go searches each directory listed in GOPATH to find source code, but
> new packages are always downloaded into the first directory in the
> list.
So if GOBIN is set, it will be non-empty, and we can use $(GOBIN)/...
If GOBIN is unset, 'go env GOBIN' will return an empty string (as it
does on Travis [2]). In that case, I'm assuming that the package in
question is in the first directory in GOPATH and using the new
FIRST_GOPATH (firstword and subst are documented in [3]). That's
probably fairly safe, since our previous GOPATH handling assumed it
only contained a single path, and nobody was complaining about that.
Using ?= allows us to skip the 'dirname' call if we end up not needing
GOPKGBASEDIR [4] (e.g. for the 'help' target). The recursive
expansion could cause an issue if the result of the shell expansions
included a '$', but those seem unlikely in GOPKGBASEDIR, GOMD2MAN, or
the manpage paths. I haven't used ?= for GOBIN, because we'll always
need the expanded value for the if check.
Using GOMD2MAN allows us to collapse old ||-based recipe into a less
confusing invocation. And using a static pattern rule [5] for
$(MANPAGES) lets us write a single rule to handle both section 1 and
section 5.
While I was updating the GOPATH handling, I moved .gopathok from the
possibly-shared $(GOPATH)/.gopathok to the
definitely-specific-to-this-project .gopathok. That may cause some
issues if you rebuild after changing your GOPATH without calling
'clean', but I don't expect folks to change their GOPATH frequently.
And the old approach would fail if different consumers were also using
the same flag path to mean something else (as CRI-O does [6]).
As part of cleaning up .gopathok, I've also collapsed clean's rm calls
into a single invocation. That will give us the same results with
less process setup/teardown penalties.
[1]: https://golang.org/cmd/go/#hdr-GOPATH_environment_variable
[2]: https://travis-ci.org/projectatomic/libpod/jobs/379345071#L459
[3]: https://www.gnu.org/software/make/manual/html_node/Text-Functions.html
[4]: https://www.gnu.org/software/make/manual/html_node/Setting.html
[5]: https://www.gnu.org/software/make/manual/html_node/Static-Usage.html
[6]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.1/Makefile#L62
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #774
Approved by: mheon
We inherited this from a031b83a (Initial checkin from CRI-O repo,
2017-11-01), but:
* The output is actually going into bin/podman, so Make will rebuild
this target every time. You'll never be able to save compilation
because the target is newer than all the prerequisites.
* Make expands prerequisites immediately when loading a Makefile [1],
and on my wimpy Chromebook SD Card, this is *slow*:
$ time hack/find-godeps.sh ~/.local/lib/go/src/github.com/projectatomic/libpod cmd/podman github.com/projectatomic/libpod
...
real 0m56.225s
user 0m44.918s
sys 0m21.918s
* Go is pretty good at this on its own, so having make call 'go build'
every time will almost certainly be faster than us trying to mimic
this in a shell script. And by punting to Go in the recipe, Make
invocations that do not need the podman target (e.g. 'make help')
can skip the dependency lookup entirely.
[1]: https://www.gnu.org/software/make/manual/html_node/Reading-Makefiles.html#Rule-Definition
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #776
Approved by: rhatdan
Currently podman build http://remote.com/Dockerfile does not work.
podman always treats this file as an Archive.
Vendoring in the latest buildah code fixes this issue. Also updated
the man pages to better explain the syntax.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #775
Approved by: TomSweeneyRedHat
- More pythonic
- Leverage context managers to help with socket leaks
- Add system unittest's
- Add image unittest's
- Add container unittest's
- Add models for system, containers and images, and their collections
- Add helper functions for datetime parsing/formatting
- GetInfo() implemented
- Add support for setuptools
- Update documentation
- Support for Python 3.4-3.6
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Closes: #748
Approved by: baude
document --format|-f in the commit man page. also, improve the error
message when user tries to use -m with the oci image format.
Resolves: 765
Signed-off-by: baude <bbaude@redhat.com>
Closes: #768
Approved by: rhatdan
When no args were provided to podman push, podman segfaults. Quick fix to avoid the condition
that triggers the segf.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #773
Approved by: mheon
Theses are from a031b83a (Initial checkin from CRI-O repo,
2017-11-01), but they're CRI-O's logos and not libpod's or podman's.
We still use CRI-O's logos in a few places, but since a031b83a all of
those references have used cdn.rawgit.com URLs pointing at the CRI-O
repository. That means there's no need to keep local copies of
another project's logos here.
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #767
Approved by: rhatdan
Allow containers that no longer exist in storage to be evicted
from the state instead of erroring.
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
Closes: #764
Approved by: rhatdan
This change updates the install.md documentation to reference the new
cni directory location. This change also restores the previously
deleted README.md with updated instructions.
Signed-off-by: Tristan Cacqueray <tdecacqu@redhat.com>
Closes: #761
Approved by: baude
Following the vndr docs [1]:
$ go get -u github.com/LK4D4/vndr
$ vndr golang.org/x/text
$ git add -A vendor/golang.org/x/text
The targeted 'git add' was because we seem to have versioned some test
files (e.g. vendor/github.com/varlink/go/varlink/varlink_test.go in
8493dba2 (Initial varlink implementation, 2018-03-26, #627). I don't
know why, possibly an old vndr version? But either way, I'm punting
that particular issue to a separate branch.
[1]: 1fc68ee0c8/README.md
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #686
Approved by: mheon
And add an argument to WithHooksDir to set it.
If the hook dir doesn't exist, the new hooks package considers that a
fatal error. When a podman caller sets
--hooks-dir-path=/some/typoed/directory, a fatal error is more helpful
than silently not loading any hooks. However, callers who call podman
without setting --hooks-dir-path may not need hooks at all. We don't
want to pester those callers with not-exist errors. With this commit,
we:
* Assume the caller knows what they're doing if they set
--hooks-dir-path and set HooksDirNotExistFatal.
* If the caller does not explicitly set --hooks-dir-path, assume they
won't mind if the hook directory is missing and set
HooksDirNotExistFatal false.
We also considered checking for the directory's existence in the code
calling WithHooksDir or from within WithHooksDir, but checks there
would race with the underlying ioutil.ReadDir in the hooks package.
By pushing the warn/error decision down into libpod's implementation,
we avoid a racy "do we expect this to work once libpod gets to it?"
pre-check.
I've also added a check to error if WithHooksDir is called with an
empty-string argument, because we haven't defined the semantics of
that (is it clearing a previous value? Is it effectively the same as
the current directory?). I agree with Matthew that a separate
WithNoHooks, or a *string argument to WithHooks, or some such would be
a better API for clearing previous values [1]. But for now, I'm just
erroring out to fail early for callers who might otherwise be
surprised that libpod ignores empty-string HooksDir.
[1]: https://github.com/projectatomic/libpod/pull/686#issuecomment-385119370
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #686
Approved by: mheon
If a .json file existed when we called ioutil.ReadDir but that file
has been removed by the time we get around to calling Read on it,
silently ignore the file. Iterating through all the files in the
directory shouldn't take particularly long, so this is an unlikely
corner case. And when it happens, silently ignoring the file gives
the same outcome as you'd have gotten if the parallel remove had
happened slightly earlier before the ioutil.ReadDir call.
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #686
Approved by: mheon
This shifts the matching logic out of libpod/container_internal and
into the hook package, where we can reuse it after vendoring into
CRI-O. It also adds unit tests with almost-complete coverage. Now
libpod is even more isolated from the hook internals, which makes it
fairly straightforward to bump the hook config file to 1.0.0. I've
dubbed the old format 0.1.0, although it doesn't specify an explicit
version. Motivation for some of my changes with 1.0.0:
* Add an explicit version field. This will make any future JSON
structure migrations more straightforward by avoiding the need for
version-guessing heuristics.
* Collect the matching properties in a new When sub-structure. This
makes the root Hook structure easier to understand, because you
don't have to read over all the matching properties when wrapping
your head around Hook.
* Replace the old 'hook' and 'arguments' with a direct embedding of
the runtime-spec's hook structure. This provides access to
additional upstream properties (args[0], env, and timeout) and
avoids the complication of a CRI-O-specific analog structure.
* Add a 'when.always' property. You can usually accomplish this
effect in another way (e.g. when.commands = [".*"]), but having a
boolean explicitly for this use-case makes for easier reading and
writing.
* Replace the previous annotations array with an annotations map. The
0.1.0 approach matched only the values regardless of key, and that
seems unreliable.
* Replace 'cmds' with 'when.commands', because while there are a few
ways to abbreviate "commands", there's only one way to write it out
in full ;). This gives folks one less thing to remember when
writing hook JSON.
* Replace the old "inject if any specified condition matches" with
"inject if all specified conditions match". This allows for more
precise targeting. Users that need more generous targeting can
recover the previous behavior by creating a separate 1.0.0 hook file
for each specified 0.1.0 condition.
I've added doc-compat support for the various pluralizations of the
0.1.0 properties. Previously, the docs and code were not in
agreement. More on this particular facet in [1].
I've updated the docs to point out that the annotations being matched
are the OCI config annotations. This differs from CRI-O, where the
annotations used are the Kubernetes-supplied annotations [2,3]. For
example, io.kubernetes.cri-o.Volumes [4] is part of CRI-O's runtime
config annotations [5], but not part of the Kubernetes-supplied
annotations CRI-O uses for matching hooks.
The Monitor method supports the CRI-O use-case [6]. podman doesn't
need it directly, but CRI-O will need it when we vendor this package
there.
I've used nvidia-container-runtime-hook for the annotation examples
because Dan mentioned the Nvidia folks as the motivation behind
annotation matching. The environment variables are documented in [7].
The 0.1.0 hook config, which does not allow for environment variables,
only works because runc currently leaks the host environment into the
hooks [8]. I haven't been able to find documentation for their usual
annotation trigger or hook-install path, so I'm just guessing there.
[1]: https://github.com/kubernetes-incubator/cri-o/pull/1235
[2]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.0/server/container_create.go#L760
[3]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.0/server/container_create.go#L772
[4]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.0/pkg/annotations/annotations.go#L97-L98
[5]: https://github.com/kubernetes-incubator/cri-o/blob/v1.10.0/server/container_create.go#L830-L834
[6]: https://github.com/kubernetes-incubator/cri-o/pull/1345/
[7]: https://github.com/NVIDIA/nvidia-container-runtime/tree/v1.3.0-1#environment-variables-oci-spec
[8]: https://github.com/opencontainers/runc/pull/1738
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #686
Approved by: mheon
Fixes a bug where the force bool was being ignored when deleting images
via the varlink interface.
Also, minor fix to the docs to add a line break between methods and types in
the doc index.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #753
Approved by: rhatdan
containernetworking/plugins@a0eac8d7 (pkg/ns: remove namespace
creation, 2018-03-16) removed NewNS, which we use in
libpod/networking.go. Pinning to the previous commit,
containernetworking/plugins@1fb94a42 (Merge pull request #96 from
DennisDenuto/denuto/master, 2018-03-14), allows us to run vndr without
breaking our build. This is a short term fix; moving forward we'll
want to either drop this dependency or catch up with the new upstream
API.
The upstream package seems to have been fairly stable in the meantime,
because even with the new pinned version, a vndr re-vendor generates
no changes:
$ vndr github.com/containernetworking/plugins
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #751
Approved by: mheon
Until we get Systemd cgroup manager working, this will
cause a validation error.
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
Closes: #507
Approved by: baude
This directory just had Markdown and vendor.conf. I'm not sure why we
have it in our version control, maybe old versions of vndr kept it?
Or maybe folk dropped it into vendor/ by hand without using vndr? The
history of that vendored directory is:
* 619637a9 (Handle Linux Capabilities from command line, 2017-11-03,
#17) added the three files to our version control.
* c344fe61 (Update vendoring, 2017-11-22, #60) bumped hack/README.md.
* af64e104 (Vendor in lots of kubernetes stuff to shrink image size,
2018-03-26, #554) bumped hack/README.md.
* 27107fda (Vendor in latest containers/image and contaners/storage,
2018-04-18, #509) removed the files.
* a824186a (Use buildah commit and bud in podman, 2018-04-25, #681)
added the files back.
* I'm removing them again in this commit.
With this commit,
$ vndr github.com/docker/docker
becomes a no-op.
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #752
Approved by: baude
The varlinfo info returns the same information as podman info but always includes
the so-called debug information.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #745
Approved by: baude
vendor.conf has been pinned at containerd/cgroups@7a5fdd83 (Merge pull
request #26 from onorua/error-ignore-example, 2017-08-24) since libpod
forked from CRI-O with a031b83a (Initial checkin from CRI-O repo,
2017-11-01). The content in vendor/github.com/containerd/cgroups was
bumped to containerd/cgroups@77e62851 (Use /proc/diskstats to get
device names, 2018-01-31) in ae89dc28 (Update containerd/cgroups repo
fix perf issue, 2018-02-01, #284), but ae89dc28 forgot to update
vendor.conf. With this commit:
$ vndr github.com/containerd/cgroups
no longer changes anything under vendor/github.com/containerd/cgroups.
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #749
Approved by: mheon
Neither the nominal version nor vendored content had changed since
a031b83a (Initial checkin from CRI-O repo, 2017-11-01):
$ git ls-tree origin/master -- vendor/github.com/containernetworking/cni
040000 tree 744c091cc1dbb869b2cf714cf6b5e8d33fe17d9c vendor/github.com/containernetworking/cni
$ git ls-tree a031b83a -- vendor/github.com/containernetworking/cni
040000 tree 744c091cc1dbb869b2cf714cf6b5e8d33fe17d9c vendor/github.com/containernetworking/cni
$ git grep /cni a031b83a -- vendor.conf
a031b83a:vendor.conf:github.com/containernetworking/cni v0.4.0
I'm not quite sure which upstream version the old vendored content
came from, but it certainly wasn't v0.4.0. I've bumped our nominal
version to v0.6.0 and re-vendored with:
$ vndr github.com/containernetworking/cni
$ git add -A vendor/github.com/containernetworking/cni
to generate this commit. The only change to the vendored content is
text in the README (unlikely to break anything ;), and sitting on an
upstream tag is nice, so I think this is good enough.
Signed-off-by: W. Trevor King <wking@tremily.us>
Closes: #750
Approved by: mheon
When using varlink to pull an image, we should expose the actual error to the caller.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #744
Approved by: rhatdan
Like podman pull, when you push an image, podman should check
if the registry is listed as insecure and if so, it should
--tls-verify=false unless the user overrides this.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #738
Approved by: mheon
