756 Commits

Author SHA1 Message Date
fadb143399 Introduce struct pullGoal
The eventual goal is to cleanly capture semantics like "pull all images
for DockerArchive" and "did a search through $registries" without
hard-coding it through; and to allow a pullImage variant where
the caller can pass an imageReference directly.

For now, this just wraps []pullRefPair and should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
bf0ab88eac Use []pullRefPair instead of []*pullRefPair
We are passing the values, don't really need the pointer sharing semantics,
and the structures are small enough, and the arrays short enough,
that we very likely lose on the indirect accesses more than we save on
quicker copying of the slices when extending them.  Value semantics
is safer anyway.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
dae6200662 Use []pullRefName instead of []*pullRefName
We are passing the values, don't really need the pointer sharing semantics,
and the structures are small enough, and the arrays short enough,
that we very likely lose on the indirect accesses more than we save on
quicker copying of the slices when extending them.  Value semantics
is safer anyway.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
83f40de965 Introduce singlePullRefNameGoal
All but two cases returning a []*pullRefName only return a single
item.  Introduce a helper for that case, which seems not
worth it now, but the return value will get a bit more complex
and introducing the helper now will minimize code changes in future
commits.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
1efbc40999 Use an early return from refNamesFromPossiblyUnqualifiedName
We will introduce helpers for the "single image" case, and having a separate
return statement will make them applicable here.

(Also allows us to reduce the scope of some variables a bit.)

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
4dab4d97de RFC: Rename Image.PushImage to Image.PushImageToHeuristicDestination
The goal is to be very explicit about which functions try to heuristically
guess what is the expected format of the string.  Not quite "shaming"
the users, but making sure they stand out.

RFC:
- Is this at all acceptable? Desirable?
- varlink ExportImage says "destination must have transport type";
  should it be using alltransports.ParseImageReference
  + PushImageToReference, then?

(While touching the call in cmd/podman, also remove a commented-out
older version of the call.)

Should not change behavior (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
86fb1bf8eb Remove an unnecessary use of alltransports.ParseImageName
When the string is formatted including a constant transport name,
just call the transport to create or parse a reference explicitly.

This avoids unnecessary string formatting and parsing.

Then drop image.TarballTransport, which has no remaining users.

Should not change behavior (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
891392339f Split Image.PushImageToReference from Image.PushImage
This retains the existing string parsing heuristic for users
who must continue to use it (notably the varlink API - or is
it still subject to change?), but allows callers who can get
precise references to supply them without having to deal
with string formatting.

Should not change behavior (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
1153486ab0 Don't format to string and re-parse a DockerReference()
We already have a c/image/docker/reference.Named; no need to
round-trip it through a string.  This also eliminates the theoretical
parsing failure, and the unchecked .(reference.Named) cast.

Also add a check for DockerReference() == nil to be extra paranoid,
although that should never happen.

Should not change behavior (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
190e074459 Remove the :// end from DockerTransport
(... but keep it in DefaultTransport, which remains irregular.)

This makes DockerTransport consistent with the others, and much more importantly,
allows several instances to do
> imgRef.Transport().Name() == DockerTransport
instead of the current
> strings.HasPrefix(DockerTransport, imgRef.Transport().Name())
, which currently works but is pretty nonsensical (it does not check
the "docker://" prefix against the _full reference_, but it checks
the _transport name_ as a prefix of "docker://", i.e.  a transport named
"d" would be accepted.

Should not change behavior, because the only currently existing transport
which has a name that is a prefix of "docker://" is c/image/docker.Transport
(but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
9770ed257e Remove the TransportNames arrays
They are not used anywhere AFAICS, and the underlying idea
that transport-specific image names are reusable across transports
is very dubious anyway.  So, drop them instead of documenting
or fixing them.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
adfd3930c1 Document the properties of DefaultTransport a bit better.
This has no ambition to change the design, just to be clear about
what the design is.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
b3e6e908ab Eliminate the "dest" variable.
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
e8f7442831 Use an early exit if a docker-archive: image has no repo tags
This avoids another "append an only item to an empty array"
pattern, and will allow us to get rid of the "dest" variable
entirely.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
d4dbe66774 Reorganize the tag loading in DockerArchive case
This should not change behavior, only to make future edits
for an early exit easier to review.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
0ef38ba079 Return early in refNamesFromImageReference instead of appending to pullNames
Almost all paths appended to pullNames exactly once; just construct a
single-element array in place and return it.

That way we can add empty lines as separators, and still come out shorter.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
ecc1db39b5 Use srcRef.StringWithinTransport() instead of parsing imgName again
Because srcRef is created by parsing imgName, both hard-code assumptions
about transport-specific formats of the strings, so that is neither better nor worse;
but we do less explicit parsing.

Should not change behavior for dir:, nor for fully-correct docker-archive:.

docker-archive:, though, also supports docker-archive:path:reference, where
the reference is ignored (with a warning) on read; in such cases the previous
code would use the reference only (not the path), the new code uses both
as the path.  Neither works, we just change the failure mode (but
"error opening path:reference" is now more suggestive of the correct usage).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
24da27c3e9 Use a switch instead of if/if else/.../else
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
7c37b25b4d Remove the error return value from getPullRefName
... it is always nil.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
6ddf81f07d Rename getPullListFromRef to refPairsFromImageReference
This is a bit more specific as to what "ref" or "list" means,
and consistent with refPairsFromPossiblyUnqualifiedName

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
d61bed2b2d Split refNamesFromImageReference from Runtime.getPullListFromRef
Again, that makes the core logic independent from Runtime == containers-storage,
and easier to test independently.

So, this also adds tests.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
9c9401a96c Replace getPullRefPair with getPullRefName
... and use pullRefPairsFromRefNames to convert to the
desired data structure later.

This will make both getPullRefName, and later the bulk of
getPullListFromRef, independent of the storage, and thus much easier to test.

Then add tests for getPullRefName.  (Ideally they should be shorter,
e.g. hopefully the .image member can be eliminated.)

Should not change behavior, except that error messages on invalid
dstName will now include the value.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
8e7b4944f0 Include the rejected reference when parsing it fails in pullRefPairsFromRefNames
This will make any failures easier to attribute to the cause.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
8e1ef558eb Add --force to podman umount to force the unmounting of the rootfs
podman umount will currently only unmount file system if not other
process is using it, otherwise the umount decrements the container
storage to indicate that the caller is no longer using the mount
point, once the count gets to 0, the file system is actually unmounted.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1184
Approved by: TomSweeneyRedHat
2018-08-01 17:53:30 +00:00
1a439f9fcb Ensure container and pod refresh picks up a State
refresh() is the only major command we had that did not perform a
sync before running, and thus was not guaranteed to pick up a
good copy of the state. Fix this by updating the state before a
refresh().

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
21f50b9f34 Fix build on non-linux platforms
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
c7c56d800c Rework state testing to allow State structs to be empty
Pod and container State structs are now allowed to be empty on
first being retrieved from the database. Rework pod and container
equality functions used in testing to account for this change.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
f4120f9662 Add additional comments on accessing state in API
The new state changes are potentially confusing to people writing
API functions on containers or pods. Add comments to the structs
on how to safely use them.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
1db70cce34 Do not fetch pod and ctr State on retrieval in Bolt
It's not necessary to fill in state immediately, as we'll be
overwriting it on any API call accessing it thanks to
syncContainer(). It is also causing races when we fetch it
without holding the container lock (which syncContainer() does).
As such, just don't retrieve the state on initial pull from the
database with Bolt.

Also, refactor some Linux-specific netns handling functions out
of container_internal_linux.go into boltdb_linux.go.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
cfcd928476 network: add support for rootless network with slirp4netns
slirp4netns is required to setup the network namespace:

https://github.com/rootless-containers/slirp4netns

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #1156
Approved by: rhatdan
2018-07-31 13:39:29 +00:00
87d8edb4c1 podman rmi shouldn't delete named referenced images
If an image is created from another and it is deleted,
only delete the actual image and not the parent images
if the parent images have names/references.

Signed-off-by: umohnani8 <umohnani@redhat.com>

Closes: #1174
Approved by: mheon
2018-07-28 01:40:28 +00:00
f258e43c7c Add pod pause/unpause
Added Pause() and Unpause() to libpod/pod.go

Added man pages, tests and completions

Signed-off-by: haircommander <pehunt@redhat.com>

Closes: #1126
Approved by: rhatdan
2018-07-27 14:20:08 +00:00
8c52aa15f0 Fix handling of Linux network namespaces
The CNI plugins upstream removed their network namespace creation
code, making it a test package only. Copy it into our repository
and slightly modify it for our use (most notably, use MNT_DETACH
when unmounting namespaces).

This new CNI code splits closing and unmounting network
namespaces, which allows us to greatly reduce the number of
occasions on which we call teardownNetwork() and make more errors
in that function fatal instead of warnings. Instead, we can call
Close() and just close the open file descriptor in cases where
the namespace has already been cleaned up.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1165
Approved by: baude
2018-07-27 02:48:15 +00:00
2322f272c4 Use the Linux version BoltState.getContainerFromDB on all platforms.
This just muves the Linux implementation, unchanged, to the
platform-agnostic file.  Should not change behavior on Linux.

On non-Linux platforms, reading containers from BoltDB now works
(and rejects containers with namespace data).  The checkRuntimeConfig
validation ensures that each BoltDB database is only used on one platform,
so network namespaces should never exist in non-Linux BoltDB files.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
eba6bf0018 Split parseNetNSBoltData from BoltState.getContainerFromDB
This is the actual platform-specific part of getContainerFromDB.

Factor it out, unchanged, on Linux. On other platforms, introduce
a stub which fails if any data exists; this stub is not yet called.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
24fe6e950c Use testify/require in a few places to avoid panics in tests
Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
9ff4f40094 Skip unit tests which require storage when not running as root
On macOS NewImageRuntimeFromOptions fails with chown EPERM because the
"vfs" driver tries to chown its home to root:root 0700; in fact running
as root seems to be a generic requirement.  So, skip the tests if not
running as root.

(This could maybe benefit from an extra state, maybe an environment
variable like RUNNING_IN_CI, to make sure the tests are actually
run often enough.)

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
159f7f179b vendor latest containers/psgo
Signed-off-by: Valentin Rothberg <vrothberg@suse.com>

Closes: #1162
Approved by: rhatdan
2018-07-26 17:01:40 +00:00
d9ae17400d Merge pull request #1158 from mheon/prevent_multiple_boltdb_conns
Add a mutex to BoltDB state to prevent lock issues
2018-07-26 10:41:30 -04:00
8ce0e0b246 Added pod restart
With tests, man page and completions.

Signed-off-by: haircommander <pehunt@redhat.com>

Closes: #1152
Approved by: rhatdan
2018-07-25 17:54:27 +00:00
7789284cbe Added pod.Restart() functionality to libpod.
Moved contents of RestartWithTimeout to restartWithTimeout in container_internal to be able to call restart without locking in function.
Refactored startNode to be able to either start or restart a node.
Built pod Restart() with new startNode with refresh true.

Signed-off-by: haircommander <pehunt@redhat.com>

Closes: #1152
Approved by: rhatdan
2018-07-25 17:54:27 +00:00
42bd9d3880 Add a mutex to BoltDB state to prevent lock issues
Per https://www.sqlite.org/src/artifact/c230a7a24?ln=994-1081,
POSIX file advisory locks are unsafe to use within a single
process if multiple file descriptors are open for the same file.
Unfortunately, this has a strong potential to happen for
multithreaded usage of libpod, and could result in DB corruption.

To prevent this, wrap all access to BoltDB within a single
libpod instance in a mutex to ensure concurrent access cannot
occur.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-25 11:37:15 -04:00
1b51e88098 Update comments in BoltDB and In-Memory states
Better explain the inner workings of both state types in comments
to make reviews and changes easier.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-24 16:12:31 -04:00
486c5c87bc Add missing runtime.go lines to set namespace
Also add namespace to inspect output to verify its presence

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-24 16:12:31 -04:00
7a358e4277 Address first round of review comments
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-24 16:12:31 -04:00
fc95f68247 Set namespace for new pods/containers based on runtime
New containers and pods will default to the namespace of the
runtime, but this can be overridden by With... options if
desired.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-24 16:12:31 -04:00
3ae0c80806 Add --namespace flag to Podman
Allows joining libpod to a specific namespace when running a
Podman command.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-24 16:12:31 -04:00
8f91678a49 Update documentation for the State interface
Include details on how namespaces interact with the
state.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-24 16:12:31 -04:00
84afa32493 Ensure pods are part of the set namespace when added
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-24 16:12:31 -04:00
7b30659629 Enforce namespace checks on container add
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-24 16:12:31 -04:00