1396 Commits

Author SHA1 Message Date
190e074459 Remove the :// end from DockerTransport
(... but keep it in DefaultTransport, which remains irregular.)

This makes DockerTransport consistent with the others, and much more importantly,
allows several instances to do
> imgRef.Transport().Name() == DockerTransport
instead of the current
> strings.HasPrefix(DockerTransport, imgRef.Transport().Name())
, which currently works but is pretty nonsensical (it does not check
the "docker://" prefix against the _full reference_, but it checks
the _transport name_ as a prefix of "docker://", i.e.  a transport named
"d" would be accepted.

Should not change behavior, because the only currently existing transport
which has a name that is a prefix of "docker://" is c/image/docker.Transport
(but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
9770ed257e Remove the TransportNames arrays
They are not used anywhere AFAICS, and the underlying idea
that transport-specific image names are reusable across transports
is very dubious anyway.  So, drop them instead of documenting
or fixing them.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
adfd3930c1 Document the properties of DefaultTransport a bit better.
This has no ambition to change the design, just to be clear about
what the design is.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
b3e6e908ab Eliminate the "dest" variable.
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
e8f7442831 Use an early exit if a docker-archive: image has no repo tags
This avoids another "append an only item to an empty array"
pattern, and will allow us to get rid of the "dest" variable
entirely.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
d4dbe66774 Reorganize the tag loading in DockerArchive case
This should not change behavior, only to make future edits
for an early exit easier to review.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
0ef38ba079 Return early in refNamesFromImageReference instead of appending to pullNames
Almost all paths appended to pullNames exactly once; just construct a
single-element array in place and return it.

That way we can add empty lines as separators, and still come out shorter.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
ecc1db39b5 Use srcRef.StringWithinTransport() instead of parsing imgName again
Because srcRef is created by parsing imgName, both hard-code assumptions
about transport-specific formats of the strings, so that is neither better nor worse;
but we do less explicit parsing.

Should not change behavior for dir:, nor for fully-correct docker-archive:.

docker-archive:, though, also supports docker-archive:path:reference, where
the reference is ignored (with a warning) on read; in such cases the previous
code would use the reference only (not the path), the new code uses both
as the path.  Neither works, we just change the failure mode (but
"error opening path:reference" is now more suggestive of the correct usage).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
24da27c3e9 Use a switch instead of if/if else/.../else
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
7c37b25b4d Remove the error return value from getPullRefName
... it is always nil.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
6ddf81f07d Rename getPullListFromRef to refPairsFromImageReference
This is a bit more specific as to what "ref" or "list" means,
and consistent with refPairsFromPossiblyUnqualifiedName

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
d61bed2b2d Split refNamesFromImageReference from Runtime.getPullListFromRef
Again, that makes the core logic independent from Runtime == containers-storage,
and easier to test independently.

So, this also adds tests.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
9c9401a96c Replace getPullRefPair with getPullRefName
... and use pullRefPairsFromRefNames to convert to the
desired data structure later.

This will make both getPullRefName, and later the bulk of
getPullListFromRef, independent of the storage, and thus much easier to test.

Then add tests for getPullRefName.  (Ideally they should be shorter,
e.g. hopefully the .image member can be eliminated.)

Should not change behavior, except that error messages on invalid
dstName will now include the value.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
8e7b4944f0 Include the rejected reference when parsing it fails in pullRefPairsFromRefNames
This will make any failures easier to attribute to the cause.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
8e1ef558eb Add --force to podman umount to force the unmounting of the rootfs
podman umount will currently only unmount file system if not other
process is using it, otherwise the umount decrements the container
storage to indicate that the caller is no longer using the mount
point, once the count gets to 0, the file system is actually unmounted.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1184
Approved by: TomSweeneyRedHat
2018-08-01 17:53:30 +00:00
a8ae7eae9c Integration Test Improvements #3
Third round of speed improvements to the integration tests.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1193
Approved by: rhatdan
2018-08-01 13:01:44 +00:00
1a439f9fcb Ensure container and pod refresh picks up a State
refresh() is the only major command we had that did not perform a
sync before running, and thus was not guaranteed to pick up a
good copy of the state. Fix this by updating the state before a
refresh().

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
21f50b9f34 Fix build on non-linux platforms
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
c7c56d800c Rework state testing to allow State structs to be empty
Pod and container State structs are now allowed to be empty on
first being retrieved from the database. Rework pod and container
equality functions used in testing to account for this change.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
f4120f9662 Add additional comments on accessing state in API
The new state changes are potentially confusing to people writing
API functions on containers or pods. Add comments to the structs
on how to safely use them.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
1db70cce34 Do not fetch pod and ctr State on retrieval in Bolt
It's not necessary to fill in state immediately, as we'll be
overwriting it on any API call accessing it thanks to
syncContainer(). It is also causing races when we fetch it
without holding the container lock (which syncContainer() does).
As such, just don't retrieve the state on initial pull from the
database with Bolt.

Also, refactor some Linux-specific netns handling functions out
of container_internal_linux.go into boltdb_linux.go.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
cfcd928476 network: add support for rootless network with slirp4netns
slirp4netns is required to setup the network namespace:

https://github.com/rootless-containers/slirp4netns

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #1156
Approved by: rhatdan
2018-07-31 13:39:29 +00:00
5b9c60cc10 varlink ImageRemove should always return image ID
When removing an image via varlink, we should always return the
ID of the image even in the case where the image has multiple
repository names and one was only untagged.

Reported by jhonce during integration testing.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1191
Approved by: jwhonce
2018-07-31 11:38:48 +00:00
00caebde61 Add documentations on how to setup /etc/subuid and /etc/subgid
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1185
Approved by: giuseppe
2018-07-31 08:35:20 +00:00
5a4e5902a0 Integration Test Improvements #2
This is the second round of performance improvements for out
integration tests.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1190
Approved by: rhatdan
2018-07-30 23:53:08 +00:00
8694e5414c avoid spewing fds do to restore of cached images
due to how cstorage is designed, we were spewing thousands of fds when
we restored cached images causing unwieldy rlimits.  we now use podman
load to restore the images thereby not tripping the issue.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1188
Approved by: baude
2018-07-30 20:28:30 +00:00
49bdd8421b Add load test for xz compressed images
The auto decompression functionality was already vendored in
with containers/image. Adding a test for it.

Signed-off-by: umohnani8 <umohnani@redhat.com>

Closes: #1137
Approved by: rhatdan
2018-07-30 16:56:11 +00:00
49b3647410 Speed up test results
Stop all containers with a zero timeout prior to trying to rm -fa.  This results
in quicker teardown times by not waiting for timeouts.

Also, with wait tests, no need to wait the full 10 second sleep.  1 will do.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1181
Approved by: rhatdan
2018-07-30 12:26:39 +00:00
433cbd5254 Show duration for each ginkgo test and test speed improvements
Because our tests are getting so long, we want to be able to audit which tests are taking
the longest to complete.  This may indicate a bad test, bad CI, bad code, etc and therefore
should be auditable.

Also, make speed improvements to tests by making sure we only unpack caches images that
actually get used.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1178
Approved by: mheon
2018-07-28 22:51:08 +00:00
a4a667eac9 vendor: update containers/storage
update to version 956a1971694f18fd602b1203c0a2d192e2cc88a1

inherit support for IDs shifting when fuse-overlayfs is used.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #1177
Approved by: mheon
2018-07-28 14:41:07 +00:00
5aa36c1861 Clean up pylint warnings and errors for podman
* Some of the pedantic errors were not corrected
* Clean up prep for porting to MacOS and PyPi hosting
* Fix broken unittest

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Closes: #1159
Approved by: rhatdan
2018-07-28 11:49:11 +00:00
87d8edb4c1 podman rmi shouldn't delete named referenced images
If an image is created from another and it is deleted,
only delete the actual image and not the parent images
if the parent images have names/references.

Signed-off-by: umohnani8 <umohnani@redhat.com>

Closes: #1174
Approved by: mheon
2018-07-28 01:40:28 +00:00
bd7566f170 Merge pull request #1173 from mheon/bump-0.7.4
Bump to 0.7.4
2018-07-27 12:11:50 -04:00
8a10bf5f42 Bump gitvalidation epoch
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-27 12:05:42 -04:00
07a1f33edc Bump to v0.8.1-dev
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-27 12:05:41 -04:00
80612fb046 Bump to v0.7.4
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
v0.7.4
2018-07-27 12:05:38 -04:00
f258e43c7c Add pod pause/unpause
Added Pause() and Unpause() to libpod/pod.go

Added man pages, tests and completions

Signed-off-by: haircommander <pehunt@redhat.com>

Closes: #1126
Approved by: rhatdan
2018-07-27 14:20:08 +00:00
50fea69fbc Fix up docker compatibility messages
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1170
Approved by: mheon
2018-07-27 13:40:56 +00:00
02e7efc2b3 Update vendored version of runc,buildah,containers/image
There is a compiler warning that has been fixed in the
upstream, so I figured we should update to fix.

Also vendor in latest buildah to get better support for running builds in rootless
mode.

Vendor in latest containers/image to allow daemon support to be pluggable.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1169
Approved by: mheon
2018-07-27 13:07:19 +00:00
876a30590b Refactored method of getting pods
Now, for commands that have --latest and --all, the context flags are checked, and pods are grabbed in a single function

Signed-off-by: haircommander <pehunt@redhat.com>

Closes: #1161
Approved by: rhatdan
2018-07-27 12:39:12 +00:00
3dd577e93c Fix godoc comment in pkg/netns
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1165
Approved by: baude
2018-07-27 02:48:16 +00:00
8c52aa15f0 Fix handling of Linux network namespaces
The CNI plugins upstream removed their network namespace creation
code, making it a test package only. Copy it into our repository
and slightly modify it for our use (most notably, use MNT_DETACH
when unmounting namespaces).

This new CNI code splits closing and unmounting network
namespaces, which allows us to greatly reduce the number of
occasions on which we call teardownNetwork() and make more errors
in that function fatal instead of warnings. Instead, we can call
Close() and just close the open file descriptor in cases where
the namespace has already been cleaned up.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1165
Approved by: baude
2018-07-27 02:48:15 +00:00
54967d7a10 Update containernetworking/plugins to current master
We need to pick up changes to the netns packages that are not yet
in a released tag.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1165
Approved by: baude
2018-07-27 02:48:15 +00:00
f9152d075a Cleanup descriptions and help information
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1167
Approved by: baude
2018-07-27 02:17:29 +00:00
c142409598 Skip seccomp-dependent tests on non-Linux
Currently, getRuntimeSpec always fails on non-Linux because
spec.CreateConfigToOCISpec always fails, because the podman CLI
sets up a seccomp path, and processing that on non-Linux is not supported.  This
breaks testing of entirely unrelated options.

We can either skip the tests on non-Linux, or explicitly disable seccomp
inside the tests.  Linux testing matters much more than other platforms,
and the tests are more reliable when they don't change supposedly
unrelated options; so, skip the tests on non-Linux.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
2322f272c4 Use the Linux version BoltState.getContainerFromDB on all platforms.
This just muves the Linux implementation, unchanged, to the
platform-agnostic file.  Should not change behavior on Linux.

On non-Linux platforms, reading containers from BoltDB now works
(and rejects containers with namespace data).  The checkRuntimeConfig
validation ensures that each BoltDB database is only used on one platform,
so network namespaces should never exist in non-Linux BoltDB files.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
eba6bf0018 Split parseNetNSBoltData from BoltState.getContainerFromDB
This is the actual platform-specific part of getContainerFromDB.

Factor it out, unchanged, on Linux. On other platforms, introduce
a stub which fails if any data exists; this stub is not yet called.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
24fe6e950c Use testify/require in a few places to avoid panics in tests
Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
9ff4f40094 Skip unit tests which require storage when not running as root
On macOS NewImageRuntimeFromOptions fails with chown EPERM because the
"vfs" driver tries to chown its home to root:root 0700; in fact running
as root seems to be a generic requirement.  So, skip the tests if not
running as root.

(This could maybe benefit from an extra state, maybe an environment
variable like RUNNING_IN_CI, to make sure the tests are actually
run often enough.)

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00
a0575d6e46 Don't pollute the build output with failures to build checkseccomp
(make localunit) complains about the unbuildable subpackage; silence that.

Actually, I can't find any user of checkseccomp in this repo; maybe
it is entirely unused?

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1115
Approved by: rhatdan
2018-07-26 20:47:31 +00:00