varlink build

Add the endpoint and methods for build so users can build an image
with varlink.  build can also use the more method for streaming
output back more regularily; however, it looks like a bug in buildah
does not output all build output to the writer provided.

Tidy up some create fixes and add endpoint for GetImage requested by
jhonce.

Signed-off-by: baude <bbaude@redhat.com>

API.md

@@ -5,7 +5,7 @@ in the [API.md](https://github.com/projectatomic/libpod/blob/master/API.md) file
 
 [func AttachToContainer() NotImplemented](#AttachToContainer)
 
-[func BuildImage() NotImplemented](#BuildImage)
+[func BuildImage(build: BuildInfo) []string](#BuildImage)
 
 [func Commit(name: string, image_name: string, changes: []string, author: string, message: string, pause: bool) string](#Commit)
 
@@ -29,6 +29,8 @@ in the [API.md](https://github.com/projectatomic/libpod/blob/master/API.md) file
 
 [func GetContainerStats(name: string) ContainerStats](#GetContainerStats)
 
+[func GetImage(name: string) ImageInList](#GetImage)
+
 [func GetInfo() PodmanInfo](#GetInfo)
 
 [func GetVersion() Version](#GetVersion)
@@ -83,6 +85,8 @@ in the [API.md](https://github.com/projectatomic/libpod/blob/master/API.md) file
 
 [func WaitContainer(name: string) int](#WaitContainer)
 
+[type BuildInfo](#BuildInfo)
+
 [type ContainerChanges](#ContainerChanges)
 
 [type ContainerMount](#ContainerMount)
@@ -144,8 +148,10 @@ This method has not be implemented yet.
 ### <a name="BuildImage"></a>func BuildImage
 <div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
 
-method BuildImage() [NotImplemented](#NotImplemented)</div>
+method BuildImage(build: [BuildInfo](#BuildInfo)) [[]string](#[]string)</div>
-This function is not implemented yet.
+BuildImage takes a [BuildInfo](#BuildInfo) structure and builds an image.  At a minimum, you must provide the
+'dockerfile' and 'tags' options in the BuildInfo structure.  Upon a successful build, it will
+return the ID of the container.
 ### <a name="Commit"></a>func Commit
 <div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
 
@@ -161,7 +167,16 @@ the resulting image's ID will be returned as a string.
 <div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
 
 method CreateContainer(create: [Create](#Create)) [string](https://godoc.org/builtin#string)</div>
-CreateContainer creates a new container from an image.  It uses a (Create)[#Create] type for input.
+CreateContainer creates a new container from an image.  It uses a [Create](#Create) type for input. The minimum
+input required for CreateContainer is an image name.  If the image name is not found, an [ImageNotFound](#ImageNotFound)
+error will be returned.  Otherwise, the ID of the newly created container will be returned.
+#### Example
+~~~
+$ varlink call unix:/run/podman/io.projectatomic.podman/io.projectatomic.podman.CreateContainer '{"create": {"image": "alpine"}}'
+{
+  "container": "8759dafbc0a4dc3bcfb57eeb72e4331eb73c5cc09ab968e65ce45b9ad5c4b6bb"
+}
+~~~
 ### <a name="CreateImage"></a>func CreateImage
 <div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
 
@@ -221,7 +236,7 @@ $ varlink call -m unix:/run/io.projectatomic.podman/io.projectatomic.podman.GetA
 method GetContainer(name: [string](https://godoc.org/builtin#string)) [ListContainerData](#ListContainerData)</div>
 GetContainer takes a name or ID of a container and returns single ListContainerData
 structure.  A [ContainerNotFound](#ContainerNotFound) error will be returned if the container cannot be found.
-See also [ListContainers](ListContainers) and [InspectContainer](InspectContainer).
+See also [ListContainers](ListContainers) and [InspectContainer](#InspectContainer).
 ### <a name="GetContainerLogs"></a>func GetContainerLogs
 <div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
 
@@ -258,6 +273,12 @@ $ varlink call -m unix:/run/podman/io.projectatomic.podman/io.projectatomic.podm
   }
 }
 ~~~
+### <a name="GetImage"></a>func GetImage
+<div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
+
+method GetImage(name: [string](https://godoc.org/builtin#string)) [ImageInList](#ImageInList)</div>
+GetImage returns a single image in an [ImageInList](#ImageInList) struct.  You must supply an image name as a string.
+If the image cannot be found, an [ImageNotFound](#ImageNotFound) error will be returned.
 ### <a name="GetInfo"></a>func GetInfo
 <div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
 
@@ -349,7 +370,7 @@ an image currently in storage.  See also [InspectImage](InspectImage).
 method PauseContainer(name: [string](https://godoc.org/builtin#string)) [string](https://godoc.org/builtin#string)</div>
 PauseContainer takes the name or ID of container and pauses it.  If the container cannot be found,
 a [ContainerNotFound](#ContainerNotFound) error will be returned; otherwise the ID of the container is returned.
-See also [UnpauseContainer](UnpauseContainer).
+See also [UnpauseContainer](#UnpauseContainer).
 ### <a name="Ping"></a>func Ping
 <div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
 
@@ -471,7 +492,7 @@ be found, an [ImageNotFound](#ImageNotFound) error will be returned; otherwise,
 method UnpauseContainer(name: [string](https://godoc.org/builtin#string)) [string](https://godoc.org/builtin#string)</div>
 UnpauseContainer takes the name or ID of container and unpauses a paused container.  If the container cannot be
 found, a [ContainerNotFound](#ContainerNotFound) error will be returned; otherwise the ID of the container is returned.
-See also [PauseContainer](PauseContainer).
+See also [PauseContainer](#PauseContainer).
 ### <a name="UpdateContainer"></a>func UpdateContainer
 <div style="background-color: #E8E8E8; padding: 15px; margin: 10px; border-radius: 10px;">
 
@@ -485,6 +506,57 @@ WaitContainer takes the name or ID of a container and waits until the container
 code of the container is returned. If the container container cannot be found by ID or name,
 a [ContainerNotFound](#ContainerNotFound) error is returned.
 ## Types
+### <a name="BuildInfo"></a>type BuildInfo
+
+BuildInfo is used to describe user input for building images
+
+dockerfile [[]string](#[]string)
+
+tags [[]string](#[]string)
+
+add_hosts [[]string](#[]string)
+
+cgroup_parent [string](https://godoc.org/builtin#string)
+
+cpu_period [int](https://godoc.org/builtin#int)
+
+cpu_quota [int](https://godoc.org/builtin#int)
+
+cpu_shares [int](https://godoc.org/builtin#int)
+
+cpuset_cpus [string](https://godoc.org/builtin#string)
+
+cpuset_mems [string](https://godoc.org/builtin#string)
+
+memory [string](https://godoc.org/builtin#string)
+
+memory_swap [string](https://godoc.org/builtin#string)
+
+security_opts [[]string](#[]string)
+
+shm_size [string](https://godoc.org/builtin#string)
+
+ulimit [[]string](#[]string)
+
+volume [[]string](#[]string)
+
+squash [bool](https://godoc.org/builtin#bool)
+
+pull [bool](https://godoc.org/builtin#bool)
+
+pull_always [bool](https://godoc.org/builtin#bool)
+
+force_rm [bool](https://godoc.org/builtin#bool)
+
+rm [bool](https://godoc.org/builtin#bool)
+
+label [[]string](#[]string)
+
+annotations [[]string](#[]string)
+
+build_args [map[string]](#map[string])
+
+image_format [string](https://godoc.org/builtin#string)
 ### <a name="ContainerChanges"></a>type ContainerChanges
 
 ContainerChanges describes the return struct for ListContainerChanges
@@ -695,7 +767,7 @@ security_opts [[]string](#[]string)
 ### <a name="CreateResourceConfig"></a>type CreateResourceConfig
 
 CreateResourceConfig is an input structure used to describe host attributes during
-container creation.  It is only valid inside a (Create)[#Create] type.
+container creation.  It is only valid inside a [Create](#Create) type.
 
 blkio_weight [int](https://godoc.org/builtin#int)
 

cmd/podman/varlink/io.projectatomic.podman.varlink

@@ -248,7 +248,7 @@ type Create (
 )
 
 # CreateResourceConfig is an input structure used to describe host attributes during
-# container creation.  It is only valid inside a (Create)[#Create] type.
+# container creation.  It is only valid inside a [Create](#Create) type.
 type CreateResourceConfig (
     blkio_weight: int,
     blkio_weight_device: []string,
@@ -291,6 +291,35 @@ type IDMap (
     size: int
 )
 
+# BuildInfo is used to describe user input for building images
+type BuildInfo (
+    # paths to one or more dockerfiles
+    dockerfile: []string,
+    tags: []string,
+    add_hosts: []string,
+    cgroup_parent: string,
+    cpu_period: int,
+    cpu_quota: int,
+    cpu_shares: int,
+    cpuset_cpus: string,
+    cpuset_mems: string,
+    memory: string,
+    memory_swap: string,
+    security_opts: []string,
+    shm_size: string,
+    ulimit: []string,
+    volume: []string,
+    squash: bool,
+    pull: bool,
+    pull_always: bool,
+    force_rm: bool,
+    rm: bool,
+    label: []string,
+    annotations: []string,
+    build_args: [string]string,
+    image_format: string
+)
+
 # Ping provides a response for developers to ensure their varlink setup is working.
 # #### Example
 # ~~~
@@ -317,10 +346,19 @@ method ListContainers() -> (containers: []ListContainerData)
 
 # GetContainer takes a name or ID of a container and returns single ListContainerData
 # structure.  A [ContainerNotFound](#ContainerNotFound) error will be returned if the container cannot be found.
-# See also [ListContainers](ListContainers) and [InspectContainer](InspectContainer).
+# See also [ListContainers](ListContainers) and [InspectContainer](#InspectContainer).
 method GetContainer(name: string) -> (container: ListContainerData)
 
-# CreateContainer creates a new container from an image.  It uses a (Create)[#Create] type for input.
+# CreateContainer creates a new container from an image.  It uses a [Create](#Create) type for input. The minimum
+# input required for CreateContainer is an image name.  If the image name is not found, an [ImageNotFound](#ImageNotFound)
+# error will be returned.  Otherwise, the ID of the newly created container will be returned.
+# #### Example
+# ~~~
+# $ varlink call unix:/run/podman/io.projectatomic.podman/io.projectatomic.podman.CreateContainer '{"create": {"image": "alpine"}}'
+# {
+#   "container": "8759dafbc0a4dc3bcfb57eeb72e4331eb73c5cc09ab968e65ce45b9ad5c4b6bb"
+# }
+# ~~~
 method CreateContainer(create: Create) -> (container: string)
 
 # InspectContainer data takes a name or ID of a container returns the inspection
@@ -429,12 +467,12 @@ method RenameContainer() -> (notimplemented: NotImplemented)
 
 # PauseContainer takes the name or ID of container and pauses it.  If the container cannot be found,
 # a [ContainerNotFound](#ContainerNotFound) error will be returned; otherwise the ID of the container is returned.
-# See also [UnpauseContainer](UnpauseContainer).
+# See also [UnpauseContainer](#UnpauseContainer).
 method PauseContainer(name: string) -> (container: string)
 
 # UnpauseContainer takes the name or ID of container and unpauses a paused container.  If the container cannot be
 # found, a [ContainerNotFound](#ContainerNotFound) error will be returned; otherwise the ID of the container is returned.
-# See also [PauseContainer](PauseContainer).
+# See also [PauseContainer](#PauseContainer).
 method UnpauseContainer(name: string) -> (container: string)
 
 # This method has not be implemented yet.
@@ -482,8 +520,14 @@ method DeleteStoppedContainers() -> (containers: []string)
 # an image currently in storage.  See also [InspectImage](InspectImage).
 method ListImages() -> (images: []ImageInList)
 
-# This function is not implemented yet.
+# GetImage returns a single image in an [ImageInList](#ImageInList) struct.  You must supply an image name as a string.
-method BuildImage() -> (notimplemented: NotImplemented)
+# If the image cannot be found, an [ImageNotFound](#ImageNotFound) error will be returned.
+method GetImage(name: string) -> (image: ImageInList)
+
+# BuildImage takes a [BuildInfo](#BuildInfo) structure and builds an image.  At a minimum, you must provide the
+# 'dockerfile' and 'tags' options in the BuildInfo structure.  Upon a successful build, it will
+# return the ID of the container.
+method BuildImage(build: BuildInfo) -> (image: []string)
 
 # This function is not implemented yet.
 method CreateImage() -> (notimplemented: NotImplemented)

pkg/varlinkapi/containers.go

@@ -182,6 +182,9 @@ func (i *LibpodAPI) GetContainerLogs(call ioprojectatomicpodman.VarlinkCall, nam
 			logs = append(logs, line)
 		}
 	}
+
+	call.Continues = false
+
 	return call.ReplyGetContainerLogs(logs)
 }
 

pkg/varlinkapi/containers_create.go

@@ -22,14 +22,6 @@ import (
 
 // CreateContainer ...
 func (i *LibpodAPI) CreateContainer(call ioprojectatomicpodman.VarlinkCall, config ioprojectatomicpodman.Create) error {
-	//mappings, err := util.ParseIDMapping(config.Uidmap, config.Gidmap, config.Subuidmap, config.Subgidmap)
-	//if err != nil {
-	//	return err
-	//}
-	//storageOpts := storage.DefaultStoreOptions
-	//storageOpts.UIDMap = mappings.UIDMap
-	//storageOpts.GIDMap = mappings.GIDMap
-
 	runtime, err := libpodruntime.GetRuntime(i.Cli)
 	if err != nil {
 		return call.ReplyRuntimeError(err.Error())
@@ -120,7 +112,7 @@ func varlinkCreateToCreateConfig(ctx context.Context, create ioprojectatomicpodm
 	if len(inputCommand) > 0 {
 		// User command overrides data CMD
 		command = append(command, inputCommand...)
-	} else if len(data.ContainerConfig.Cmd) > 0 && len(create.Entrypoint) > 0 {
+	} else if len(data.ContainerConfig.Cmd) > 0 && len(command) == 0 {
 		// If not user command, add CMD
 		command = append(command, data.ContainerConfig.Cmd...)
 	}

pkg/varlinkapi/images.go

@@ -3,12 +3,21 @@ package varlinkapi
 import (
 	"encoding/json"
 	"fmt"
+	"io"
+	"path/filepath"
+	"strings"
+	"time"
 
+	"bytes"
 	"github.com/containers/image/docker"
+	"github.com/containers/image/types"
+	"github.com/docker/go-units"
 	"github.com/opencontainers/image-spec/specs-go/v1"
+	"github.com/pkg/errors"
 	"github.com/projectatomic/buildah"
+	"github.com/projectatomic/buildah/imagebuildah"
 	"github.com/projectatomic/libpod/cmd/podman/libpodruntime"
-	ioprojectatomicpodman "github.com/projectatomic/libpod/cmd/podman/varlink"
+	"github.com/projectatomic/libpod/cmd/podman/varlink"
 	"github.com/projectatomic/libpod/libpod"
 	"github.com/projectatomic/libpod/libpod/image"
 	sysreg "github.com/projectatomic/libpod/pkg/registries"
@@ -28,9 +37,9 @@ func (i *LibpodAPI) ListImages(call ioprojectatomicpodman.VarlinkCall) error {
 	}
 	var imageList []ioprojectatomicpodman.ImageInList
 	for _, image := range images {
-		//size, _:= image.Size(getContext())
 		labels, _ := image.Labels(getContext())
 		containers, _ := image.Containers()
+		size, _ := image.Size(getContext())
 
 		i := ioprojectatomicpodman.ImageInList{
 			Id:          image.ID(),
@@ -38,7 +47,7 @@ func (i *LibpodAPI) ListImages(call ioprojectatomicpodman.VarlinkCall) error {
 			RepoTags:    image.Names(),
 			RepoDigests: image.RepoDigests(),
 			Created:     image.Created().String(),
-			//Size: size,
+			Size:        int64(*size),
 			VirtualSize: image.VirtualSize,
 			Containers:  int64(len(containers)),
 			Labels:      labels,
@@ -48,10 +57,197 @@ func (i *LibpodAPI) ListImages(call ioprojectatomicpodman.VarlinkCall) error {
 	return call.ReplyListImages(imageList)
 }
 
+// GetImage returns a single image in the form of a ImageInList
+func (i *LibpodAPI) GetImage(call ioprojectatomicpodman.VarlinkCall, name string) error {
+	runtime, err := libpodruntime.GetRuntime(i.Cli)
+	if err != nil {
+		return call.ReplyRuntimeError(err.Error())
+	}
+	newImage, err := runtime.ImageRuntime().NewFromLocal(name)
+	if err != nil {
+		return call.ReplyErrorOccurred(err.Error())
+	}
+	labels, err := newImage.Labels(getContext())
+	if err != nil {
+		return err
+	}
+	containers, err := newImage.Containers()
+	if err != nil {
+		return err
+	}
+	size, err := newImage.Size(getContext())
+	if err != nil {
+		return err
+	}
+
+	il := ioprojectatomicpodman.ImageInList{
+		Id:          newImage.ID(),
+		ParentId:    newImage.Parent,
+		RepoTags:    newImage.Names(),
+		RepoDigests: newImage.RepoDigests(),
+		Created:     newImage.Created().String(),
+		Size:        int64(*size),
+		VirtualSize: newImage.VirtualSize,
+		Containers:  int64(len(containers)),
+		Labels:      labels,
+	}
+	return call.ReplyGetImage(il)
+}
+
 // BuildImage ...
-// TODO Waiting for buildah to be vendored into libpod to do this only one
+func (i *LibpodAPI) BuildImage(call ioprojectatomicpodman.VarlinkCall, config ioprojectatomicpodman.BuildInfo) error {
-func (i *LibpodAPI) BuildImage(call ioprojectatomicpodman.VarlinkCall) error {
+	var (
-	return call.ReplyMethodNotImplemented("BuildImage")
+		memoryLimit int64
+		memorySwap  int64
+	)
+
+	runtime, err := libpodruntime.GetRuntime(i.Cli)
+	if err != nil {
+		return call.ReplyRuntimeError(err.Error())
+	}
+	defer runtime.Shutdown(false)
+
+	systemContext := types.SystemContext{}
+	dockerfiles := config.Dockerfile
+	contextDir := ""
+
+	for i := range dockerfiles {
+		if strings.HasPrefix(dockerfiles[i], "http://") ||
+			strings.HasPrefix(dockerfiles[i], "https://") ||
+			strings.HasPrefix(dockerfiles[i], "git://") ||
+			strings.HasPrefix(dockerfiles[i], "github.com/") {
+			continue
+		}
+		absFile, err := filepath.Abs(dockerfiles[i])
+		if err != nil {
+			return errors.Wrapf(err, "error determining path to file %q", dockerfiles[i])
+		}
+		contextDir = filepath.Dir(absFile)
+		dockerfiles[i], err = filepath.Rel(contextDir, absFile)
+		if err != nil {
+			return errors.Wrapf(err, "error determining path to file %q", dockerfiles[i])
+		}
+		break
+	}
+
+	pullPolicy := imagebuildah.PullNever
+	if config.Pull {
+		pullPolicy = imagebuildah.PullIfMissing
+	}
+
+	if config.Pull_always {
+		pullPolicy = imagebuildah.PullAlways
+	}
+
+	format := "oci"
+	if config.Image_format != "" {
+		format = config.Image_format
+	}
+
+	if strings.HasPrefix(format, "oci") {
+		format = imagebuildah.OCIv1ImageFormat
+	} else if strings.HasPrefix(format, "docker") {
+		format = imagebuildah.Dockerv2ImageFormat
+	} else {
+		return call.ReplyErrorOccurred(fmt.Sprintf("unrecognized image type %q", format))
+	}
+
+	if config.Memory != "" {
+		memoryLimit, err = units.RAMInBytes(config.Memory)
+		if err != nil {
+			return call.ReplyErrorOccurred(err.Error())
+		}
+	}
+
+	if config.Memory_swap != "" {
+		memorySwap, err = units.RAMInBytes(config.Memory_swap)
+		if err != nil {
+			return call.ReplyErrorOccurred(err.Error())
+		}
+	}
+
+	output := bytes.NewBuffer([]byte{})
+	commonOpts := &buildah.CommonBuildOptions{
+		AddHost:      config.Add_hosts,
+		CgroupParent: config.Cgroup_parent,
+		CPUPeriod:    uint64(config.Cpu_period),
+		CPUQuota:     config.Cpu_quota,
+		CPUSetCPUs:   config.Cpuset_cpus,
+		CPUSetMems:   config.Cpuset_mems,
+		Memory:       memoryLimit,
+		MemorySwap:   memorySwap,
+		ShmSize:      config.Shm_size,
+		Ulimit:       config.Ulimit,
+		Volumes:      config.Volume,
+	}
+
+	options := imagebuildah.BuildOptions{
+		ContextDirectory: contextDir,
+		PullPolicy:       pullPolicy,
+		Compression:      imagebuildah.Gzip,
+		Quiet:            false,
+		//SignaturePolicyPath:
+		Args: config.Build_args,
+		//Output:
+		AdditionalTags: config.Tags,
+		//Runtime: runtime.
+		//RuntimeArgs: ,
+		OutputFormat:    format,
+		SystemContext:   &systemContext,
+		CommonBuildOpts: commonOpts,
+		Squash:          config.Squash,
+		Labels:          config.Label,
+		Annotations:     config.Annotations,
+		ReportWriter:    output,
+	}
+
+	if call.WantsMore() {
+		call.Continues = true
+	}
+
+	c := build(runtime, options, config.Dockerfile)
+	var log []string
+	done := false
+	for {
+		line, err := output.ReadString('\n')
+		if err == nil {
+			log = append(log, line)
+			continue
+		} else if err == io.EOF {
+			select {
+			case err := <-c:
+				if err != nil {
+					return call.ReplyErrorOccurred(err.Error())
+				}
+				done = true
+			default:
+				if !call.WantsMore() {
+					time.Sleep(1 * time.Second)
+					break
+				}
+				call.ReplyBuildImage(log)
+				log = []string{}
+			}
+		} else {
+			return call.ReplyErrorOccurred(err.Error())
+		}
+		if done {
+			break
+		}
+	}
+	call.Continues = false
+	return call.ReplyBuildImage(log)
+}
+
+func build(runtime *libpod.Runtime, options imagebuildah.BuildOptions, dockerfiles []string) chan error {
+	c := make(chan error)
+	go func() {
+		err := runtime.Build(getContext(), options, dockerfiles...)
+		c <- err
+		close(c)
+	}()
+
+	return c
 }
 
 // CreateImage ...