Using the internal Wait() API over the events API as this is much more
efficient. Reading events will need to read a lot of data otherwise.
For the function here it should work fine and it is even better as it
does not depend on the event logger at all.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Moving from Go module v4 to v5 prepares us for public releases.
Move done using gomove [1] as with the v3 and v4 moves.
[1] https://github.com/KSubedi/gomove
Signed-off-by: Matt Heon <mheon@redhat.com>
There is a problem where our tail code does not handles correctly
partial log lines. This makes podman logs --tail output possibly
incorrect lines when k8s-file is used.
This manifests as flake in CI because partial lines are only sometimes
written, basically always when the output is flushed before writing a
newline.
For our code we must not count partial lines which was already done but
the important thing we must keep reading backwards until the next full
(F) line. This is because all partial (P) lines still must be added to
the full line. See the added tests for details on how the log file looks
like.
While fixing this, I rework the tail logic a bit, there is absolutely no
reason to read the lines in a separate goroutine just to pass the lines
back via channel. We can do this in the same routine.
The logic is very simple, read the lines backwards, append lines to
result and then at the end invert the result slice as tail must return
the lines in the correct order. This more efficient then having to
allocate two different slices or to prepend the line as this would
require a new allocation for each line.
Lastly the readFromLogFile() function wrote the lines back to the log
line channel in the same routine as the log lines we read, this was bad
and causes a deadlock when the returned lines are bigger than the
channel size. There is no reason to allocate a big channel size we can
just write the log lines in a different goroutine, in this case the main
routine were read the logs anyway.
A new system test and unit tests have been added to check corner cases.
Fixes#19545
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When run with --cgroups=split mode (e.g. quadlet) we do not use the a
separate cgroup for the container and just run in the unit cgroup.
When we filter logs we thus must match the unit name.
Added a small test to the quadlet test to make sure it will work.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The passthrough driver is designed for use in systemd units. By default
we can expect systemd to log the output on journald unless the unit sets
differen StandardOutput/StandardError settings.
At the moment podman logs just errors out when the passthrough driver is
used. With this change we will read the journald for the unit messages.
The logic is actually very similar to the existing one, we just need to
change the filter. We now filter by SYSTEMD_UNIT wich equals to the
contianer cgroup, this allows us the actually filter on a per contianer
basis even when multiple contianers are started in the same unit, i.e.
via podman-kube@.service.
The only difference a user will see is that journald will merge
stdout/err into one stream so we loose the separation there.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When you use podman logs with --until and --follow it should exit after
the requested until time and not keep hanging forever.
This fixes the behavior for the k8s-file backend.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When running a single podman logs this is not really important since we
will exit when we finish reading the logs. However for the system
service this is very important. Leaking goroutines will cause an
increased memory and CPU ussage over time.
Both the the event and log backend have goroutine leaks with both the
file and journald drivers.
The journald backend has the problem that journal.Wait(IndefiniteWait)
will block until we get a new journald event. So when a client closes
the connection the goroutine would still wait until there is a new
journal entry. To fix this we just wait for a maximum of 5 seconds,
after that we can check if the client connection was closed and exit
correctly in this case.
For the file backend we can fix this by waiting for either the log line
or context cancel at the same time. Currently it would block waiting for
new log lines and only check afterwards if the client closed the
connection and thus hang forever if there are no new log lines.
[NO NEW TESTS NEEDED] I am open to ideas how we can test memory leaks in
CI.
To test manually run a container like this:
`podman run --log-driver $driver --name test -d alpine sh -c 'i=1; while [ "$i" -ne 1000 ]; do echo "line $i"; i=$((i + 1)); done; sleep inf'`
where `$driver` can be either `journald` or `k8s-file`.
Then start the podman system service and use:
`curl -m 1 --output - --unix-socket $XDG_RUNTIME_DIR/podman/podman.sock -v 'http://d/containers/test/logs?follow=1&since=0&stderr=1&stdout=1' &>/dev/null`
to get the logs from the API and then it closes the connection after 1 second.
Now run the curl command several times and check the memory usage of the service.
Fixes#14879
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
[NO NEW TESTS NEEDED]
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
The backend should not convert partial lines to full log lines. While
this works for most cases it cannot work when the last line is partial
since it will just be lost. The frontend logic can already display
partial lines correctly. The journald driver also works correctly since
it does not such conversion.
Fixes#14458
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The hpcloud is not maintained anymore. nxadm is already part of the
dependency chain, so it won't add work for downstream packages.
While testing upcoming changes to the events-file backend, I noticed
that nxadm is shipping required fixes.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
it allows to pass the current std streams down to the container.
conmon support: https://github.com/containers/conmon/pull/289
[NO TESTS NEEDED] it needs a new conmon.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Fix a dead lock in the file log driver where one goroutine would wait on
the tail to hit EOF but reading is blocked for the function to return.
Fixes: 11461
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
For docker compat include information about available volume, log and
network drivers which should be listed under the plugins key.
Fixes#11265
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Commit 84b55eec2796 attempted to fix a race waiting for the container
died event. Previously, Podman slept for duration of the polling
frequence which I considerred to be a mistake. As it turns out, I was
mistaken since the file logger will, in fact, NOT read until EOF and
then stop logging but stop logging immediately _after_ it woke up.
[NO TESTS NEEDED] as the race condition cannot be hit reliably.
Fixes: #10675
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Fix the suprious "Error: nil" messages. Also add some more context to
logged error messages which makes error sources more obvious.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Fix a race in the k8s-file logs driver. When "following" the logs,
Podman will print the container's logs until the end. Previously,
Podman logged until the state transitioned into something non-running
which opened up a race with the container still running, possibly in
the "stopping" state.
To fix the race, log until we've seen the wait event for the specific
container. In that case, conmon will have finished writing all logs to
the file, and Podman will read it until EOF.
Further tweak the integration tests for testing `logs -f` on a running
container. Previously, the test only checked for one of two lines
stating that there was a race. Indeed the race was in using `run --rm`
where a log file may be removed before we could fully read it.
Fixes: #10596
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Some log tests were duplicated, and some didn't need to be repeated for
every driver. Also, added some comments
Signed-off-by: Ashley Cui <acui@redhat.com>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add a `context.Context` to the log APIs to allow for cancelling
streaming (e.g., via `podman logs -f`). This fixes issues for
the remote API where some go routines of the server will continue
writing and produce nothing but heat and waste CPU cycles.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules. While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.
Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`. The renaming of the imports
was done via `gomove` [1].
[1] https://github.com/KSubedi/gomove
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This incorporates code from PR #6591 and #6614 but does not use
event channels to detect container state and rather uses timers
with a defined wait duration before calling t.StopAtEOF() to
ensure the last log entry is output before a container exits.
The polling interval is set to 250 milliseconds based on polling
interval defined in hpcloud/tail here:
https://github.com/hpcloud/tail/blob/v1.0.0/watch/polling.go#L117
Co-authored-by: Qi Wang <qiwan@redhat.com>
Signed-off-by: jgallucci32 <john.gallucci.iv@gmail.com>
When the container uses journald logging, we don't want to
automatically use the same driver for its exec sessions. If we do
we will pollute the journal (particularly in the case of
healthchecks) with large amounts of undesired logs. Instead,
force exec sessions logs to file for now; we can add a log-driver
flag later (we'll probably want to add a `podman logs` command
that reads exec session logs at the same time).
As part of this, add support for the new 'none' logs driver in
Conmon. It will be the default log driver for exec sessions, and
can be optionally selected for containers.
Great thanks to Joe Gooch (mrwizard@dok.org) for adding support
to Conmon for a null log driver, and wiring it in here.
Fixes#6555
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This fixes a condition when a container is removed while
following the logs and prints an error when the container
is removed forcefully.
Signed-off-by: jgallucci32 <john.gallucci.iv@gmail.com>
Fixes an issue with the previous PR where a container would exit while following logs and the log tail continued to follow. This creates a subroutine which checks the state of the container and instructs the tailLog to stop when it reaches EOF.
Tested the following conditions:
* Tail and follow logs of running container
* Tail and follow logs of stopped container
* Tail and follow logs of running container which exits after some time
Signed-off-by: jgallucci32 <john.gallucci.iv@gmail.com>
Add a journald reader that translates the journald entry to a k8s-file formatted line, to be added as a log line
Note: --follow with journald hasn't been implemented. It's going to be a larger undertaking that can wait.
Signed-off-by: Peter Hunt <pehunt@redhat.com>
when running podman logs on a created container (which has no logs),
podman should return gracefully (like docker) with a 0 return code. if
multiple containers are provided and one is only in the created state
(and no follow is used), we still display the logs for the other ids.
fixes issue #2677
Signed-off-by: baude <bbaude@redhat.com>
add the ability for users to specify more than one container at a time
while using podman logs. If more than one container is being displayed,
podman will also prepend a shortened container id of the container on
the log line.
also, enabled the podman-remote logs command during the refactoring of
the above ability.
fixes issue #2219
Signed-off-by: baude <bbaude@redhat.com>
