This fixes a regression caused by commit 7e6e267329, unfortunately this
was not caught during review as for some reason this works fine rootless
and only fails as root.
Because we set the systemd log level to notice in order to hide the unit
started/stopped messages to prevent spamming the journal the issue is
that this now also causes systemd to ignore the events we write to
journald as we also send them as info level.
To fix this we simply send health_status events now on notice level. I
decided against sending all events on notice as I think info is fine for
them. Whenever the notice level is right is of course debatable but
given it may contain the unhealthy message I think having this a notice
should be ok.
The main reason this made it through testing is because we do not rely
on the systemd unit to fire healthchecks in the tests as this is flaky.
There is one test were we rely on it though and I added a check there
to make sure events are displayed correctly when trigger via systemd.
Fixes#20342
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When streaming events, prevent returning duplicates after a log rotation
by marking a beginning and an end for rotated events. Before starting to
stream, get a timestamp while holding the event lock. The timestamp
allows for detecting whether a rotation event happened while reading the
log file and to skip all events between the begin and end rotation
event.
In an ideal scenario, we could detect rotated events by enforcing a
chronological order when reading and skip those detected to not be more
recent than the last read event. However, events are not always
_written_ in chronological order. While this can be changed, existing
event files could not be read correctly anymore.
Fixes: #17665
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The new version contains the ginkgolinter, which makes sure the
assertions are more helpful.
Also replace the deprecated os.SEEK_END with io.SeekEnd.
There is also a new `musttag` linter which checks if struct that are
un/marshalled all have json tags. This results in many warnings so I
disabled the check for now. We can reenable it if we think it is worth
it but for now it way to much work to fix all report problems.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
I noticed this while running some things in parallel, podman events
would show events from other users. Because all events are written to
the journal everybody can see them. So when we read the journal we must
filter events for only the current UID.
To reproduce run `podman events` as user then in another window create a
container as root for example. After this patch it will correctly ignore
these events from other users.
[NO NEW TESTS NEEDED] I don't think we can test with two users at the same
time.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When you use podman logs with --until and --follow it should exit after
the requested until time and not keep hanging forever.
To make this work I reworked the code to use the better journald event
reading code for logs as well. this correctly uses the sd_journal API
without having to compare the cursors to find the EOF.
The same problems exists for the k8s-file driver, I will fix this in the
next commit.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Instead of reading the full journal which can be expensive we can seek
based on the time.
If you have a journald with many podman events just compare the time
`time podman events --since 1s --stream=false` with and without this
patch.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When labes map is too big we may get syslog entry truncated.
This breaks JSON parsing making event loading impossible.
[NO NEW TESTS NEEDED]
Signed-off-by: Matej Vasek <mvasek@redhat.com>
When the new `events_container_create_inspect_data` option is enabled in
containers.conf set the `ContainersInspectData` event field for each
container-create event.
The data was requested for the purpose of auditing (e.g., intrusion
detection).
Jira: https://issues.redhat.com/browse/RUN-1702
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This allows tools like Cockpit to know that the pod in question
has also been updated, so they can refresh the list of containers
in the pod.
Fixes#15408
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Package `io/ioutil` was deprecated in golang 1.16, preventing podman from
building under Fedora 37. Fortunately, functionality identical
replacements are provided by the packages `io` and `os`. Replace all
usage of all `io/ioutil` symbols with appropriate substitutions
according to the golang docs.
Signed-off-by: Chris Evich <cevich@redhat.com>
This function changed from public to private which broke the FreeBSD build.
Sadly, adding FreeBSD to the cross build isn't currently possible since
github.com/godbus/dbus relies on cgo on FreeBSD. I've tried to fix this
upstream but my PR is going nowhere - I think this dependency is only
needed for systemd which isn't a thing on FreeBSD so it might be
possible to work around the problem in libpod by making the systemd code
conditional on linux.
[NO NEW TESTS NEEDED]
Signed-off-by: Doug Rabson <dfr@rabson.org>
The current code only sets EventsLogFilePath when the tmp is overwritten
from the db. We should always set the default when no path was set in
containers.conf.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Currently podman events will just fail with `Error: failed to get journal
cursor: failed to get cursor: cannot assign requested address` when the
journal contains zero podman events.
The problem is that we are using the journal accessors wrong. There is no
need to call GetCursor() and compare them manually. The Next() return an
integer which tells if it moved to the next or not. This means the we can
remove GetCursor() which would fail when there is no entry.
This also includes another bug fix. Previously the logic called Next()
twice for the first entry which caused us to miss the first entry.
To reproduce this issue you can run the following commands:
```
sudo journalctl --rotate
sudo journalctl --vacuum-time=1s
```
Note that this will delete the full journal.
Now run podman events and it fails but with this patch it works.
Now generate a single event, i.e. podman pull alpine, and run
podman events --until 1s.
I am not sure how to get a reliable test into CI, I really do not want
to delete the journal and developer or CI systems.
Fixes second part of #15688
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
podman --events-backend none events should return with an error since it
will never be able to actually list events.
Fixes part three of #15688
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
podman --events-backend file events --stream=false should never hang. The
problem is that our tail library will wait for the file to be created
which makes sense when we do not run with --stream=false. To fix this we
can just always create the file when the logger is initialized. This
would also help to report errors early on in case the file is not
accessible.
Fixes part one from #15688
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When running a single podman logs this is not really important since we
will exit when we finish reading the logs. However for the system
service this is very important. Leaking goroutines will cause an
increased memory and CPU ussage over time.
Both the the event and log backend have goroutine leaks with both the
file and journald drivers.
The journald backend has the problem that journal.Wait(IndefiniteWait)
will block until we get a new journald event. So when a client closes
the connection the goroutine would still wait until there is a new
journal entry. To fix this we just wait for a maximum of 5 seconds,
after that we can check if the client connection was closed and exit
correctly in this case.
For the file backend we can fix this by waiting for either the log line
or context cancel at the same time. Currently it would block waiting for
new log lines and only check afterwards if the client closed the
connection and thus hang forever if there are no new log lines.
[NO NEW TESTS NEEDED] I am open to ideas how we can test memory leaks in
CI.
To test manually run a container like this:
`podman run --log-driver $driver --name test -d alpine sh -c 'i=1; while [ "$i" -ne 1000 ]; do echo "line $i"; i=$((i + 1)); done; sleep inf'`
where `$driver` can be either `journald` or `k8s-file`.
Then start the podman system service and use:
`curl -m 1 --output - --unix-socket $XDG_RUNTIME_DIR/podman/podman.sock -v 'http://d/containers/test/logs?follow=1&since=0&stderr=1&stdout=1' &>/dev/null`
to get the logs from the API and then it closes the connection after 1 second.
Now run the curl command several times and check the memory usage of the service.
Fixes#14879
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* Correct spelling and typos.
* Improve language.
Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
[NO NEW TESTS NEEDED]
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Previously, health status events were not being generated at all. Both
the API and `podman events` will generate health_status events.
```
{"status":"health_status","id":"ae498ac3aa6c63db8b69a37583a6eae1a9cefbdbdbeeadcf8e1d66d745f0df63","from":"localhost/healthcheck-demo:latest","Type":"container","Action":"health_status","Actor":{"ID":"ae498ac3aa6c63db8b69a37583a6eae1a9cefbdbdbeeadcf8e1d66d745f0df63","Attributes":{"containerExitCode":"0","image":"localhost/healthcheck-demo:latest","io.buildah.version":"1.26.1","maintainer":"NGINX Docker Maintainers \u003cdocker-maint@nginx.com\u003e","name":"healthcheck-demo"}},"scope":"local","time":1656082205,"timeNano":1656082205882271276,"HealthStatus":"healthy"}
```
```
2022-06-24 11:06:04.886238493 -0400 EDT container health_status ae498ac3aa6c63db8b69a37583a6eae1a9cefbdbdbeeadcf8e1d66d745f0df63 (image=localhost/healthcheck-demo:latest, name=healthcheck-demo, health_status=healthy, io.buildah.version=1.26.1, maintainer=NGINX Docker Maintainers <docker-maint@nginx.com>)
```
Signed-off-by: Jake Correnti <jcorrenti13@gmail.com>
Lint the systemd code and fix the reported problems.
The remoteclient tag is no longer used so I just removed it.
[NO NEW TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Create an auto-update event for each invocation, independent if images
and containers are updated or not. Those events will be indicated in
the events already but users will now know why.
Fixes: #14283
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The completion suggested incorrect values for `podman events --filter
type=` . It should only list types not the event status. Also make sure
to use the constants instead of duplicating the strings.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Update the recent events-log changes to fix the build error.
[NO NEW TESTS NEEDED] since there's no functional change.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add new functions to logfile.go for rotating and truncating
the events log file once the log file and its contents
exceed the maximum size limit while keeping 50% of the
log file's content
Also add tests to verify log rotation and truncation
Signed-off-by: Niall Crowe <nicrowe@redhat.com>
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The hpcloud is not maintained anymore. nxadm is already part of the
dependency chain, so it won't add work for downstream packages.
While testing upcoming changes to the events-file backend, I noticed
that nxadm is shipping required fixes.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Following commit ensures that podman generates a valid event on `podman
container rename` where event specifies that it is a rename event and
container name swtichted to the latest name.
Signed-off-by: Aditya R <arajan@redhat.com>
Standardize on no-trunc through the code.
Alias notruncate where necessary.
Standardize on the man page display of no-trunc.
Fixes: https://github.com/containers/podman/issues/8941
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
we were adding a negative duration in podman events, causing inputs like
-5s to be correct and 5s to be incorrect.
fixes#11158
Signed-off-by: cdoern <cdoern@redhat.com>
When making Exec Cleanup processes mandatory, I introduced a race
wherein attached exec sessions could be cleaned up and removed by
the cleanup process before the frontend had a chance to get their
exit code. Fortunately, we've dealt with this issue before in
containers, and the same solution can be applied here. I added an
event for an exec session's process exiting, `exec_died` (Docker
has an identical event, so this actually improves our
compatibility there) that includes the exit code of the exec
session. If the race happens and the exec session no longer
exists when we go to remove it, pick up exit code from the event
and exit cleanly.
Signed-off-by: Matthew Heon <mheon@redhat.com>
While different filters are applied in conjunction, the same filter (but
with different values) should be applied in disjunction. This allows,
for instance, to query the events of two containers.
Fixes: #10507
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The podman events aren't read until the given timestamp if the
timestamp is in the future. It just reads all events until now
and exits afterwards.
This does not make sense and does not match docker. The correct
behavior is to read all events until the given time is reached.
This fixes a bug where the wrong event log file path was used
when running first time with a new storage location.
Fixes#8694
This also fixes the events api endpoint which only exited when
an error occurred. Otherwise it just hung after reading all events.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
