18365 Commits

Author SHA1 Message Date
723e879077 Merge pull request #17964 from umohnani8/detach
Add debug to --wait test
2023-03-28 20:55:16 +02:00
62321a2fce Merge pull request #17953 from vrothberg/fix-17859
sqlite: do not `Ping()` after connecting
2023-03-28 20:44:21 +02:00
ac2efdb862 Merge pull request #17949 from baude/hypervset
implement podman machine set for hyperv
2023-03-28 17:41:40 +02:00
c335ff241a Add debug to --wait test
Add a debug line to the wait to test to see which container
is being left behind after the cleaup where the race is happening.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-03-28 10:01:02 -05:00
365131e0b7 Merge pull request #17930 from ygalblum/quadlet-systemd-specifiers
Quadlet - treat paths starting with systemd specifiers as absolute
2023-03-28 16:18:54 +02:00
71c5c6efa0 Merge pull request #17915 from containers/dependabot/go_modules/github.com/onsi/gomega-1.27.5
build(deps): bump github.com/onsi/gomega from 1.27.4 to 1.27.5
2023-03-28 15:24:58 +02:00
cdb5b3e990 sqlite: do not Ping() after connecting
`Ping()` requires the DB lock, so we had to move it into a transaction
to fix #17859. Since we try to access the DB directly afterwards, I
prefer to let that fail instead of paying the cost of a transaction
which would lock the DB for _all_ processes.

[NO NEW TESTS NEEDED] as it's a hard to reproduce race.

Fixes: #17859
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-03-28 11:27:43 +02:00
2cfb6e1c0e Merge pull request #17951 from edsantiago/containers_conf_override
system tests: use CONTAINERS_CONF_OVERRIDE
2023-03-28 04:47:17 -04:00
be9fb7bc8c Merge pull request #17945 from edsantiago/retry_golangcilint
CI: retry the golangci install
2023-03-28 04:43:56 -04:00
da96ff6cc7 Quadlet - treat paths starting with systemd specifiers as absolute
If a path (Yaml, ConfigMap, EnvFile) starts with a systemd path
specifier, treat the path as absolute
Add tests - unit, e2e and bats

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2023-03-28 10:30:33 +03:00
d85c8d7e84 system tests: use CONTAINERS_CONF_OVERRIDE
...not CONTAINERS_CONF. At least for most tests.

Nearly every system test currently using CONTAINERS_CONF=tmpfile
should be using CONTAINERS_CONF_OVERRIDE.

Simple reason: runtime (crun/runc), database_backend (bolt/sqlite),
logger, and other important settings from /etc/c.conf are not
usually written into the tmpfile. Those tests, therefore, are
not running podman as configured on the system.

Much more discussion: #15413

This PR is a prerequisite for enabling sqlite system tests. For
the sake of simplicity and sanity, I choose to submit the sqlite
switch as a separate PR once this passes and merges.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-03-27 15:18:09 -06:00
51d068ecc7 implement podman machine set for hyperv
add the ability to set cpu and processor counts on an existing vm.

Signed-off-by: Brent Baude <bbaude@redhat.com>

[NO NEW TESTS NEEDED]
2023-03-27 16:11:24 -05:00
905dc6de48 Merge pull request #17895 from vrothberg/containers_conf_extra
add CONTAINERS_CONF_OVERRIDE
2023-03-27 16:48:44 -04:00
6222d3bd19 Merge pull request #17894 from baude/hypervloadfromfilesystemfirst
hyperv: lookup machine on local filesystem first
2023-03-27 16:45:20 -04:00
a5621311e3 Merge pull request #17814 from containers/dependabot/go_modules/google.golang.org/protobuf-1.30.0
build(deps): bump google.golang.org/protobuf from 1.29.0 to 1.30.0
2023-03-27 16:41:58 -04:00
ec56f8df84 Merge pull request #17944 from edsantiago/sdnotify_race
system tests: fix racey sdnotify test
2023-03-27 16:38:18 -04:00
c09bd20d27 Merge pull request #17941 from Luap99/fix-IsNotExist-check
fix os.IsNotExist() CI check
2023-03-27 15:26:43 -04:00
dd8574022d CI: retry the golangci install
It flakes once or twice a day:

   VERSION=1.51.1 ./hack/install_golangci.sh
   Installing golangci-lint v1.51.1 into ./bin/golangci-lint
   golangci/golangci-lint info checking GitHub for tag 'v1.51.1'
   golangci/golangci-lint crit unable to find 'v1.51.1' - use 'latest'
      or see https://github.com/golangci/golangci-lint/releases for details

No visibility into why, and no special reason to believe that
retrying five seconds later will work, but it seems worth a try.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-03-27 10:53:17 -06:00
92e0efc5d0 system tests: fix racey sdnotify test
Race introduced in #16709, which changed 'top' to 'true', so
there was only a narrow window in which '.State.ConmonPod'
would be valid. Remove the race.

Fixes: #17882

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-03-27 10:47:49 -06:00
518eafaa34 hyperv: lookup machine on local filesystem first
when looking for a machine, look it up locally first to prevent
accidental collision with non-podman machine vms.  in the cast of
`podman machine ls`, only list podman machines found by json files

Enabled remove with force.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2023-03-27 10:17:06 -05:00
4c72fc26b1 fix os.IsNotExist() CI check
The os.IsNotExist() function comment mentions that new code should use
`errors.Is(err, fs.ErrNotExist)` instead.

The check was already in CI but used the wrong function name (extra s.)

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-03-27 17:03:03 +02:00
78f1ebb80b Merge pull request #17796 from vrothberg/au-errors
auto update: return restart error
2023-03-27 09:23:40 -04:00
8bd9109fb8 Merge pull request #17917 from mheon/fix_17905
Ensure that SQLite state handles name-ID collisions
2023-03-27 07:48:37 -04:00
30619bba68 Merge pull request #17902 from mheon/fix_sqlite_validate_unique
Fix a race around SQLite DB config validation
2023-03-27 05:10:32 -04:00
7daab31f1f Ensure that SQLite state handles name-ID collisions
If a container with an ID starting with "db1" exists, and a
container named "db1" also exists, and they are different
containers - if I run `podman inspect db1` the container named
"db1" should be inspected, and there should not be an error that
multiple containers matched the name or id "db1". This was
already handled by BoltDB, and now is properly managed by SQLite.

Fixes #17905

Signed-off-by: Matt Heon <mheon@redhat.com>
2023-03-24 15:09:25 -04:00
67c98ec5de Merge pull request #17916 from Luap99/mac-helper-error
macos pkginstaller: fix regression which caused the installer to fail
2023-03-24 13:39:41 -04:00
64ba82601a macos pkginstaller: do not fail when podman-mac-helper fails
Make sure we can install podman even when the podman-mac-helper install
command fails. This used to be the behavior but commit bae07b6ea2 caused
the regression because the binary now returns 1 as exit code on errors.

[NO NEW TESTS NEEDED] I am not sure if we can test the install step in
CI.

Fixes #17910

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-03-24 16:05:56 +01:00
4f3cbf15cf podman-mac-helper: install: do not error if already installed
Since commit bae07b6ea2 we exit with 1 one errors. This caused problem
for the mac installer which fails because of the error now.
If the helper is already installed do not treat this as hard error and
just log it instead.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-03-24 15:34:30 +01:00
ade46ae6a5 build(deps): bump github.com/onsi/gomega from 1.27.4 to 1.27.5
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.27.4 to 1.27.5.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.27.4...v1.27.5)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-24 13:00:28 +00:00
f9beb0db09 Merge pull request #17896 from willmerae/willmerae-patch-1
[CI:DOCS] minor grammar fix in `--volume` description
2023-03-24 08:55:32 -04:00
8720235164 Merge pull request #17890 from Luap99/stats-compat
stats compat API: return "id" lowercase
2023-03-24 04:27:50 -04:00
e061cb968c Fix a race around SQLite DB config validation
The DB config is a single-row table, and the first Podman process
to run against the database creates it. However, there was a race
where multiple Podman processes, started simultaneously, could
try and write it. Only the first would succeed, with subsequent
processes failing once (and then running correctly once re-ran),
but it was happening often in CI and deserves fixing.

[NO NEW TESTS NEEDED] It's a CI flake fix.

Signed-off-by: Matt Heon <mheon@redhat.com>
2023-03-23 19:48:27 -04:00
c5fc44582f add CONTAINERS_CONF_OVERRIDE
Add yet another environment variable for loading containers.conf.
When CONTAINERS_CONF_OVERRIDE is set, the specified config file
will be loaded last - even when CONTAINERS_CONF is set.

This mechanism is needed to preserve system settings and other
environment variables.  Setting CONTAINERS_CONF will load only
the specified config file and ignore all system and user paths.
That makes testing hard as many Podman tests use CONTAINERS_CONF
for testing.

The intended use of CONTAINERS_CONF_OVERRIDE is to set it during tests
and point it to a specific configuration of Podman (e.g., netavark with
sqlite backend).

Similar needs have popped up talking to users in the automotive and
high-performance computing space.  In a way, such a setting allows for
specifying a specific "flavor" of Podman while preserving all existing
settings on the system.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-03-23 15:16:59 +01:00
5f86fae71f vendor containers/common@main
Also adjust the e2e tests to account for SYS_CHROOT having made it back
to the default caps.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-03-23 15:16:57 +01:00
cb18a33f63 Merge pull request #17889 from vrothberg/sqlite-fixes
More sqlite fixes
2023-03-23 09:54:59 -04:00
6825740fc0 docs: minor grammar fix in --volume description
Signed-off-by: Alex Willmer <alex.willmer@cgi.com>
2023-03-23 11:58:06 +00:00
1051456c04 Merge pull request #17887 from chuanchang/test_coverage_for_bz2177611
test: podman checkpoint/restore the latest container
2023-03-22 11:10:45 -04:00
b31d9e15f2 sqlite: do not use shared cache
SQLite developers consider it a misfeature [1], and after turning it on,
we saw a new set of flakes.  Let's turn it off and trust the developers
[1] that WAL mode is sufficient for our purposes.

Turning the shared cache off also makes the DB smaller and faster.

[NO NEW TESTS NEEDED]

[1] https://sqlite.org/forum/forumpost/1f291cdca4

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-03-22 15:44:38 +01:00
9be4a525f4 test: podman checkpoint/restore the latest container
Signed-off-by: Alex Jia <chuanchang.jia@gmail.com>
2023-03-22 22:00:37 +08:00
5185e7e39e stats compat API: return "id" lowercase
We use the same endpoint for libpod and docker compat API. However as
reported docker returns "id" lowercase. Because we cannot break the
libpod API right now keep the output for the libpod endpoint and only
change the docker one.

To do so simply use two types that we can cast with different JSON tags.

Fixes #17869

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-03-22 14:36:35 +01:00
6b9f3140fa Merge pull request #17874 from mheon/sqlite_fixes
Sqlite fixes
2023-03-22 08:13:29 -04:00
67431a7439 Merge pull request #17879 from rhatdan/docs
Run make codespell
2023-03-21 18:37:14 -04:00
5f274e45f2 Run make codespell
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-03-21 16:00:54 -04:00
3925cd653b Drop SQLite max connections
The SQLite transaction lock Valentin found is (slightly) faster.
So let's go with that.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2023-03-21 14:20:34 -04:00
3820554aa9 Merge pull request #17735 from cevich/bench_stuff
Cirrus: Store podman machine benchmark data
2023-03-21 14:05:54 -04:00
d9cbfdde73 Merge pull request #17861 from edsantiago/bump_min_nfiles
New ulimit test: bump up minimum nfiles
2023-03-21 13:28:28 -04:00
0fbc325156 sqlite: set connection attributes on open
The symptoms in #17859 indicate that setting the PRAGMAs in individual
EXECs outside of a transaction can lead to concurrency issues and
failures when the DB is locked.  Hence set all PRAGMAs when opening
the connection.  Move them into individual constants to improve
documentation and readability.

Further make transactions exclusive as #17859 also mentions an error
that the DB is locked during a transaction.

[NO NEW TESTS NEEDED] - existing tests cover the code.

Fixes: #17859
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>

<MH: Cherry-picked on top of my branch>

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2023-03-21 12:51:31 -04:00
5b1345c772 Merge pull request #17872 from edsantiago/rhel8_rootless_journal_skip_blah_blah
quadlet tests: skip on RHEL8 rootless
2023-03-21 11:08:06 -04:00
9f0e0e8331 Fix database locked errors with SQLite
I was searching the SQLite docs for a fix, but apparently that
was the wrong place; it's a common enough error with the Go
frontend for SQLite that the fix is prominently listed in the API
docs for go-sqlite3. Setting cache mode to 'shared' and using a
maximum of 1 simultaneous open connection should fix.

Performance implications of this are unclear, but cache=shared
sounds like it will be a benefit, not a curse.

[NO NEW TESTS NEEDED] This fixes a flake with concurrent DB
access.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2023-03-21 09:57:56 -04:00
cf5df5b805 quadlet tests: skip on RHEL8 rootless
skip in setup() if journald unavailable.

To be pedantic, this is overkill: some quadlet tests pass
because they don't run journald. Too bad.

Also skip a play-kube test that requires journal

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-03-21 07:18:14 -06:00