The analysis options have gotten behind; this re-syncs to the current state of flutter/flutter. For options that are non-trivial to enable, either because they are non-trivial to fix, or touch a very large number of files, they are locally disabled with clear "LOCAL CHANGE" markers so that it's obvious where we are out of sync. For options that are simple to resolve, they are enabled in the PR.
Part of https://github.com/flutter/flutter/issues/76229
Improve package targeting:
- `--run-on-changed-packages` now includes only changed packages in a federated plugin, not all packages. Include all packages isn't useful since (without changes that would cause them to be included anyway) package dependencies are not path-based between packages.
- `--packages` now allows specifying package names of federated plugins. E.g., `--packages=path_provider_ios` will now work, instead of requiring `--packages=path_provider/path_provider_ios`. The fully qualified form still works as well (and is still needed for app-facing packages to disambiguate from the plugin group).
Fixes https://github.com/flutter/flutter/issues/94618
Adds a new command that adds `dependency_overrides` to any packages in the repository that depend on a list of target packages, including an option to target packages that will publish a non-breaking change in a given diff.
Adds a new CI step that uses the above in conjunction with a new `--run-on-dirty-packages` to adjust the dependencies of anything in the repository that uses a to-be-published package and then re-run analysis on just those packages. This will allow us to catch in presubmit any changes that are not breaking from a semver standpoint, but will break us due to our strict analysis in CI.
Fixes https://github.com/flutter/flutter/issues/89862
The currently documented repository policy is to:
- require version updates for packages changes that don't meet specific exemptions, and
- require CHANGELOG changes for essentially all changes.
This adds tooling that enforces that policy, with a mechanism for overriding it via PR descriptions, to avoid cases where they are accidentally omitted without reviewers catching it.
In order to facilitate testing (which require mocking another `git` command), this also updates the existing `version-check` tests:
- Replaces the custom git result injection/validating with the newer bind-to-process-mocks approach that is now used in the rest of the tool tests.
- Fixes some tests that were only checking for `ToolExit` to also check the error output, in order to ensure that failure tests are not accidentally passing for the wrong reason (as is being done in general as tests in the tooling are updated).
Fixes https://github.com/flutter/flutter/issues/93790
Adds a `--log-timing` flag that can be passed to cause package-looping
commands to log relative start time and per-package elapsed time, to
help with future efforts to improve CI times (e.g., finding unusually
slow packages, or designing changes to sharding).
Adds this flag to the CI scripts to enable timing there.
Treat `main` the same as `master` for branch-based switching, in
preparation for switching the branch names in Flutter repositories.
Also updates all of the tests that used `master` as the explicit base to
use `main` instead; what the tests use is arbitrary, so they can be
switched now even though the repo itself hasn't switched.
Part of https://github.com/flutter/flutter/issues/90476
pub.dev deducts points for having a pubspec.yaml `description` that is too short or too long; several of our plugins are losing points on this. To ensure that we are following—and modeling—best practices, this adds a check that our `description` fields meet pub.dev expectations.
Fixes our existing violations. Two are not published even though this only takes effect once published:
- camera: We change this plugin pretty frequently, so this should go out soon without adding a release just for this trivial issue.
- wifi_info_flutter: This is deprecated, so we don't plan to release it. It has to be fixed to allow the tool change to land though.
The build-examples command was filtering what it attempted to build by plugin platform, which means it never does anything for non-plugin packages. flutter/packages has steps that run this command, which suggests it used to work and regressed at some point, but nobody noticed; this will re-enable those builds so that we are getting CI coverage that the examples in flutter/packages build.
Mostly fixes https://github.com/flutter/flutter/issues/88435 (needs a flutter/packages tool pin roll to pick this up)
Eliminates the remaining logic from tool_runner.sh, completing the goal of migrating repository tooling off of bash (both to make maintenance easier, and to better support Windows both locally and in CI). Its branch-based logic is now part of the tool itself, via a new `--packages-for-branch` flag (which is hidden in help since it's only useful for CI).
Part of https://github.com/flutter/flutter/issues/86113
Implements support for `--windows` in `native-test`, for unit tests only. The structure of the new code has most of the new functionality in a generic utility for running GoogleTest test binaries, so that it can be trivially extended to Linux support in a follow-up once the Linux test PoC has landed.
This runs the recently-added `url_launcher_windows` unit test. However, it's not yet run in CI since it needs LUCI bringup; that will be done one this support is in place.
Requires new logic to check if a plugin contains native code, and some new test utility plumbing to generate plugins whose pubspecs indicate that they only contain Dart code to test it, to allow filtering filtering out the FFI-based Windows plugins.
Part of flutter/flutter#82445
This allows building UWP plugin examples with `build-examples --winuwp`. As with previous pre-stable-template desktop support, this avoids the issue of unstable app templates by running `flutter create` on the fly before trying to build, so a template that will bitrot doesn't need to be checked in.
Also adds no-op "support" for `drive-examples --winuwp`, with warnings about it not doing anything. This is to handle the fact that the LUCI recipe is shared between Win32 and UWP, and didn't conditionalize `drive`. Rather than change that, then change it back later, this just adds the no-op support now (since changing the tooling is much easier than changing LUCI recipes currently).
This required some supporting tool changes:
- Adds the ability to check for the new platform variants in a pubspec
- Adds the ability to write test pubspecs that include variants, for testing
Part of https://github.com/flutter/flutter/issues/82817
Packages are the primary conceptual object in the tool, but currently they are represented simply as Directory (or occasionally a path string). This introduces an object for packages and:
- moves a number of existing utility methods into it
- sweeps the code for the obvious cases of using `Directory` to represent a package, especially in method signatures and migrates them
- notes a few places where we should migrate later, to avoid ballooning the size of the PR
There are no doubt other cases not caught in the sweep, but this gives us a foundation both for new code, and to migrate incrementally toward as we find existing code that was missed.
The mock process runner used in most of the tests had poor handling of
stdout/stderr:
- By default it would return the `List<int>` output from the mock
process, which was never correct since the parent process runner
interface always sets encodings, thus the `dynamic` should always be
of type `String`
- The process for setting output on a `MockProcess` was awkward, since
it was based on a `Stream<Lint<int>>`, even though in every case what
we actually want to do is just set the full output to a string.
- A hack was at some point added (presumably due to the above issues)
to bypass that flow at the process runner level, and instead return a
`String` set there. That meant there were two ways of setting output
(one of which that only worked for one of the ways of running
processes)
- That hack wasn't updated when the ability to return multiple mock
processes instead of a single global mock process was added, so the
API was even more confusing, and there was no way to set different
output for different processes.
This changes the test APIs so that:
- `MockProcess` takes stdout and stderr as strings, and internally
manages converting them to a `Stream<List<int>>`.
- `RecordingProcessRunner` correctly decodes and returns the output
streams when constructing a process result.
It also removes the resultStdout and resultStderr hacks, as well as the
legacy `processToReturn` API, and converts all uses to the new
structure, which is both simpler to use, and clearly associates output
with specific processes.
Adds a new `lint-android` command to run `gradlew lint` on Android plugins.
Also standardizes the names of the Cirrus tasks that run all the build and platform-specific (i.e., not Dart unit test) tests for each platform, as they were getting unnecessarily long and complex in some cases.
Fixes https://github.com/flutter/flutter/issues/87071
Makes commands that use the package-looping base command track and
report exclusions. This will make it much easier to debug/audit
situations where tests aren't running when expected (e.g., when enabling
a new type of test for a package that previously had to be explicitly
excluded from that test to avoid failing for having no tests, but
forgetting to remove the package from the exclusion list).
Also fixes a latent issue with using different exclusion lists on
different commands in a single CI task when using sharding could cause
unexpected failures due to different sets of plugins being included for
each step (e.g., build+drive with an exclude list on drive could
potentially try to drive a plugin that hadn't been built in that shard)
by sharding before filtering out excluded packages.
Adds testing for sharding in general, as there was previously none.
Currently the tool accepts `--custom-analysis` to allow a list of packages for which custom `analysis_options.yaml` are allowed, and `--exclude` to exclude a set of packages when running a command against all, or all changed, packages. This results in these exception lists being embedded into CI configuration files (e.g., .cirrus.yaml) or scripts, which makes them harder to maintain, and harder to re-use in other contexts (local runs, new CI systems).
This adds support for both flags to accept paths to YAML files that contain the lists, so that they can be maintained separately, and with inline comments about the reasons things are on the lists.
Also updates the CI to use this new support, eliminating those lists from `.cirrus.yaml` and `tool_runner.sh`
Fixes https://github.com/flutter/flutter/issues/86799
To prep for making a combined command to run native tests across different platforms, rework `xctest`:
- Split analyze out into a new `xcode-analyze` command:
- Since the analyze step runs a new build over everything with different flags, this is only a small amount slower than the combined version
- This makes the logic easier to follow
- This allows us to meaningfully report skips, to better notice missing tests.
- Add the ability to target specific test bundles (RunnerTests or RunnerUITests)
To share code between the commands, this extracts a new `Xcode` helper class.
Part of https://github.com/flutter/flutter/issues/84392 and https://github.com/flutter/flutter/issues/86489
The purpose of this PR is to make running all unit tests on Windows pass (vs failing a large portion of the tests as currently happens). This does not mean that the commands actually work when run on Windows, or that Windows support is tested, only that it's possible to actually run the tests themselves. This is prep for actually supporting parts of the tool on Windows in future PRs.
Major changes:
- Make the tests significantly more hermetic:
- Make almost all tools take a `Platform` constructor argument that can be used to inject a mock platform to control what OS the command acts like it is running on under test.
- Add a path `Context` object to the base command, whose style matches the `Platform`, and use that almost everywhere instead of the top-level `path` functions.
- In cases where Posix behavior is always required (such as parsing `git` output), explicitly use the `posix` context object for `path` functions.
- Start laying the groundwork for actual Windows support:
- Replace all uses of `flutter` as a command with a getter that returns `flutter` or `flutter.bat` as appropriate.
- For user messages that include relative paths, use a helper that always uses Posix-style relative paths for consistent output.
This bumps the version since quite a few changes have built up, and having a cut point before starting to make more changes to the commands to support Windows seems like a good idea.
Part of https://github.com/flutter/flutter/issues/86113
Most of the tool operates on packages in general, and the targetting
done currently by the `--plugins` flag is not actually restricted to
plugins, so this makes the name less confusing.
Part of https://github.com/flutter/flutter/issues/83413
To support this command's --machine flag, which moves all normal output into a field in a JSON struct, adds a way of capturing output and providing it to the command subclass on completion.
Part of flutter/flutter#83413
- Updates the remaining tests (other than one that still needs to be
converted to the new base command, which will be fixed then) that
aren't using runCapturingPrint to do so to reduce test log spam.
- Simplifies and standardizes the matcher used for ToolExit in tests.
Add a summary to the end of successful runs for everything using the new looping base command, similar to what we do for summarizing failures. This will make it easy to manually check results for PRs that we know should be changing the set of run packages (adding a new package, adding a new test type to a package, adding a new test type to the tool), as well as spot-checking when we see unexpected results (e.g., looking back and why a PR didn't fail CI when we discover that it should have).
To support better surfacing skips, this restructures the return value of `runForPackage` to have "skip" as one of the options. As a result of it being a return value, packages that used `printSkip` to indicate that *parts* of the command were being skipped have been changed to no longer do that.
Fixes https://github.com/flutter/flutter/issues/85626
Most of our commands are generally of the form:
```
for (each plugin as defined by the tool flags)
check some things for success or failure
print a summary all of the failing things
exit non-zero if anything failed
```
Currently all that logic not consistent, having been at various points copied and pasted around, modified, in some cases rewritten. There's unnecessary boilerplate in each new command, and there's unnecessary variation that makes it harder both to maintain the tool, and to consume the test output:
- There's no standard format for separating each plugin's run to search within a log
- There's no standard format for the summary at the end
- In some cases commands have been written to ToolExit on failure, which means we don't actually get the rest of the runs
This makes a new base class for commands that follow this structure to use, with shared code for all the common bits. This makes it harder to accidentally write new commands incorrectly, easier to maintain the code, and lets us standardize output so that searching within large logs will be easier.
This ports two commands over as a proof of concept to demonstrate that it works; more will be converted in follow-ups.
Related to https://github.com/flutter/flutter/issues/83413
common.dart is a large-and-growing file containing all shared code,
which makes it hard to navigate. To make maintenance easier, this splits
the file (and its test file) into separate files for each major
component or category.
