Adds initial file-based filtering. This does not attempt to be comprehensive, just to get some low-hanging fruit, and to create a blueprint for anyone to follow in the future when adding more filtering. I expect that once this is in place, what will happen is that as we notice cases where PRs are hitting slow or flaky tests that they clearly don't need to, we can incrementally improve the filtering on demand.
Fixes https://github.com/flutter/flutter/issues/136394
Consolidates the code to find all changed file paths into the `PackageLoopingCommand` class that is the base of almost all of the repo tooling commands. This in a preparatory PR for a future change to allow each command to define a list of files or file patterns that definitively *don't* affect that test, so that CI can be smarter about what tests to run (e.g., not running expensive integration tests for README changes).
A side effect of this change is that tests of almost all commands now need a mock `GitDir` instance. This would add a lot of copy/pasted boilerplate to the test setup, and there is already too much of that, so instead this refactors common test setup:
- Creating a memory file system
- Populating it with a packages directory
- Creating a RecordingProcessRunner to mock out process calls
- Creating a mock GitDir that forwards to a RecordingProcessRunner
into a helper method (using records and destructuring to easily return multiple values). While some tests don't need all of these steps, those that don't can easily ignore parts of it, and it will make it much easier to update tests in the future if they need them, and it makes the setup much more consistent which makes it easier to reason about test setup in general.
Prep for https://github.com/flutter/flutter/issues/136394
Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm.
It also removes the `build-examples` step from web integration tests, in some isolated testing:
| platform tests shard | With build-examples | Without build-examples |
|---|-----|-----|
| 1 | 30m | 21m |
| 2 | 13m | 11m |
| 3 | 17m | 10m |
## Issues
* Fixes https://github.com/flutter/flutter/issues/151664
Configures the flutter/packages CI to run web tests using the `canvaskit` renderer, which is the new default for web apps.
It also does some minor changes:
* Skips tests on the web for `package:palette_generator` (cleanup issue created below)
* Makes the `palette_generator` example run on the web. This was used to manually verify that the package still works as expected with canvaskit.
* Tweaks the README of the `pointer_interceptor` example apps, so they match what's happening in there.
## Issues
* Fixes https://github.com/flutter/flutter/issues/143543
Co-authored-by: David Iglesias <ditman@gmail.com>
The logic for handling Dart unit test `test_on` directives was incorrect, causing it to skip packages that required vm testing, even when run in vm mode. This PR:
- Adds the missing tests for false negatives.
- Reworks the logic to explicitly fail for anything that isn't one of the exact patterns we are expecting, to make it much harder to re-introduce a bug like this in the future.
The web tests currently all assume to be using the html renderer. `flutter test` previously erroneously was serving files for the HTML renderer by default (which is not the intended behavior). After this was fixed, the tests started running on the CanvasKit renderer and some of them broke. If we are relying on the html renderer, we should explicitly pass it when running `flutter test`
Adds new LUCI targets in bringup mode to run all possible Dart unit tests in Chrome.
This is a new test (see linked issue), not a port of a Cirrus test, so it involves changes to tooling and packages:
- The tooling now accepts an explicit platform. The default behavior if none is provided is the previous behavior of running in VM for everything but web plugin implementations (since that's convenient locally).
- The tooling now has a basic understanding of `dart_test.yaml` `test_on` directives, to know when to skip.
- Packages that don't support web have opt-out files.
- Packages that do support web but have a few tests that fail on web have those tests opted out.
- Packages that do support web but have a lot of failures on web have temporary opt-out files with TODOs + issue links.
Most of https://github.com/flutter/flutter/issues/128979
Adds aliases for all commands that put the verb first, since currently they are inconsistent which can make it hard to remember (in particular, I often write `check-foo` instead of `foo-check` when running locally, and it fails).
Also does the long-overdue renaming of `test` to `dart-test`, since we now have `native-test`, `custom-test`, etc. `test` continues to work as an alias for individual muscle memory.
