I was unsure if we wanted to add a new platform option for this or add a `--wasm` flag to the command. Adding a `--wasm` flag seems more complicated since we would probably want to add extra handling around proper use of the flag, whereas adding a `web-wasm` platform should require no additional checks.
This is in preparation for https://github.com/flutter/flutter/issues/151664
Configures the flutter/packages CI to run web tests using the `canvaskit` renderer, which is the new default for web apps.
It also does some minor changes:
* Skips tests on the web for `package:palette_generator` (cleanup issue created below)
* Makes the `palette_generator` example run on the web. This was used to manually verify that the package still works as expected with canvaskit.
* Tweaks the README of the `pointer_interceptor` example apps, so they match what's happening in there.
## Issues
* Fixes https://github.com/flutter/flutter/issues/143543
Co-authored-by: David Iglesias <ditman@gmail.com>
The web tests currently all assume to be using the html renderer. `flutter test` previously erroneously was serving files for the HTML renderer by default (which is not the intended behavior). After this was fixed, the tests started running on the CanvasKit renderer and some of them broke. If we are relying on the html renderer, we should explicitly pass it when running `flutter test`
Expand the multiple-test-files workaround in `drive-examples` from macOS to all of the desktop platforms, now that we know it's not macOS-specific.
See https://github.com/flutter/flutter/issues/135673
Endorses the macOS implementation of `video_player`, updating the constraint of the existing `video_player_avfoundation` dependency to ensure macOS support is present. Adds macOS support to the example, and adds macOS discussion to the README.
This is the first macOS plugin with multiple integration test files, which turned up a bug in `flutter test`, so this also includes a repo tooling change to work around that bug for now by running each integration test file separately on macOS (as we already have to do with `flutter drive` on web).
Fixes https://github.com/flutter/flutter/issues/41688
Adds web platform tests for LUCI in bringup mode.
The Cirrus version of the test starts `chromedriver` at the beginning and just leaves it running, which is fine since it's a fresh docker image each run. For LUCI we don't want that behavior though, so instead this adds tooling support for running chromedriver as part of running the integration tests, controllable with a flag. This will also be useful for running locally.
Part of https://github.com/flutter/flutter/issues/114373
For non-web platforms, `flutter drive` is deprecated. This switches those platforms from `flutter drive` to `flutter test`. This makes the logic to check for test driver files web-specific, since `flutter test` doesn't require a driver.
Removes support for the legacy test-in-test_driver-directory structure, which is no longer used anywhere in the repository.
Also includes a minor drive-by fix to the way we do process output, noticed while manually testing. Instead of adding all stdout, and only then adding all stderr, this adds both then waits for both, which should allow interleaving of stdout and stderr in the terminal.
Fixes https://github.com/flutter/flutter/issues/105634
Switches the web tests from using the version of Chrome installed by the Dockerfile, which is whatever happened to be stable when the image is generated, and thus not hermetic, to a pinned version of Chromium. This uses a slightly modified version of the script that is already used for flutter/packages.
Since Chromium doesn't support mp4 playback, this updates the `video_player` integration tests to use WebM on web instead, to avoid having all the tests fail in CI.
Part of https://github.com/flutter/flutter/issues/84712
This allows building UWP plugin examples with `build-examples --winuwp`. As with previous pre-stable-template desktop support, this avoids the issue of unstable app templates by running `flutter create` on the fly before trying to build, so a template that will bitrot doesn't need to be checked in.
Also adds no-op "support" for `drive-examples --winuwp`, with warnings about it not doing anything. This is to handle the fact that the LUCI recipe is shared between Win32 and UWP, and didn't conditionalize `drive`. Rather than change that, then change it back later, this just adds the no-op support now (since changing the tooling is much easier than changing LUCI recipes currently).
This required some supporting tool changes:
- Adds the ability to check for the new platform variants in a pubspec
- Adds the ability to write test pubspecs that include variants, for testing
Part of https://github.com/flutter/flutter/issues/82817
The mock process runner used in most of the tests had poor handling of
stdout/stderr:
- By default it would return the `List<int>` output from the mock
process, which was never correct since the parent process runner
interface always sets encodings, thus the `dynamic` should always be
of type `String`
- The process for setting output on a `MockProcess` was awkward, since
it was based on a `Stream<Lint<int>>`, even though in every case what
we actually want to do is just set the full output to a string.
- A hack was at some point added (presumably due to the above issues)
to bypass that flow at the process runner level, and instead return a
`String` set there. That meant there were two ways of setting output
(one of which that only worked for one of the ways of running
processes)
- That hack wasn't updated when the ability to return multiple mock
processes instead of a single global mock process was added, so the
API was even more confusing, and there was no way to set different
output for different processes.
This changes the test APIs so that:
- `MockProcess` takes stdout and stderr as strings, and internally
manages converting them to a `Stream<List<int>>`.
- `RecordingProcessRunner` correctly decodes and returns the output
streams when constructing a process result.
It also removes the resultStdout and resultStderr hacks, as well as the
legacy `processToReturn` API, and converts all uses to the new
structure, which is both simpler to use, and clearly associates output
with specific processes.
The purpose of this PR is to make running all unit tests on Windows pass (vs failing a large portion of the tests as currently happens). This does not mean that the commands actually work when run on Windows, or that Windows support is tested, only that it's possible to actually run the tests themselves. This is prep for actually supporting parts of the tool on Windows in future PRs.
Major changes:
- Make the tests significantly more hermetic:
- Make almost all tools take a `Platform` constructor argument that can be used to inject a mock platform to control what OS the command acts like it is running on under test.
- Add a path `Context` object to the base command, whose style matches the `Platform`, and use that almost everywhere instead of the top-level `path` functions.
- In cases where Posix behavior is always required (such as parsing `git` output), explicitly use the `posix` context object for `path` functions.
- Start laying the groundwork for actual Windows support:
- Replace all uses of `flutter` as a command with a getter that returns `flutter` or `flutter.bat` as appropriate.
- For user messages that include relative paths, use a helper that always uses Posix-style relative paths for consistent output.
This bumps the version since quite a few changes have built up, and having a cut point before starting to make more changes to the commands to support Windows seems like a good idea.
Part of https://github.com/flutter/flutter/issues/86113
* [flutter_plugin_tools] Work around banner in drive-examples
Strip off any unexpected output before parsing `flutter devices
--machine` output. Works around a bug in the Flutter tool that can
result in banners being printed in `--machine` mode.
Fixes https://github.com/flutter/flutter/issues/86052
Many commands had insufficient failure testing. This adds new tests that
ensure that for every Process call, at least one test fails if a failure
from that process were ignored (with the exception of calls in the
`publish` command, which has a custom process mocking system, so was out
of scope here; it already has more coverage than most tests did though.)
For a few existing failure tests, adds output checks to ensure that they
are testing for the *right* failures.
Other changes:
- Adds convenience constructors to MockProcess for the common cases of a
mock process that just exits with a 0 or 1 status, to reduce test
verbosity.
- Fixes a few bugs that were found by the new tests.
- Minor test cleanup, especially cases where a mock process was being
set up just to make all calls succeed, which is the default as of
recent changes.
The intent of package-looping commands is that we always want to run them for all the targeted packages, accumulating failures, and then report them all at the end, so we don't end up with the problem of only finding errors one at a time. However, some of them were using `exitOnError: true` for the underlying commands, causing the first error to be fatal to the test.
This PR:
- Removes all use of `exitOnError: true` from the package-looping commands. (It's still used in `format`, but fixing that is a larger issue, and less important due to the way `format` is currently structured.)
- Fixes the mock process runner used in our tests to correctly simulate `exitOrError: true`; the fact that it didn't was hiding this problem in test (e.g., `drive-examples` had a test that asserted that all failures were summarized correctly, which passed because in tests it was behaving as if `exitOnError` were false)
- Adjusts the mock process runner to allow setting a list of mock result for a specific executable instead of one result for all calls to anything, in order to fix some tests that were broken by the two changes above and unfixable without this (e.g., a test of a command that had been calling one executable with `exitOnError: true` then another with ``exitOnError: false`, where the test was asserting things about the second call failing, which only worked because the first call's failure wasn't actually checked). To limit the scope of the PR, the old method of setting a single result for all calls is still supported for now as a fallback.
- Fixes the fact that the mock `run` and `runAndStream` had opposite default exit code behavior when no mock process was set (since that caused me a lot of confusion while fixing the above until I figured it out).
Add a summary to the end of successful runs for everything using the new looping base command, similar to what we do for summarizing failures. This will make it easy to manually check results for PRs that we know should be changing the set of run packages (adding a new package, adding a new test type to a package, adding a new test type to the tool), as well as spot-checking when we see unexpected results (e.g., looking back and why a PR didn't fail CI when we discover that it should have).
To support better surfacing skips, this restructures the return value of `runForPackage` to have "skip" as one of the options. As a result of it being a return value, packages that used `printSkip` to indicate that *parts* of the command were being skipped have been changed to no longer do that.
Fixes https://github.com/flutter/flutter/issues/85626
Significantly restructures drive-examples:
- Migrates it to the new package-looping base command
- Enforces that only one platform is passed, since in practice multiple platforms never actually worked. (The logic is structured so that it will be easy to enable multi-platform if `flutter drive` gains multi-platform support.)
- Fixes the issue where `--ios` and `--android` were semi-broken, by doing explicit device targeting for them rather than relying on the default device being the right kind
- Extracts much of the logic to helpers so it's easier to understand the flow
- Removes support for a legacy integration test file structure that is no longer used
- Adds more test coverage; previously no failure cases were actually tested.
Fixes https://github.com/flutter/flutter/issues/85147
Part of https://github.com/flutter/flutter/issues/83413
Rather than taking a list of list of path elements, just accept a list
of Posix-style paths. In practice, the API was already being partially
used that way.
common.dart is a large-and-growing file containing all shared code,
which makes it hard to navigate. To make maintenance easier, this splits
the file (and its test file) into separate files for each major
component or category.
Eliminates the global test filesystem and global test packages directory, in favor of local versions. This guarantees that each test runs with a clean filesystem state, rather than relying on cleanup. It also simplifies understanding the tests, since everything is done via params and return values instead of needing to know about the magic global variables and which methods mutate them.
- Replaces most explicit use of `fileSystem` with path construction using the `child*` utility methods
- Removes explicit passing of a filesystem to the commands; we're already passing a `Directory` for the
root where the tool operates, and we should never be using a different filesystem than that directory's
filesystem, so passing it was both redundant, and a potential source of test bugs.
- Updates dependencies to null-safe versions
- Migrates common.dart (which doesn't depend on anything)
- Migrates common_tests.dart and its one dependency, utils.dart
- Adds build_runner for Mockito mock generation
- Adds a new utility methods for getting arguments that handle both the casting and the removal of nullability to address a common problematic pattern while migrating code.
- Converts all files, not just the migrated ones, to those new helpers.
Migrating common.dart and utils.dart should unblock a command-by-command migration to null safety.
Reverts the separate of podspect lints into a step that doesn't do a Flutter upgrade
(https://github.com/flutter/plugins/pull/3700) because without that step we had a
version of Dart too old to run null-safe tooling.
First step of https://github.com/flutter/flutter/issues/81912
Many of our integration tests weren't actually being run on CI because they were in the wrong place and/or missing the driver file, and it seems we've just never noticed. This makes a number of changes:
- Ensures that all packages with integration tests also have a driver file in the right location.
- Also standardizes the format of those files, as the boilerplate `main()` is available in `integration_test`.
- Ensures that all integration_test directories are in the right place.
- In a couple of places, removes a duplicate copy of the integration test file.
- Makes it an error for a plugin that's not excluded to not have integration tests, so this can't easily happen again.
- Adds logging of what's being run and skipped, so if something does go wrong in the future it's easy to determine what from the logs.
- Excludes `*_platform_interface` since the logging was (potentially confusingly) reporting that they were skipped because they don't support the current platform. Skipping them is correct, just not for that reason.
- Excludes the plugins that currently have no integration tests, with references to issues about adding them.
Fixes https://github.com/flutter/flutter/issues/81929
Removes the legacy analysis options override and fixes all resulting issues. This is a combination of dart fix and manual changes (mostly mechanical, but some small restructuring to address warnings more cleanly, such as creating typed structs from args when they are used repeatedly to avoid repeated casting, or making things that were unnecessarily public private).
One small opportunistic extra cleanup is that the handling of null-safety prerelease versions is removed, as any new plugin would be written null-safe from the start, so we no longer need to allow those versions.
Part of flutter/flutter#76229
This change enables the integration_tests of the following packages to run in Cirrus CI:
* google_sign_in_web
* connectivity_for_web
* google_maps_flutter_web
* url_launcher_web
Standardizes all first-party copyrights on a single year, as is done in flutter/flutter and flutter/engine. All code now uses 2013, which is the earliest year that was in any existing copyright notice.
The script checks now enforce the exact format of first-party licenses and copyrights.
Fixesflutter/flutter#78448
In all copyright messages (and in the Xcode project organization name) standardize on "The Flutter Authors", adding "The Chromium Authors" to the Flutter AUTHORS list. This reduces inconsistency in the copyright lines in this repository, moving closer to a single consistent copyright+license (as in flutter/engine and flutter/flutter)
Updates the validation script to no longer accept "The Chromium Authors" or "the Chromium project authors" in first-party code.
Adds a new CI check that all code files have a copyright+license block (and that it's one we are expecting to see).
Fixes the ~350 files (!) that did not have them. This includes all of the files in the .../example/ directories, following the example of flutter/flutter. (This does mean some manual intervention will be needed when generating new example directories in the future, but it's one-time per example.)
Also standardized some variants that used different line breaks than most of the rest of the repo (likely added since I standardized them all a while ago, but didn't add a check for at the time to enforce going forward), to simplify the checks.
Fixesflutter/flutter#77114
