15 Commits

Author SHA1 Message Date
bb5a25815c [ci] Upload screenshots, logs, and Xcode test results for drive and integration_test runs (#7430)
1. Native Xcode tests will output a helpful "xcresult" package on failure containing logs, screenshots, and screen recordings.  Zip and upload these results when tests fail.

2. Pass `flutter test --debug-logs-dir` flag to upload logs  like https://github.com/flutter/flutter/pull/142643.

3. Pass `flutter drive --screenshot` flag to upload screenshots on timeout like https://github.com/flutter/flutter/pull/96973.

Example of [failing Xcode analyzer build](https://ci.chromium.org/ui/p/flutter/builders/try/Mac_arm64%20ios_platform_tests_shard_5%20master/17374/overview) has the [zipped xcresult](https://storage.googleapis.com/flutter_logs/flutter/ff98c32e-18ca-4ad4-a910-9db1d7f7e4b0/xcode%20analyze/ff98c32e-18ca-4ad4-a910-9db1d7f7e4b0/xcodebuild-2024-10-25T09:56:46.440913.zip) attached as a log.
![Screenshot 2024-10-25 at 10 10 36 AM](https://github.com/user-attachments/assets/dd7ae9bc-6161-4381-8a4f-f10b8c981801)

The unzipped xcresult looks like this in Xcode:

![Screenshot 2024-10-25 at 10 11 55 AM](https://github.com/user-attachments/assets/d4dd8420-f272-459c-9785-ab0c03887a74)

A [failing "native test" step build](https://ci.chromium.org/ui/p/flutter/builders/try/Mac_arm64%20macos_platform_tests%20master%20-%20packages/17315/overview):

![Screenshot 2024-10-25 at 10 19 55 AM](https://github.com/user-attachments/assets/76a86a15-2150-482a-8b15-e3e7ac90485e)

Fixes https://github.com/flutter/flutter/issues/144795
2024-11-06 06:58:26 +00:00
9bcf4bfa49 [ci] Add LUCI web platform tests (#4391)
Adds web platform tests for LUCI in bringup mode.

The Cirrus version of the test starts `chromedriver` at the beginning and just leaves it running, which is fine since it's a fresh docker image each run. For LUCI we don't want that behavior though, so instead this adds tooling support for running chromedriver as part of running the integration tests, controllable with a flag. This will also be useful for running locally.

Part of https://github.com/flutter/flutter/issues/114373
2023-07-07 01:10:14 +00:00
902c337f6f [ci] Pin Chromium version for web tests (#4620)
Switches the web tests from using the version of Chrome installed by the Dockerfile, which is whatever happened to be stable when the image is generated, and thus not hermetic, to a pinned version of Chromium. This uses a slightly modified version of the script that is already used for flutter/packages.

Since Chromium doesn't support mp4 playback, this updates the `video_player` integration tests to use WebM on web instead, to avoid having all the tests fail in CI.

Part of https://github.com/flutter/flutter/issues/84712
2022-01-05 12:53:05 -08:00
74cf0287f9 [flutter_plugin_tools] Improve process mocking (#4254)
The mock process runner used in most of the tests had poor handling of
stdout/stderr:
- By default it would return the `List<int>` output from the mock
  process, which was never correct since the parent process runner
  interface always sets encodings, thus the `dynamic` should always be
  of type `String`
- The process for setting output on a `MockProcess` was awkward, since
  it was based on a `Stream<Lint<int>>`, even though in every case what
  we actually want to do is just set the full output to a string.
- A hack was at some point added (presumably due to the above issues)
  to bypass that flow at the process runner level, and instead return a
  `String` set there. That meant there were two ways of setting output
  (one of which that only worked for one of the ways of running
  processes)
  - That hack wasn't updated when the ability to return multiple mock
    processes instead of a single global mock process was added, so the
    API was even more confusing, and there was no way to set different
    output for different processes.

This changes the test APIs so that:
- `MockProcess` takes stdout and stderr as strings, and internally
  manages converting them to a `Stream<List<int>>`.
- `RecordingProcessRunner` correctly decodes and returns the output
  streams when constructing a process result.

It also removes the resultStdout and resultStderr hacks, as well as the
legacy `processToReturn` API, and converts all uses to the new
structure, which is both simpler to use, and clearly associates output
with specific processes.
2021-08-24 11:42:21 -07:00
77460f03f2 [flutter_plugin_tools] Make unit tests pass on Windows (#4149)
The purpose of this PR is to make running all unit tests on Windows pass (vs failing a large portion of the tests as currently happens). This does not mean that the commands actually work when run on Windows, or that Windows support is tested, only that it's possible to actually run the tests themselves. This is prep for actually supporting parts of the tool on Windows in future PRs.

Major changes:
- Make the tests significantly more hermetic:
  - Make almost all tools take a `Platform` constructor argument that can be used to inject a mock platform to control what OS the command acts like it is running on under test.
  - Add a path `Context` object to the base command, whose style matches the `Platform`, and use that almost everywhere instead of the top-level `path` functions.
  - In cases where Posix behavior is always required (such as parsing `git` output), explicitly use the `posix` context object for `path` functions.
- Start laying the groundwork for actual Windows support:
  - Replace all uses of `flutter` as a command with a getter that returns `flutter` or `flutter.bat` as appropriate.
  - For user messages that include relative paths, use a helper that always uses Posix-style relative paths for consistent output.

This bumps the version since quite a few changes have built up, and having a cut point before starting to make more changes to the commands to support Windows seems like a good idea.

Part of https://github.com/flutter/flutter/issues/86113
2021-07-09 19:38:13 -04:00
d2761ab1de [flutter_plugin_tool] Add more failure test coverage (#4132)
Many commands had insufficient failure testing. This adds new tests that
ensure that for every Process call, at least one test fails if a failure
from that process were ignored (with the exception of calls in the
`publish` command, which has a custom process mocking system, so was out
of scope here; it already has more coverage than most tests did though.)

For a few existing failure tests, adds output checks to ensure that they
are testing for the *right* failures.

Other changes:
- Adds convenience constructors to MockProcess for the common cases of a
  mock process that just exits with a 0 or 1 status, to reduce test
  verbosity.
- Fixes a few bugs that were found by the new tests.
- Minor test cleanup, especially cases where a mock process was being
  set up just to make all calls succeed, which is the default as of
  recent changes.
2021-07-02 10:49:04 -07:00
b98034dd76 [flutter_plugin_tools] Migrate more commands to NNBD (#4026)
Migrates:
- `all_plugins_app`
- `podspecs`
- `firebase-test-lab`

Minor functional changes to `firebase-test-lab` based on issues highlighted by the migration:
- The build ID used in the path is now a) passable, and b) given a fallback value in the path that
  isn't "null"
- Flag setup will no longer assume that `$HOME` must be set in the environment.
- Adds a --build-id flag to `firebase-test-lab` instead of hard-coding the use of  `CIRRUS_BUILD_ID`.
  The default is still `CIRRUS_BUILD_ID` so no CI changes are needed.

Part of https://github.com/flutter/flutter/issues/81912
2021-06-09 11:00:04 -07:00
181fe18e27 [flutter_plugin_tools] Migrate xctest command to NNBD (#4024) 2021-06-07 11:44:05 -07:00
842ae94fde Migrate some tool commands to NNBD (#3899)
Now that individual commands can be migrated, migrate several commands
that are trivially migratable.

Part of https://github.com/flutter/flutter/issues/81912
2021-05-17 11:25:01 -07:00
bd0081258a Begin migrating tools to NNBD (#3891)
- Updates dependencies to null-safe versions
- Migrates common.dart (which doesn't depend on anything)
- Migrates common_tests.dart and its one dependency, utils.dart
- Adds build_runner for Mockito mock generation
- Adds a new utility methods for getting arguments that handle both the casting and the removal of nullability to address a common problematic pattern while migrating code.
  - Converts all files, not just the migrated ones, to those new helpers.

Migrating common.dart and utils.dart should unblock a command-by-command migration to null safety.

Reverts the separate of podspect lints into a step that doesn't do a Flutter upgrade
(https://github.com/flutter/plugins/pull/3700) because without that step we had a
version of Dart too old to run null-safe tooling.

First step of https://github.com/flutter/flutter/issues/81912
2021-05-14 20:04:26 -04:00
3d9e523218 Switch script/tools over to the new analysis options (#3777)
Removes the legacy analysis options override and fixes all resulting issues. This is a combination of dart fix and manual changes (mostly mechanical, but some small restructuring to address warnings more cleanly, such as creating typed structs from args when they are used repeatedly to avoid repeated casting, or making things that were unnecessarily public private).

One small opportunistic extra cleanup is that the handling of null-safety prerelease versions is removed, as any new plugin would be written null-safe from the start, so we no longer need to allow those versions.

Part of flutter/flutter#76229
2021-04-06 13:04:25 -07:00
aef6008b7b Standardize copyright year (#3737)
Standardizes all first-party copyrights on a single year, as is done in flutter/flutter and flutter/engine. All code now uses 2013, which is the earliest year that was in any existing copyright notice.

The script checks now enforce the exact format of first-party licenses and copyrights.

Fixes flutter/flutter#78448
2021-03-19 13:18:00 -07:00
c956cb3270 Standardize Copyrights: Chromium->Flutter (#2996)
In all copyright messages (and in the Xcode project organization name) standardize on "The Flutter Authors", adding "The Chromium Authors" to the Flutter AUTHORS list. This reduces inconsistency in the copyright lines in this repository, moving closer to a single consistent copyright+license (as in flutter/engine and flutter/flutter)

Updates the validation script to no longer accept "The Chromium Authors" or "the Chromium project authors" in first-party code.
2021-03-18 19:05:33 -04:00
6141051fa2 Add missing licenses, and add a check (#3720)
Adds a new CI check that all code files have a copyright+license block (and that it's one we are expecting to see).

Fixes the ~350 files (!) that did not have them. This includes all of the files in the .../example/ directories, following the example of flutter/flutter. (This does mean some manual intervention will be needed when generating new example directories in the future, but it's one-time per example.)

Also standardized some variants that used different line breaks than most of the rest of the repo (likely added since I standardized them all a while ago, but didn't add a check for at the time to enforce going forward), to simplify the checks.

Fixes flutter/flutter#77114
2021-03-16 17:50:51 -04:00
e1b3b296f2 Move plugin tool tests over (#3606) 2021-03-01 18:15:51 -08:00