This fixes main branch part of https://github.com/flutter/flutter/issues/176433
The work left is adjusting cocoon to also run this check in release branch
The high level view of this pr.
1. Move ci_config.yaml and pending changelog parsing into thir own files and the parsed results are exposed as a getter method in RepositoryPackage so that it can be shared between branch_for_release, repo_package_info_check, and version_check
2. The format error of ci_config.yaml and changelog files are reported in repo_package_info_check. The other 2 assume they are correct formatted and throw directly if they are not.
3. update the version_check to be able to handle batch release package
## Pre-Review Checklist
**Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.
[^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
as title. This is required due to our policy
context: https://github.com/flutter/flutter/issues/179033
## Pre-Review Checklist
**Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.
[^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
Makes the analysis options changes described in
https://github.com/flutter/flutter/issues/178827:
- Adding
[omit_obvious_local_variable_types](https://dart.dev/tools/linter-rules/omit_obvious_local_variable_types)
- Adding
[specify_nonobvious_local_variable_types](https://dart.dev/tools/linter-rules/specify_nonobvious_local_variable_types)
- Adding
[specify_nonobvious_property_types](https://dart.dev/tools/linter-rules/specify_nonobvious_property_types)
- Adding
[type_annotate_public_apis](https://dart.dev/tools/linter-rules/type_annotate_public_apis)
- Removing
[always_specify_types](https://dart.dev/tools/linter-rules/always_specify_types)
After those changes, makes the following repo-wide changes:
- `dart fix --apply` in all packages and in script/tool/
- `dart format` in all packages and in script/tool/
- `update-excerpts` repo tooling command to update excerpts based on the
changes to their sources
Also updates the min Flutter/Dart SDK version to 3.35/3.9 for the
following packages, to avoid `analyze` failures in the `N-2` legacy
analysis run due to what appears to be a 3.9 change in what the Dart
analyzer continues to be an obvious local type in loop iterations:
- go_router
- google_fonts
- google_identity_services_web
- google_maps_flutter_web
- local_auth_platform_interface
- metrics_center
- multicast_dns
- pigeon
- rfw
- shared_preferences
- two_dimensional_scrollables
- vector_graphics_compiler
- mustache_template
- path_parsing
Because this is causing a significant amount of format churn already, I
took this opportunity to update the repository tooling to a min Dart SDK
of 3.8 (the N-2 stable version, so the earliest version we need the
tooling to support) to pick up the new format style, so the amount of
automated formatter change is higher in script/tool/ than in the
packages.
This does contain two manual changes (other than the repo tooling min
version):
-
d700b45c7d
changes `dynamic` to `Object?` in a few places where `dynamic` caused
analyzer warnings under the new rule set.
- Updates the repo tooling to ignore `.dart_tool/` when looking for
unexpected local `analysis_options.yaml` files, to fix issues running
the repo tool's `analyze` command locally based on recent changes in
`dart` behavior.
This does not include any CHANGELOG or version updates; even though we
normally version any changes to production code, mass automated changes
like this aren't worth the churn of releasing. This includes changes to
lib/example/main.dart and to README.md excerpts; while the style changes
will be user-visible on pub.dev, it's fine for those changes to wait for
the next release of each package.
Part of https://github.com/flutter/flutter/issues/178827
Changes the default branch used to find merge bases from `FETCH_HEAD` to `main`. We were arranging for it to be `main` in CI, and locally it was an unpredictable value, so just hard-coding to `main` is likely to give better behavior locally with no change to CI behavior.
Fixes https://github.com/flutter/flutter/issues/176738
Standardizes dependencies in build.gradle files on using parentheses rather than space. Also standardizes these lines on `"` rather than `'` while touching them, since that also moves closer to Kotlin syntax.
This doesn't enforce this syntax via repo tool checks, since it's not critical (it just reduces future churn when switching to Kotlin), and eventually the switch to Kotlin will provide the enforcement.
It does fix a latent issue where test-only dependency changes that used the () syntax wouldn't have been recognized by the tool as test-only changes for changelog/version check purposes.
Part of https://github.com/flutter/flutter/issues/176065
## Pre-Review Checklist
[^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
Per guidance from legal on current preferred practice, remove "All rights reserved" from all LICENSE files and header blocks, and update the repo tool check accordingly.
Adds initial file-based filtering. This does not attempt to be comprehensive, just to get some low-hanging fruit, and to create a blueprint for anyone to follow in the future when adding more filtering. I expect that once this is in place, what will happen is that as we notice cases where PRs are hitting slow or flaky tests that they clearly don't need to, we can incrementally improve the filtering on demand.
Fixes https://github.com/flutter/flutter/issues/136394
Consolidates the code to find all changed file paths into the `PackageLoopingCommand` class that is the base of almost all of the repo tooling commands. This in a preparatory PR for a future change to allow each command to define a list of files or file patterns that definitively *don't* affect that test, so that CI can be smarter about what tests to run (e.g., not running expensive integration tests for README changes).
A side effect of this change is that tests of almost all commands now need a mock `GitDir` instance. This would add a lot of copy/pasted boilerplate to the test setup, and there is already too much of that, so instead this refactors common test setup:
- Creating a memory file system
- Populating it with a packages directory
- Creating a RecordingProcessRunner to mock out process calls
- Creating a mock GitDir that forwards to a RecordingProcessRunner
into a helper method (using records and destructuring to easily return multiple values). While some tests don't need all of these steps, those that don't can easily ignore parts of it, and it will make it much easier to update tests in the future if they need them, and it makes the setup much more consistent which makes it easier to reason about test setup in general.
Prep for https://github.com/flutter/flutter/issues/136394
Multiple commands in the Flutter plugin tool have arguments that accepts a list of strings or a list of YAML files that contain a list of strings. This introduces a `getYamlListArg` helper so that this logic doesn't need to be duplicated multiple times.
In a subsequent pull requests, this will be used to add a list of packages that are allowed to have Xcode warnings. This will be used by the google_sign_in_ios package which will have warnings when using SwiftPM.
Part of: https://github.com/flutter/flutter/issues/146904
Fixes `--current-package` so that when run on a package in third_party/packages/ in works as expected, rather than failing with an error message saying that it must be run from inside a package.
macOS 14 added new requirements that un-codesigned sandbox apps must be granted access when changed. Waiting for this UI caused macOS tests to fail on macOS 14. Additionally, adding codesigning is not sufficient, since it must still be approved before codesigning is enough to pass the check. As a workaround, this PR disables sandboxing for macOS tests in CI.
https://developer.apple.com/documentation/updates/security#June-2023)
> App Sandbox now associates your macOS app with its sandbox container using its code signature. The operating system asks the person using your app to grant permission if it tries to access a sandbox container associated with a different app. For more information, see [Accessing files from the macOS App Sandbox](https://developer.apple.com/documentation/security/app_sandbox/accessing_files_from_the_macos_app_sandbox).
And that link explains why this is happening on a macOS 14 update:
> In macOS 14 and later, the operating system uses your appâs code signature to associate it with its sandbox container. If your app tries to access the sandbox container owned by another app, the system asks the person using your app whether to grant access. If the person denies access and your app is already running, then it canât read or write the files in the other appâs sandbox container. If the person denies access while your app is launching and trying to enter the other appâs sandbox container, your app fails to launch.
>
> The operating system also tracks the association between an appâs code signing identity and its sandbox container for helper tools, including launch agents. If a person denies permission for a launch agent to enter its sandbox container and the app fails to start, launchd starts the launch agent again and the operating system re-requests access.
Fixes packages part of https://github.com/flutter/flutter/issues/149264.
Verified tests pass:
https://ci.chromium.org/ui/p/flutter/builders/staging.shadow/Mac_arm64%20macos_platform_tests%20master%20-%20packages/6/overview
Adds the flowing to the tool:
- A new `--exact-match-only` flag to be used with `--packages` to prevent group matching (i.e., a selection like `--packages=path_provider --exact-match-only` would only run on `packages/path_provider/path_provider`, not `packages/path_provider/*`).
- Two new `publish` command flags:
- `--tag-for-auto-publish`, to do all the steps that `publish` currently does except for the real `pub publish`, so it would dry-run the publish and then create and push the tag if successful.
- `--already-tagged`, to skip the step of adding and pushing a tag, and replace it with a check that `HEAD` already has the expected tag.
This set of additions supports a workflow where the current `release` step is changed to use `--tag-for-auto-publish`, and then the separate auto-publish system would publish each package with `... publish --already-tagged --packages=<some package> --exact-match-only`.
See https://github.com/flutter/packages/pull/5005#discussion_r1344542422 for previous discussion/context.
Part of https://github.com/flutter/flutter/issues/126827
Updates the repo tooling to Dart 3, now that the N-2 version has Dart 3, which allows us to use Dart 3 features (e.g., records) going forward.
To allow the update:
- Removes `break` commands from `switch`es (all done automatically with `dart fix --apply`)
- Replaces mocking of `ProcessResult` with just creating an actual `ProcessResult` since it's a `final` data class and thus can't (but also doesn't need to be) mocked.
`script/tool_runner.sh` has only been intended for CI usage for quite a while, but it still lives in a prominent non-CI location which has been a source of confusion for developers (e.g., people looking at how CI is running things, and reasonably assuming that since the script isn't in a CI-specific location, it's for general use). While running it for simple cases works, it conflicts with any package selection, which is common thing to want to control when running locally.
This makes the following changes:
- Move the script into the .ci directory
- Adds explicit comments that it's not designed for local use, with a pointer to the corresponding local command
- Removes a stale reference to external-to-the-repo use of tool_runner.sh, as that usage no longer exists
- Also consolidates and updates the links to the external scripts
- Updates a couple of CONTRIBUTING.md comments that still said to run tool_runner.sh
- Opportunistically cleans up some tooling references to .cirrus.yml, found while cleaning up comments in tool_runner.sh and checking for other references.
The example app of a federated example is effectively test-only code, so treat changes to it (other than the one published file) as dev-only for the purposes of deciding what to flag for CHANGELOGs.
A non-Flutter package can have Flutter-based tests (e.g., cupertino_icons), in which case we need to use `flutter test` rather than `dart test` just like we would for a package with a non-dev Flutter dependency. This updates the `requiresFlutter` check to include dev dependencies as well as normal dependencies.
`filter-packages-to` didn't correctly the handle the case where the set of target packages is empty, meaning that all packages should be tested. This broke it for cases such as changing a CI configuration file, making the filter not take effect.
Creates a package filter flag that is an opt-in mirror to the `exclude` flag, for use with the automatic package selection options used in CI. Like `exclude`, it allows for YAML files as input.
This allows for creating split test runs in CI (e.g., during incremental migrations), where the same file is an inclusion filter for one run and an exclusion filter for the other, guaranteeing that tests are in one or the other without the possibility of some tests falling through the cracks.
Refactors colorization to a centralized utility file, and makes it all conditional on `stdout` having ANSI escape support. This makes the output more readable on LUCI, which unlike the Cirrus log display doesn't handle ANSI.
Fixes https://github.com/flutter/flutter/issues/89392
Adds new legacy analysis targets to LUCI in bringup mode.
Also updates the repo tooling to use a Flutter->Dart version mapping instead of having to separately maintain a Dart version, now that we already have such a map in the tool anyway.
Part of https://github.com/flutter/flutter/issues/114373
Since the tool only needs to support back to Flutter 3.3 (the oldest version we still run CI with), this updates the tool to the corresponding minimum Dart version.
This allows the use of `super` parameters, so `dart fix --apply` was run to convert them all (and remove all the includes that were only needed for setting default values).
Also opportunistically cleans up a bunch of unnecessary, very old `dart:async` includes. (Other than those removals, the changes here are all `dart fix`-generated.)
Extracts common logic for running `pub get`, and switches commands to use it. The common logic always uses `flutter pub get` for Flutter packages, rather than `dart pub get`, since the latter will fail if someone has a non-Flutter `dart` in their path before `flutter` (e.g., Dart team members contributing PRs).
The new `--current-package` flag was returning `foo` when run in the app-facing package of a federated plugin called `foo`, but `foo` as a package argument is treated as being the entire group, so it was running all for all of the packages in the plugin. This fixes it to return `foo/foo` in that case, which is how the tool targets app-facing packages specifically.
Adds new LUCI targets in bringup mode to run all possible Dart unit tests in Chrome.
This is a new test (see linked issue), not a port of a Cirrus test, so it involves changes to tooling and packages:
- The tooling now accepts an explicit platform. The default behavior if none is provided is the previous behavior of running in VM for everything but web plugin implementations (since that's convenient locally).
- The tooling now has a basic understanding of `dart_test.yaml` `test_on` directives, to know when to skip.
- Packages that don't support web have opt-out files.
- Packages that do support web but have a few tests that fail on web have those tests opted out.
- Packages that do support web but have a lot of failures on web have temporary opt-out files with TODOs + issue links.
Most of https://github.com/flutter/flutter/issues/128979
Updates the state checker to inspect changed Dart files to see if the only changes are implementation (not documentation) comment lines. In particular, this will fix the problem of CI flagging changes that do nothing but add `// ignore:` comments (for federated package changes involving deprecation, framework changes that require temporary ignores in packages to support stable, etc.) as needing version and changelog changes
Now that the repo tooling is always run from source, not via `pub global`, we no longer need to infer the repo location from the current directory. Instead, hard-code knowledge of where the repository root is. This makes it much easier to run the tooling, since it's common to be in a package directory rather than the repo root.
To make it even easier to run from within a package, this also adds a `--current-package` as an alternative to `--packages`. This makes it possible to, e.g., write local wrapper scripts that run a specific set of commands on whatever the current package happens to be (such as a generic version of the script discussed in https://github.com/flutter/packages/pull/4129).
As related cleanup, makes the tool non-publishable (we haven't been publishing it since the repo merge, but I never made it unpublishable; this is important now that it would not work if published) and remove the LICENSE and CHANGELOG since it's no longer a stand-alone package.
Fixes https://github.com/flutter/flutter/issues/128231
Fixes https://github.com/flutter/flutter/issues/128232
Adds the ability to replace portions of the `flutter create`d app with saved copies, and adds a second build-all phase for Android that uses a Flutter 2.0.6-created `android/` directory (AGP 4.1, Gradle 6.7) to catch issues like https://github.com/flutter/flutter/issues/125621 and https://github.com/flutter/flutter/issues/125482 prior to release.
Includes some incidental cleanup:
- Extracts a helper method for adjusting files, so that this doesn't add even more copies of basically identical code.
- (This was motivated by an earlier version of the PR that added modifications to several more files, which I ended up undoing, but the cleanup seemed worth keeping.)
- Adds missing unit test coverage.
- Reworks the unit tests to use a mock process manager and dummy files, instead of each test actually calling `flutter create`, which made each new test add several seconds to the unit test suite.
- While this reduces the integration-style coverage, in practice the integration tests of the repo tooling is the CI itself, so the unit tests should be true unit tests.
- Changes the non-legacy test to Kotlin; we were still testing with Java even though Kotlin has been the default for quite a while, so we weren't testing what most new users would actually be running. Since we now have a legacy test, I used Java there to cover both.
- Removes some dead code for modifying the AndroidManifest.xml; when trying to set up unit tests for it I discovered that it no longer matches anything in an actual project. It dates back to the original command, and seems to have been a camera-related hack, which we clearly no longer need since it wasn't working and camera still works in build-all.
This is captured somewhat in the README in the legacy project directory, but to document the approach here: originally I was going to add flags to change individual items (AGP version, Gradle version), but quickly ran into the fact that selective downgrading is extremely fragile. E.g.,:
- The Kotlin version set in current projects doesn't work when downgrading AGP and Gradle.
- The app template can unconditionally use anything (e.g., `namespace`) that the AGP version it uses supports, so arbitrary future breakage is possible.
It's also less useful as a real test of what a plugin client's project likely looks like. Starting with a complete platform directory, and doing whatever the minimal changes are to keep it working, will likely reflect a common real-world scenario. On the flip side, the reason this doesn't use a complete 2.0.6 project, but instead is based on specific platform directories, is that we don't want to waste time manually maintaining, e.g., old Dart code that is irrelevant to the goal of the test. For now this only uses Android because that's where we've seen problems in practice, but we can alway add other legacy platform tests later if we find a need.
Fixes https://github.com/flutter/flutter/issues/125689
- Updates all macOS plugins to 10.14 as a minimum version, matching Flutter `stable`.
- Checks in the results of building all macOS examples with stable to bring them up to current with auto-migrations (and in some cases, with having ever been built, in the case of some examples with plugins where the CocoaPods changes were never committed).
All Runner.xcodeproj and Runner.xcworkspace changes are 100% tool-generated.
Also updates the repo tooling to know about a few more build-only example files that should not be flagged as potentially interesting for changelogs.
Follow-up to https://github.com/flutter/plugins/pull/7038 to try to make
it work on LUCI. Looking at the checkout steps of a LUCI run, it looks
like we do a full `fetch` of `origin`, but likely only have a `master`
branch locally. Rather than rely on a specific local branch name
existing, this allows for checking `origin` (and just in case, since
it's another common remote name, `upstream`).
Hopefully fixes https://github.com/flutter/flutter/issues/119330
* Standardize on .g.dart
* Remove unused exclusion patterns
* Mark pigeons/ as dev-only in the tooling
* Version bumps
* Add missed files
* More new import fixes
* [tool] Improve main-branch detection
Currently main-branch detection for `--packages-for-branch` looks at
branch names, but this no longer works on LUCI which now seems to be
checking out specific hashes rather than branches. This updates the
behavior so that it will treat any hash that is an ancestor of `main` as
being part of `main`, which should allow post-submit detection to work
under LUCI.
Fixes https://github.com/flutter/flutter/issues/119330
* Fix throw
* Fix typos
* Update comment
Pigeon has an usual test structure since it generates test code to run
in a dummy plugin; add that structure to the list of recognized tests so
that changes to its platform tests won't be flagged by `version-check`.
* Add more options that are in flutter/packages
* Fix unnecessary awaits
* More option alignment
* Add and locally supress avoid_implementing_value_types
* Fix release-info for test-only changes
* Fix update-release-info handling of 'minimal'
* Update release metadata
Improves the logic used to determine whether to require a version and/or CHANGELOG change:
- Removes the requirement that dev-only (e.g., test) changes update the CHANGELOG, since in practice we were essentially always overriding in that case.
- Adds file-level analysis of `build.gradle` files to determine whether they are only changing test dependencies.
- Improves the "is this a published example file" logic to better match pub.dev's logic, to fix some false positives and false negatives (e.g., `rfw`'s `example/<foo>/lib/main.dart` being considered published).
Removes the no-longer-necessary special-case handling of some Dependabot PRs, as well as the PR-description-based system it was built on (and that turned out not to be very useful due to the way `CIRRUS_CHANGE_MESSAGE` actually worked). `build.gradle` analysis should not cover all such cases, and without the need to hard-code them by package name.
