Issue number: N/A
---------
<!-- Please do not submit updates to dependencies unless it fixes an
issue. -->
<!-- Please try to limit your pull request to one type (bugfix, feature,
etc). Submit multiple pull requests if needed. -->
## What is the current behavior?
<!-- Please describe the current behavior that you are modifying. -->
Screenshots in a branch are currently compared against the screenshots
on `main`. While we ultimately need to compare against ground truths in
`main`, this can cause tests to seemingly fail at random. For example,
this branch was created based on
eb2772c0ce.
However,
381de0b3d3
was added to `main` after the first commit and has new screenshot diffs.
I do not have the latest ground truths for the changed screenshots in
this branch, so the screenshot tests failed in
https://github.com/ionic-team/ionic-framework/actions/runs/4995702351/jobs/8947969061
even though I did not change any component implementations.
## What is the new behavior?
<!-- Please describe the behavior or changes that are being added by
this PR. -->
- Test runs in CI now run against the ground truths available in the
branch. When a commit is added to the merge queue, it will be tested
against the latest `main` and any commits ahead of it in the queue. As a
result, if there _is_ a conflict then the test run will fail at that
point and the commit will be removed from the merge queue.
## Does this introduce a breaking change?
- [ ] Yes
- [x] No
<!-- If this introduces a breaking change, please describe the impact
and migration path for existing applications below. -->
## Other information
<!-- Any other information that is important to this PR such as
screenshots of how the component looks before and after the change. -->
Issue number: N/A
---------
<!-- Please do not submit updates to dependencies unless it fixes an
issue. -->
<!-- Please try to limit your pull request to one type (bugfix, feature,
etc). Submit multiple pull requests if needed. -->
## What is the current behavior?
<!-- Please describe the current behavior that you are modifying. -->
The generator migration is complete, so we can safely remove the legacy
test infrastructure.
## What is the new behavior?
<!-- Please describe the behavior or changes that are being added by
this PR. -->
- Removes the legacy test infrastructure
## Does this introduce a breaking change?
- [ ] Yes
- [x] No
<!-- If this introduces a breaking change, please describe the impact
and migration path for existing applications below. -->
## Other information
<!-- Any other information that is important to this PR such as
screenshots of how the component looks before and after the change. -->
Note: There's still a bit of sharding imbalance that I need to look into
(test runner 17 is a few minutes longer than all the other test
runners), but I will handle that in a separate PR.
Issue number: N/A
---------
<!-- Please refer to our contributing documentation for any questions on
submitting a pull request, or let us know here if you need any help:
https://ionicframework.com/docs/building/contributing -->
<!-- Some docs updates need to be made in the `ionic-docs` repo, in a
separate PR. See
https://github.com/ionic-team/ionic-framework/blob/main/.github/CONTRIBUTING.md#modifying-documentation
for details. -->
<!-- Please do not submit updates to dependencies unless it fixes an
issue. -->
<!-- Please try to limit your pull request to one type (bugfix, feature,
etc). Submit multiple pull requests if needed. -->
This is the first PR to introduce the infrastructure required to add
test generators to the Ionic Framework project. This PR introduces the
file name changes necessary to support two playwright configs, so I
recommend reviewing the PR by commit:
1e5012cea1
- Created a `playwright.config-legacy.ts` file and updates
`package.json`.
- Running `npm run test.e2e` will run the generator tests, and running
`npm run test.e2e.legacy` will run the legacy tests.
4fe8de7df7
- Updates the GitHub Action scripts to run both the modern and legacy
E2E tests. I added command modifiers to avoid collisions with output
directories.
e8bcfaf926
- Updates `*.e2e.ts` files to have the legacy format name:
`*.e2e-legacy.ts`. This naming scheme is required for the two Playwright
configs to pull in the correct files. When migrating tests to
generators, team members will rename the file to remove the `-legacy`
part.
5bf196c36d
(warning: lots of files!)
- Updates the `*.e2e.ts-snapshots` directories to have the legacy format
name: `*.e2e-legacy.ts-snapshots`. The screenshot directory in
Playwright is generated based on the test file name which is why we are
updating the screenshot directory. When migrating tests to generators,
team members will rename the directory to remove the `-legacy` part.
## Does this introduce a breaking change?
- [ ] Yes
- [x] No
<!-- If this introduces a breaking change, please describe the impact
and migration path for existing applications below. -->
## Other information
<!-- Any other information that is important to this PR such as
screenshots of how the component looks before and after the change. -->
<!-- Please refer to our contributing documentation for any questions on
submitting a pull request, or let us know here if you need any help:
https://ionicframework.com/docs/building/contributing -->
<!-- Some docs updates need to be made in the `ionic-docs` repo, in a
separate PR. See
https://github.com/ionic-team/ionic-framework/blob/main/.github/CONTRIBUTING.md#modifying-documentation
for details. -->
<!-- Please do not submit updates to dependencies unless it fixes an
issue. -->
<!-- Please try to limit your pull request to one type (bugfix, feature,
etc). Submit multiple pull requests if needed. -->
## What is the current behavior?
<!-- Please describe the current behavior that you are modifying. -->
<!-- Issues are required for both bug fixes and features. -->
Issue URL: Internal
## What is the new behavior?
<!-- Please describe the behavior or changes that are being added by
this PR. -->
- Adds Angular 16 test app
- Introduces version test to validate which major version of Angular is
being tested
## Does this introduce a breaking change?
- [ ] Yes
- [x] No
<!-- If this introduces a breaking change, please describe the impact
and migration path for existing applications below. -->
## Other information
<!-- Any other information that is important to this PR such as
screenshots of how the component looks before and after the change. -->
BREAKING CHANGE:
Angular v14 is now required to use `@ionic/angular` and `@ionic/angular-server`. Upgrade your project to Angular v14 by following the [Angular v14 update guide](https://update.angular.io/?l=3&v=13.0-14.0).
The dev-preview `environmentInjector` property has been removed from `ion-tabs` and `ion-router-outlet`. Standalone component routing is now available without additional custom configuration. Remove the `environmentInjector` property from your `ion-tabs` and `ion-router-outlet` components.
Screenshot matrix jobs had "fail-fast: false". This enabled each test runner to provide its `playwright-report` directory so team members can view test failures. At the time, all tests ran even if previous tests failed.
In https://github.com/ionic-team/ionic-framework/pull/26447, we updated the Playwright config to have tests fail fast within a single test runner. However, other test runners continue to run. This is not helpful in the case of failing tests because the artifacts for each test runner will not be reported until all test runners are done.
This PR changes the matrix strategy to enable fail-fast (which is the default) so in-progress/queued test runners fail if a previous test runner fails.
