Commit Graph

11 Commits

Author SHA1 Message Date
Liam DeBeasi
e98620ee99 test(ci): run tests in docker container (#28893)
Issue number: Internal

---------

<!-- Please do not submit updates to dependencies unless it fixes an
issue. -->

<!-- Please try to limit your pull request to one type (bugfix, feature,
etc). Submit multiple pull requests if needed. -->

## What is the current behavior?
<!-- Please describe the current behavior that you are modifying. -->

The team currently faces two challenges:

1. Verifying visual changes locally is difficult because we cannot use
the existing ground truths as they were generated in Linux environments
and most of our team uses either macOS or Windows. While team members
can generate ground truths in the correct environment, they need to
remember to do that first before making changes.
2. Updating visual diffs is time consuming and can only be done by team
members. Our GitHub Action runs the entire test suite which can take ~10
even if only a handful of screenshots are generated. Additionally, this
job can only be run by team members meaning community contributors
cannot update/add screenshots. This limits them to non-visual tasks when
contributing. In the event that they do want to make visual changes, the
team needs to copy all their code into a branch and manually run
screenshot diffs for them.

## What is the new behavior?
<!-- Please describe the behavior or changes that are being added by
this PR. -->

- This PR introduces the ability to run all Playwright tests inside of a
Docker container using an image with Playwright dependencies. The
container will have access to the local project, so developers can make
changes and then run tests in the container after the changes are
compiled. This enables anyone to propose new screenshot changes.
However, the "update screenshot" job will still be available for folks
who do not want/are unable to use docker.
- There are some typeface differences between GH Actions and the Docker
image which is why there are a handful of screenshots that needed to be
updated.

One risk here is that the Playwright npm and Docker image versions must
be kept in sync. As a result, I also updatRenovate to allow us to auto
update the npm and Docker image versions at the same time.

## Does this introduce a breaking change?

- [ ] Yes
- [x] No

<!--
  If this introduces a breaking change:
1. Describe the impact and migration path for existing applications
below.
  3. Update the BREAKING.md file with the breaking change.
4. Add "BREAKING CHANGE: [...]" to the commit description when merging.
See
https://github.com/ionic-team/ionic-framework/blob/main/.github/CONTRIBUTING.md#footer
for more information.
-->


## Other information

<!-- Any other information that is important to this PR such as
screenshots of how the component looks before and after the change. -->


⚠️ There are still some issues I need to sort out with mounting the
local project on Windows. However, using Ubuntu with the linux subsystem
for windows can be used as a workaround. I'd like to merge this so we
can start testing it in our day-to-day workflow and ironing out any
bugs.

---------

Co-authored-by: ionitron <hi@ionicframework.com>
2024-03-18 14:08:30 +00:00
renovate[bot]
0314f1cf76 chore(deps): update download + upload artifacts to v4 (major) (#29049)
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[actions/download-artifact](https://togithub.com/actions/download-artifact)
| action | major | `v3` -> `v4` |
|
[actions/upload-artifact](https://togithub.com/actions/upload-artifact)
| action | major | `v3` -> `v4` |

---

### Release Notes

<details>
<summary>actions/download-artifact (actions/download-artifact)</summary>

###
[`v4`](https://togithub.com/actions/download-artifact/compare/v3...v4)

[Compare
Source](https://togithub.com/actions/download-artifact/compare/v3...v4)

</details>

<details>
<summary>actions/upload-artifact (actions/upload-artifact)</summary>

### [`v4`](https://togithub.com/actions/upload-artifact/compare/v3...v4)

[Compare
Source](https://togithub.com/actions/upload-artifact/compare/v3...v4)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "every weekday before 11am" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Never, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/ionic-team/ionic-framework).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xNzMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjE3My4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-03-01 15:07:17 +00:00
renovate[bot]
975bc43893 chore(deps): update actions/setup-node action to v4 (#29048)
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/setup-node](https://togithub.com/actions/setup-node) | action
| major | `v3` -> `v4` |

---

### Release Notes

<details>
<summary>actions/setup-node (actions/setup-node)</summary>

### [`v4`](https://togithub.com/actions/setup-node/compare/v3...v4)

[Compare
Source](https://togithub.com/actions/setup-node/compare/v3...v4)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "every weekday before 11am" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Never, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/ionic-team/ionic-framework).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xNzMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjIxMi4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-03-01 15:06:06 +00:00
Liam DeBeasi
fd5a1ac708 chore(ci): account for new screenshots in error logging (#28166)
Issue number: N/A

---------

<!-- Please do not submit updates to dependencies unless it fixes an
issue. -->

<!-- Please try to limit your pull request to one type (bugfix, feature,
etc). Submit multiple pull requests if needed. -->

## What is the current behavior?
<!-- Please describe the current behavior that you are modifying. -->

We checked `git diff` to see if new screenshots were generated. If no
new screenshots were generated then we print a user-friendly message
informing the dev what happened. However, `git diff` does not account
for untracked changes (i.e. new screenshots).

As a result, this prevented new screenshots from being committed.

## What is the new behavior?
<!-- Please describe the behavior or changes that are being added by
this PR. -->

- To fix this we use the `-N` flag which is short for `--intent-to-add`.
This adds an entry with no content which causes the new screenshots to
show up when running `git diff`. We later add the contents of the image.

Example test run of this working:
https://github.com/ionic-team/ionic-framework/actions/runs/6174263712/job/16759260534

## Does this introduce a breaking change?

- [ ] Yes
- [x] No

<!-- If this introduces a breaking change, please describe the impact
and migration path for existing applications below. -->


## Other information

<!-- Any other information that is important to this PR such as
screenshots of how the component looks before and after the change. -->
2023-09-13 15:57:17 +00:00
Liam DeBeasi
4b4ad75bf0 chore: add error when no new diffs generated (#28132)
Issue number: N/A

---------

<!-- Please do not submit updates to dependencies unless it fixes an
issue. -->

<!-- Please try to limit your pull request to one type (bugfix, feature,
etc). Submit multiple pull requests if needed. -->

## What is the current behavior?
<!-- Please describe the current behavior that you are modifying. -->

When the "Update Reference Screenshot" job is run but no new screenshots
are generated, the following error is logged:

```
nothing added to commit but untracked files present (use "git add" to track)
Error: Process completed with exit code 1.
```

This is happening because there are no files to commit when running `git
commit`. Brandy noted that this was confusing since it doesn't actually
tell you why there are no files to commit.

## What is the new behavior?
<!-- Please describe the behavior or changes that are being added by
this PR. -->

- If there are no diffs to commit then the update screenshot script will
log a message saying that no screenshot diffs were generated.

Example:

```
⚠️ Error: No new screenshots generated ⚠️
This means that there were zero visual diffs when running screenshot tests.
Make sure you have pushed any code changes that would result in visual diffs.
```


https://github.com/ionic-team/ionic-framework/actions/runs/6099399582/job/16552017414

## Does this introduce a breaking change?

- [ ] Yes
- [x] No

<!-- If this introduces a breaking change, please describe the impact
and migration path for existing applications below. -->


## Other information

<!-- Any other information that is important to this PR such as
screenshots of how the component looks before and after the change. -->
2023-09-06 16:38:39 +00:00
Sean Perkins
38c4da3353 chore(ci): use node v18 for github actions (#28103)
Issue number: N/A

---------

<!-- Please do not submit updates to dependencies unless it fixes an
issue. -->

<!-- Please try to limit your pull request to one type (bugfix, feature,
etc). Submit multiple pull requests if needed. -->

## What is the current behavior?
<!-- Please describe the current behavior that you are modifying. -->

`action/setup-node@v3` shipped a "breaking" change that requires node
v18. This breaks our dev build release process.


https://github.com/ionic-team/ionic-framework/actions/runs/6044495970/job/16403194005

## What is the new behavior?
<!-- Please describe the behavior or changes that are being added by
this PR. -->

- Updates the publish action to use node v18

## Does this introduce a breaking change?

- [ ] Yes
- [x] No

<!-- If this introduces a breaking change, please describe the impact
and migration path for existing applications below. -->


## Other information

<!-- Any other information that is important to this PR such as
screenshots of how the component looks before and after the change. -->

Here is a successful run with these changes:
https://github.com/ionic-team/ionic-framework/actions/runs/6044521110
2023-09-01 17:57:10 +00:00
Liam DeBeasi
5e4892676c chore(ci): update actions versions (#26100) 2022-10-13 13:04:01 -04:00
Liam DeBeasi
e5e0e24f76 fix(vue): canDismiss definition is now exposed (#25195) 2022-04-26 14:16:27 -04:00
Liam DeBeasi
ff1429ba7a chore(ci): fix update screenshot action to account for existing screenshot (#25190) 2022-04-25 18:17:48 -04:00
Liam DeBeasi
12fd19dd4d chore(playwright): screenshots are now ignored locally (#25043) 2022-04-01 12:06:40 -04:00
Liam DeBeasi
0aa6d124d6 test(e2e): add infrastructure for migration to playwright (#25033) 2022-03-31 11:23:21 -04:00