javascript/ionic-framework
1
0
Fork 0
mirror of https://github.com/ionic-team/ionic-framework.git synced 2026-03-13 10:22:08 +08:00
Code Issues Packages Projects Releases Wiki Activity

chore(docs): consolidate the developer resource files into a docs/ directory (#29266)

Browse Source 
Start your review here 👉
[docs/README.md](https://github.com/ionic-team/ionic-framework/blob/FW-6107/docs/README.md)

## What is the current behavior?

Documentation files with information on how to contribute, component
implementations, testing, etc. are scattered throughout various folders
in this repository.

## What is the new behavior?

Consolidates the documentation files into a root `docs/` directory for
easier discovery and organization.

`/docs` tree:

```
├── _config.yml
├── component-guide.md
├── CONTRIBUTING.md
├── README.md
├── sass-guidelines.md
├── angular
│   ├── README.md
│   └── testing.md
├── core
│   ├── README.md
│   └── testing
│       ├── README.md
│       ├── api.md
│       ├── best-practices.md
│       ├── preview-changes.md
│       └── usage-instructions.md
├── react
│   ├── README.md
│   └── testing.md
├── react-router
│   ├── README.md
│   └── testing.md
├── vue
│   ├── README.md
│   └── testing.md
└── vue-router
    ├── README.md
    └── testing.md
```

**Migrates the following:**

| Previous Location | New Location |
| ----------------------------------------------------------- |
----------------------------------------- |
| `.github/COMPONENT-GUIDE.md` | `docs/component-guide.md` |
| `.github/CONTRIBUTING.md` | `docs/CONTRIBUTING.md` |
| `core/scripts/README.md` | `docs/core/testing/preview-changes.md` |
| `core/src/utils/test/playwright/docs/api.md` |
`docs/core/testing/api.md` |
| `core/src/utils/test/playwright/docs/best-practices.md` |
`docs/core/testing/best-practices.md` |
| `core/src/utils/test/playwright/docs/README.md` |
`docs/core/testing/README.md` |
| `core/src/utils/test/playwright/docs/usage-instructions.md` |
`docs/core/testing/usage-instructions.md` |
| `packages/angular/test/README.md` | `docs/angular/testing.md` |
| `packages/react-router/test/README.md` |
`docs/react-router/testing.md` |
| `packages/react/test/README.md` | `docs/react/testing.md` |
| `packages/react/test/base/README.md` | `docs/react/testing.md` |
| `packages/vue/test/README.md` | `docs/vue/testing.md` |

**Adds the following:**

| File | Description |
| ----------------------------- |
-----------------------------------------------------------------------
|
| `docs/sass-guidelines.md` | Sass Variable guidelines taken from
`ionic-framework-design-documents` |
| `docs/README.md` | Entry file that should link to all other files |
| `docs/_config.yml` | Config file for use with GitHub pages |
| `docs/core/README.md` | Description of core, links to contributing and
testing |
| `docs/angular/README.md` | Description of angular, links to
contributing and testing |
| `docs/react/README.md` | Description of react, links to contributing
and testing |
| `docs/react-router/README.md` | Description of react-router, links to
contributing and testing |
| `docs/vue/README.md` | Description of vue, links to contributing and
testing |
| `docs/vue-router/README.md` | Description of vue-router, links to
contributing and testing |
| `docs/vue-router/testing.md` | Testing file for vue-router, populated
from vue-router's main README |

**Does not** add any files for `angular-server`. This is because the
README is essentially empty and there is no testing in that directory. I
can add blank files if we want to have something to add to later.

**Does not** migrate the content of the packages' root `README.md`
files. These files are used for their npm package descriptions so we
should not edit them.

## Hosting Documentation

We can (and should) host these files using GitHub Pages. I have
duplicated them in a personal repository to see how this would look:
[docs-consolidation](https://brandyscarney.github.io/docs-consolidation/).

Doing so will require some formatting fixes (see [Sass
Guidelines](https://brandyscarney.github.io/docs-consolidation/sass-guidelines.html#-reusable-values))
so I did not publish them now but we can easily enable GitHub pages by
toggling a setting in this repository.

## Other information

- Verify that no documentation files were missed in the migration
- You can use these commands to search for `*.md` files in a directory:
    - `find core/src -type f -name "*.md" -print`
- `find packages/angular -type f -name "*.md" -not -path
"**/node_modules/*" -print`
- I did add some redirect links in some of the existing markdown files
so they might still exist for that reason
- We should probably break up the contributing + component guide
documentation into smaller files, such as including best practices, but
I wanted to get everything in the same place first
- The contributing has sections on each of the packages that we could
move to that package's docs folder:
https://github.com/ionic-team/ionic-framework/blob/main/.github/CONTRIBUTING.md#core

---------

Co-authored-by: Maria Hutt <thetaPC@users.noreply.github.com>
This commit is contained in:
Brandy Carney
2024-04-08 15:06:26 -04:00
committed by GitHub
parent 7b6c330f17
commit b315b0cb29
32 changed files with 1140 additions and 382 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
docs/core/README.md Normal file
View File

@@ -0,0 +1,11 @@
# Ionic Core
The [@ionic/core](https://www.npmjs.com/package/@ionic/core) package contains the Web Components that make up the reusable UI building blocks of Ionic Framework. These components are designed to be used in traditional frontend view libraries/frameworks (such as React, Angular, or Vue), or on their own through traditional JavaScript in the browser.
## Contributing
See our [Contributing Guide](/docs/CONTRIBUTING.md).
## Testing
Refer to the [Core Testing documentation](./testing/README.md) for testing the Core package.

13
docs/core/testing/README.md Normal file
View File

@@ -0,0 +1,13 @@
# Core Testing
## Directory
| Directory | Description |
| --------------------------------------------- | --------------------------------------------------------------------------------------------- |
| [Preview Changes](./preview-changes.md) | Steps on building core and previewing changes |
| [Usage Instructions](./usage-instructions.md) | How to run tests and update screenshots |
| [Best Practices](./best-practices.md) | Contains information on conventions to follow as well as pitfalls to avoid when writing tests |
| [API](./api.md) | Documents the custom functionality that has been built on top of Playwright |

458
docs/core/testing/api.md Normal file
View File

@@ -0,0 +1,458 @@
# Playwright Test Utils
The testing directory within Ionic's codebase contains utilities that can be used to more easily test Stencil projects with Playwright.
## Table of Contents
- [`test` function](#test-function)
- [`page` fixture](#page-fixture)
- [Generators](#generators)
- [Matchers](#matchers)
## `test` Function
The default [`test` function](https://playwright.dev/docs/api/class-test) has been extended to provide two custom options.
| Fixture | Type | Description |
| ------- | ---- | ----------- |
| page | [E2EPage](https://github.com/ionic-team/ionic-framework/blob/main/core/src/utils/test/playwright/playwright-declarations.ts) | An extension of the base `page` test fixture within Playwright |
| skip | [E2ESkip](https://github.com/ionic-team/ionic-framework/blob/main/core/src/utils/test/playwright/playwright-declarations.ts) | Used to skip tests based on text direction, mode, or browser |
<details>
<summary>Usage</summary>
**`page`**
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.goto('path/to/file', config);
});
});
});
```
**`skip.mode` (DEPRECATED)**
Deprecated: Use a [generator](#generators) instead.
```typescript
import { test } from '@utils/test/playwright';
test('my custom test', ({ page, skip }) => {
skip.mode('md', 'This test is iOS-specific.');
await page.goto('path/to/file');
});
```
**`skip.rtl` (DEPRECATED)**
Deprecated: Use a [generator](#generators) instead.
```typescript
import { test } from '@utils/test/playwright';
test('my custom test', ({ page, skip }) => {
skip.rtl('This test does not have RTL-specific behaviors.');
await page.goto('path/to/file');
});
```
**`skip.browser`**
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page, skip }) => {
skip.browser('webkit', 'This test does not work in WebKit yet.');
await page.goto('path/to/file', config);
});
});
});
```
**`skip.browser` with callback**
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page, skip }) => {
skip.browser((browserName: string) => browserName !== 'webkit', 'This tests a WebKit-specific behavior.');
await page.goto('path/to/file', config);
});
});
});
```
</details>
## `page` fixture
The [page fixture](https://playwright.dev/docs/test-fixtures) has been extended to provide additional methods:
| Method | Description |
| - | - |
| `goto` | The [page.goto](https://playwright.dev/docs/api/class-page#page-goto) method extended to support a config from a [generator](#generators) and to automatically wait for Stencil components to initialize. |
| `setContent` | The [page.setContent](https://playwright.dev/docs/api/class-page#page-set-content) method extended to support a config from a [generator](#generators) and to automatically wait for Stencil components to initialize. |
| `locator` | The [page.locator](https://playwright.dev/docs/api/class-page#page-locator) method extended to support `spyOnEvent`. |
| `setIonViewport` | Resizes the browser window to fit the entire height of `ion-content` on screen. Only needed when taking fullsize screenshots with `ion-content`. |
| `waitForChanges` | Waits for Stencil to re-render before proceeeding. This is typically only needed when you update a property on a component. |
| `spyOnEvent` | Creates an event spy that can be used to wait for a [CustomEvent](https://developer.mozilla.org/en-US/docs/Web/API/CustomEvent) to be emitted. |
<details>
<summary>Usage</summary>
### Using `goto`
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.goto('src/components/test/alert/test/basic', config);
});
});
});
```
### Using `setContent`
`setContent` should be used when you only need to render a small amount of markup.
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.setContent(`
<ion-button>My Button</ion-button>
<style>
ion-button {
--background: green;
}
</style>
`, config);
});
});
});
```
### Using `locator`
Locators can be used even if the target element is not in the DOM yet.
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.goto('src/components/test/alert/test/basic', config);
// Alert is not in the DOM yet
const alert = page.locator('ion-alert');
await page.click('#open-alert');
// Alert is in the DOM
await expect(alert).toBeVisible();
});
});
});
```
### Using `setIonViewport`
`setIonViewport` is only needed when a) you are using `ion-content` and b) you need to take a screenshot of the full page (including content that may overflow offscreen).
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, screenshot, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.goto('src/components/test/alert/test/basic', config);
await page.setIonViewport();
await expect(page).toHaveScreenshot(screenshot('alert'));
});
});
});
```
### Using `waitForChanges`
`waitForChanges` is only needed when you must wait for Stencil to re-render before proceeding. This is commonly used when manually updating properties on Stencil components.
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.goto('src/components/test/modal/test/basic', config);
const modal = page.locator('ion-modal');
await modal.evaluate((el: HTMLIonModalElement) => el.canDismiss = false);
// Wait for Stencil to re-render with the canDismiss changes
await page.waitForChanges();
});
});
});
```
### Using `spyOnEvent`
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, screenshot, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.goto('src/components/test/modal/test/basic', config);
// Create spy to listen for event
const ionModalDidPresent = await page.spyOnEvent('ionModalDidPresent');
await page.click('#present-modal');
// Wait for the next emission of `ionModalDidPresent`
await ionModalDidPresent.next();
});
});
});
```
</details>
## Generators
Ionic generates tests to test different modes (iOS or MD), layouts (LTR or RTL), and themes (default or dark).
### Customizing the test configs
The `configs` function accepts an object containing all the configurations you want to test. It then returns an array of each individual configuration combination. This result is iterated over and one or more tests are generated in each iteration.
<details>
<summary>Usage</summary>
**Example 1: Default config**
```typescript
import { configs, test } from '@utils/test/playwright';
/**
* This will generate the following test configs
* iOS, LTR
* iOS, RTL
* Material Design, LTR
* Material Design, RTL
*/
configs().forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
...
});
});
});
```
**Example 2: Configuring the mode**
```typescript
import { configs, test } from '@utils/test/playwright';
/**
* This will generate the following test configs
* iOS, LTR
* iOS, RTL
*/
configs({ mode: ['ios'] }).forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
...
});
});
});
```
**Example 3: Configuring the direction**
```typescript
import { configs, test } from '@utils/test/playwright';
/**
* This will generate the following test configs
* Material Design, RTL
* iOS, RTL
*/
configs({ directions: ['rtl'] }).forEach(({ config, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
...
});
});
});
```
</details>
### Using the return value from each configuration
Each value in the array returns by `configs` contains the following information:
| Name | Description |
| - | - |
| `config` | An object containing a single test configuration. This gets passed to `page.goto` or `page.setContent`. |
| `screenshot` | A helper function that generates a unique screenshot name based on the test configuration. |
| `title` | A helper function that generates a unique test title based on the test configuration. Playwright requires that each test has a unique title since it uses that to generate a test ID. |
<details>
<summary>Usage</summary>
**Example**
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
/**
* Use the "title" function to generate
* a "my test block" title with the test
* config appended to make it unique.
* Example: my test block ios/ltr
* Using "title" on the describe block
* avoids the need to use "title" on each
* inner test block.
*/
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
/**
* Pass a single config object to
* load the page with the correct mode,
* text direction, and theme.
*/
await page.goto('/src/components/alert/test/basic', config);
/**
* Use the "screenshot" function to generate
* a "alert" screenshot title with the test
* config appended to make it unique. Playwright
* will also append the browser and platform.
* Example: alert-ios-ltr-chrome-linux.png
*/
await expect(page).toHaveScreenshot(screenshot('alert'));
});
});
});
```
</details>
## Matchers
Playwright comes with [a set of matchers to do test assertions](https://playwright.dev/docs/test-assertions). However, Ionic has additional custom assertions.
| Assertion | Description |
| - | - |
| `toHaveReceivedEvent` | Ensures an event has received an event at least once. |
| `toHaveReceviedEventDetail` | Ensures an event has been received with a specified [CustomEvent.detail](https://developer.mozilla.org/en-US/docs/Web/API/CustomEvent/detail) payload. |
| `toHaveReceivedEventTimes` | Ensures an event has been received a certain number of times. |
<details>
<summary>Usage</summary>
### Using `toHaveReceivedEvent`
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, screenshot, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.setContent(`
<ion-input label="Email"></ion-input>
`, config);
const ionChange = await page.spyOnEvent('ionChange');
const input = page.locator('ion-input');
await input.type('hi@ionic.io');
// In this case you can also use await ionChange.next();
await expect(ionChange).toHaveReceivedEvent();
});
});
});
```
### Using `toHaveReceivedEventDetail`
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, screenshot, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.setContent(`
<ion-input label="Email"></ion-input>
`, config);
const ionChange = await page.spyOnEvent('ionChange');
const input = page.locator('ion-input');
await input.type('hi@ionic.io');
await ionChange.next();
await expect(ionChange).toHaveReceivedEventDetail({ value: 'hi@ionic.io' });
});
});
});
```
### Using `toHaveReceivedEventTimes`
```typescript
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ config, screenshot, title }) => {
test.describe(title('my test block'), () => {
test('my custom test', ({ page }) => {
await page.setContent(`
<ion-input label="Email"></ion-input>
`, config);
const ionChange = await page.spyOnEvent('ionChange');
const input = page.locator('ion-input');
await input.type('hi@ionic.io');
await ionChange.next();
await input.type('goodbye@ionic.io');
await ionChange.next();
await expect(ionChange).toHaveReceivedEventTimes(2);
});
});
});
```
</details>

266
docs/core/testing/best-practices.md Normal file
View File

@@ -0,0 +1,266 @@
# Best Practices
This guide details best practices that should be followed when writing E2E tests.
## Table of Contents
- [Use the customized `test` function](#practice-test)
- [Break up test directories per-feature](#practice-per-feature)
- [Follow the file format](#practice-file-format)
- [Ensure each component has a `basic` test directory with an `index.html` file](#practice-basic)
- [Use `test.describe` blocks to describe groups of tests](#practice-test-describe)
- [Place `configs` generator outside the `test.describe` block](#practice-config-describe)
- [Test rendering and functionality in separate `test.describe` blocks](#practice-describe-type)
- [Break up large or slow-running tests across multiple files](#practice-slow-tests)
- [Use standard viewport sizes](#practice-viewport)
- [Avoid using screenshots as a way of verifying functionality](#practice-screenshot-functionality)
- [Avoid tests that compare computed values](#practice-test-computed)
- [Test for positive and negative cases](#practice-positive-negative)
- [Start your test with the configuration or layout in place if possible](#practice-test-config)
- [Place your test closest to the fix or feature](#practice-test-close)
- [Account for different locales when writing tests](#practice-locales)
<h2 id="practice-test">Use the customized `test` function</h2>
Do not import the `test` fixture from `@playwright/test`. Instead, use the `test` fixture defined in `@utils/test/playwright`. This is a custom Playwright fixture that extends the built-in `test` fixture and has logic to wait for the Stencil app to load before proceeding with the test. If you do not use this test fixture, your screenshots will likely be blank.
Since this fixture extends the built in `test` fixture, all of the normal methods found in the Playwright documentation still apply.
This is the only custom fixture you need. All of the other fixtures such as `expect` can be imported from `@playwright/test`.
**Note**: `@utils` is an alias defined in `tsconfig.json` that points to `/src/utils`. This lets us avoid doing `../../../../` if we are several folders deep when importing.
<h2 id="practice-per-feature">Break up test directories per-feature</h2>
Tests should be broken up per-feature. This makes it easy for team members to quickly find tests for a particular feature. Additionally, the names of test directories should use kebab-case.
```diff
basic/component.e2e.ts
feature-a/component.e2e.ts
feature-b/component.e2e.ts
feature-c/component.e2e.ts
```
**Example:**
[Datetime Tests](https://github.com/ionic-team/ionic-framework/tree/main/core/src/components/datetime/test)
The `first-day-of-week` directory has all the tests for the `firstDayOfWeek` property on `ion-datetime`, and the `color` directory has all the tests for the `color` property usage.
Some features can be combined together, and so it is acceptable in this instance to test features together in a single directory.
<h2 id="practice-file-format">Follow the file format</h2>
E2E test files should follow this format:
```tsx
[component name].e2e.ts
```
It is recommended to have one E2E test file per directory.
**Example:**
```tsx
/basic
button.e2e.ts
/anchor
button.e2e.ts
/form
button.e2e.ts
```
In the event you need multiple E2E files per directory, add a modifier to the file that makes it unique.
**Example:**
```tsx
/basic
modal-controller.e2e.ts // E2E tests for ion-modal via modalController
modal-inline.e2e.ts // E2E tests for ion-modal via <ion-modal>
```
<h2 id="practice-basic">Ensure each component has a `basic` test directory with an `index.html` file</h2>
At a minimum, each component with a `tests` directory must also have a `basic` directory with an `index.html` file. This is done so team members can easily paste usage examples to test out when developing or reviewing PRs. The `basic` directory may have E2E tests, but they should be limited to testing the default (or basic) behavior of a component.
<h2 id="practice-test-describe">Use `test.describe` blocks to describe groups of tests</h2>
Each E2E test file should have at least 1 `test.describe` block which defines the component and the feature you are testing.
**Example:**
```tsx
// src/components/button/test/basic/button.e2e.ts
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ title }) => {
test.describe(title('button: disabled state'), () => {
...
});
});
```
There should be one or more `test` blocks which have individual tests. The test name describes what the test should do.
**Example:**
```jsx
// src/components/button/test/basic/button.e2e.ts
import { configs, test } from '@utils/test/playwright';
configs().forEach(({ title }) => {
test.describe(title('button: disabled state'), () => {
test('should not have any visual regressions', async ({ page }) => {
...
});
});
});
```
<h2 id="practice-config-describe">Place `configs` generator outside the `test.describe` block</h2>
The `configs()` generator should be done outside of the `test.describe` block. The benefit of this is it lets you use `test.beforeEach` to run a common `page.goto` or `page.setContent` while passing in the correct config. It also lets you use the `title` function just on the `test.describe` block instead of each `test` inside the block.
❌ Incorrect
```typescript
import { configs test } from '@utils/test/playwright';
test.describe('button: disabled state', () => {
configs().forEach(({ config, title }) => {
/**
* This will generate a `test.beforeEach` for each test
* config, and all of the generated `test.beforeEach` blocks
* will be run on each test.
*/
test.beforeEach(async ({ page }) => {
await page.goto('/src/components/button/test', config);
});
test(title('should not have any visual regressions'), async ({ page }) => {
...
});
});
});
```
✅ Correct
```typescript
import { configs test } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('button: disabled state'), () => {
/**
* This will generate one `test.beforeEach` for
* each `test.describe` block rather than for
* each `test` inside of the `test.describe` block.
*/
test.beforeEach(async ({ page }) => {
await page.goto('/src/components/button/test', config);
});
/**
* Test title is still unique because of our use of title()
* on the test.describe block.
*/
test('should not have any visual regressions', async ({ page }) => {
...
});
});
});
```
<h2 id="practice-describe-type">Test rendering and functionality in separate `test.describe` blocks</h2>
Avoid mixing tests that take screenshots with tests that check functionality. These types of tests often have different requirements that can make a single `test.describe` block hard to understand. For example, a screenshot test might check both iOS and MD modes with LTR and RTL text directions, but a functionality test may not need that if the functionality is consistent across modes and directions.
If using multiple `test.describe` blocks creates a large test file, consider breaking up these tests across multiple files.
<h2 id="practice-slow-tests">Break up large or slow-running tests across multiple files</h2>
Tests are distributed across many test runners on continuous integration (CI) to improve performance. However, Playwright [distributes tests](https://playwright.dev/docs/test-parallel) based on test file, not individual test. This means test files that are particularly slow will negatively impact the overall CI performance.
<h2 id="practice-viewport">Use standard viewport sizes</h2>
By default, we run tests on mobile viewports only (think iPhone sized viewports). However, there are some components that have different layouts on tablet viewports. Two examples are `ion-split-pane` and the card variant of `ion-modal`.
For this scenario, developers must write tests that target the tablet viewport. This can be done by using [page.setViewportSize](https://playwright.dev/docs/api/class-page#page-set-viewport-size). The Playwright test utils directory also contains a `Viewports` constant which contains some common viewport presets. Developers should feel free to add new viewports to this as is applicable.
**Example:**
```javascript
import { configs, test, Viewports } from '@utils/test/playwright';
configs().forEach(({ config, title }) => {
test.describe(title('thing: rendering'), () => {
test('it should do a thing on tablet viewports', async ({ page }) => {
await page.setViewportSize(Viewports.tablet.portrait);
...
// test logic goes here
});
});
});
````
<h2 id="practice-screenshot-functionality">Avoid using screenshots as a way of verifying functionality</h2>
Screenshots are great for verifying visual differences, but using it to verify functionality can lead to issues. Instead, try testing for the existence of elements in the DOM, events emitted, etc.
❌ Incorrect
This test ensures that the `isOpen` property opens a modal when `true`. It does not verify that the inner contents of a modal are rendered as expected as that is outside the scope of this test. As a result, using a screenshot would not be appropriate here.
```typescript
configs().forEach(({ config, title }) => {
test.describe(title('modal: rendering') => {
test('it should open a modal', async ({ page }) => {
await page.setContent('<ion-modal is-open="true">...</ion-modal>', config);
await expect(page).toHaveScreenshot(screenshot('modal-open'));
});
});
});
````
✅ Correct
Instead, we can use a `toBeVisible` assertion to verify that `isOpen` does present a modal when `true`.
```typescript
configs().forEach(({ config, title }) => {
test.describe(title('modal: rendering') => {
test('it should open a modal', async ({ page }) => {
await page.setContent('<ion-modal is-open="true">...</ion-modal>', config);
const modal = page.locator('ion-modal');
await expect(modal).toBeVisible();
});
});
});
````
<h2 id="practice-test-computed">Avoid tests that compare computed values</h2>
All browsers render web content in slightly different manners. Instead of testing computed values such as exact pixel values, screenshots are a great way to ensure that elements are being rendered in a consistent manner across browsers.
<h2 id="practice-positive-negative">Test for positive and negative cases</h2>
It’s important to test that your code works when the API is used as intended, but what happens if someone makes a mistake? While some errors are fine, incorrect usage should not cause catastrophic failures such as data loss. While TypeScript helps catch incorrect usages, not everyone uses TypeScript in Ionic apps. Additionally, developers will sometimes typecast as `any`.
<h2 id="practice-test-config">Start your test with the configuration or layout in place if possible</h2>
This allows tests to remain fast on CI as we can focus on the test itself instead of navigating to the state where the test begins. For example, a simple [scrollIntoViewIfNeeded in Playwright](https://playwright.dev/docs/api/class-locator#locator-scroll-into-view-if-needed) can take around 300ms on CI. Since we run a single test for multiple configurations, that 300ms can add up quickly. Consider setting up your test in a way that the element you want to test is already in view when the test starts.
<h2 id="practice-test-close">Place your test closest to the fix or feature</h2>
Tests should be placed closest to where the fix or feature was implemented. This means that if a fix was written for `ion-button`, then the test should be placed in `src/components/button/tests`.
<h2 id="practice-locales">Account for different locales when writing tests</h2>
Tests ran on CI may not run on the same locale as your local machine. It's always a good idea to apply locale considerations to components that support it, when writing tests (i.e. `ion-datetime` should specify `locale="en-US"`).

29
docs/core/testing/preview-changes.md Normal file
View File

@@ -0,0 +1,29 @@
# Preview Changes
## Build
### 1. Clone ionic
git clone https://github.com/ionic-team/ionic-framework.git
cd ionic-framework
### 2. Run `npm install`
cd core
npm install
Notice that `@ionic/core` lives in `core`.
### 3. Run `npm start`
Make sure you are inside the `core` directory.
npm start
With the `start` command, Ionic components will be built with [Stencil](https://stenciljs.com/), changes to source files are watched, a local http server will startup, and [http://localhost:3333/](http://localhost:3333/) will open in a browser.
### 4. Preview
Navigate to [http://localhost:3333/src/components/](http://localhost:3333/src/components/). Each component has small e2e apps found in the `test` directory, for example: [http://localhost:3333/src/components/button/test/basic](http://localhost:3333/src/components/button/test/basic)
As changes are made in an editor to source files, the e2e app will live-reload.

214
docs/core/testing/usage-instructions.md Normal file
View File

@@ -0,0 +1,214 @@
# Usage Instructions
E2E tests verify Ionic components in a real browser. This is useful for testing user interaction and catching visual regressions. We use Playwright as it allows us to test in multiple browsers. Tests can be written and run using Playwright's public API.
## Table of Contents
- [Installing Dependencies](#installing-dependencies)
- [Configuring Docker](#configuring-docker)
- [Running Tests](#running-tests)
- [Managing Screenshots](#managing-screenshots)
- [Further Reading](#further-reading)
## Installing Dependencies
Follow these steps to install Playwright dependencies. These steps must also be run whenever the installed version of Playwright changes to ensure that you are testing with the correct browser binaries.
1. Install the Playwright dependency in the `core` directory: `npm ci`
2. Download the correct browsers: `npx playwright install`
## Configuring Docker
Ionic uses [Docker](https://www.docker.com) to provide a way to run tests locally in the same environment that is used on CI. Using Docker is **optional** as all tests can be run locally on your host machine, but there are a few reasons why you might want to use Docker to run tests locally:
1. You want to run screenshot tests against the same ground truths used on CI to test your work. Screenshots must be tested in a consistent environment otherwise there will be screenshot mismatches. Without Docker, you would first need to generate ground truths in your local environment.
2. You want to update ground truths locally. Ground truths can be updated using a GitHub Action on CI, but this can take ~15 minutes.
3. You are a community contributor and you do not have access to the GitHub Action to update ground truths.
4. You want to debug an issue that only happen on CI. While this is rare, there are sometimes Linux-specific issues that pop up during development.
The [Running Tests](#running-tests) and [Managing Screenshots](#managing-screenshots) sections show how to perform various tasks in Playwright with Docker. The section below shows how to configure your environment to get set up with Docker.
### Installing Docker
Docker can be installed by [following the steps on the Docker website](https://docs.docker.com/get-docker/).
### Configuring Docker for Headed Tests (Optional)
Additional software is needed to run headed tests inside of Docker. The Docker-specific test commands such as `npm run test.e2e.docker` are configured to use this additional software, but it is up to the developer to ensure that the software is installed and running.
Playwright relies on [XServer](https://www.x.org/wiki/XServer/), a windowing system used to draw and move windows on a display, in order to run tests in headed mode. Follow the steps below to install XServer on your computer.
> [!NOTE]
> The following instructions are based off https://www.oddbird.net/2022/11/30/headed-playwright-in-docker/
#### macOS
macOS uses [XQuartz](https://www.xquartz.org) to use XServer on macOS.
1. Install [Homebrew](https://brew.sh) if not already installed. You can run `brew --version` to check if Homebrew is installed.
2. Install XQuartz: `brew install --cask xquartz`
3. Open XQuartz, go to `Preferences > Security`, and check “Allow connections from network clients”.
4. Restart your computer.
5. Start XQuartz from the command line: `xhost +localhost`
6. Open Docker Desktop and edit settings to give access to `/tmp/.X11-unix` in `Preferences > Resources > File sharing`.
7. In the `core` directory run `echo host.docker.internal:0 > docker-display.txt`. This information is used to set the `DISPLAY` environment variable which tells Playwright how to render a headed UI from the Docker container.
8. In the `core` directory run `echo /tmp/.X11-unix:/tmp/.X11-unix > docker-display-volume.txt`. This information is used to make XServer available inside of the Docker container.
#### Windows
Windows has a native XServer called [WSLg](https://github.com/microsoft/wslg#readme) that is included as part of the [Windows Subsystem for Linux (WSL)](https://apps.microsoft.com/store/detail/9P9TQF7MRM4R?hl=en-us&gl=US). If you are running Docker Desktop on Windows 10 or 11 you likely already have both WSL and WSLg installed. The following steps show how to verify the WSL and WSLg are installed as well as how to configure your environment for headed tests in Docker.
If either of the below verification checks fail, then developers should [download the latest version of WSL](https://apps.microsoft.com/store/detail/9P9TQF7MRM4R?hl=en-us&gl=US).
1. To verify WSL is installed, launch "WSL" from the start menu. If "WSL" does not show up in the start menu then you do not have WSL installed.
2. With WSL open, verify that WSLg is installed: `ls -a -w 1 /mnt/wslg`. If the command fails with `No such file or directory` then your system is either missing WSL or running an old version.
3. In the `core` directory run `echo :0 > docker-display.txt`. This information is used to set the `DISPLAY` environment variable which tells Playwright how to render a headed UI from the Docker container.
4. In the `core` directory run `echo /tmp/.X11-unix:/tmp/.X11-unix > docker-display-volume.txt`. This information is used to make XServer available inside of the Docker container.
## Running Tests
### Running All Test Files
All E2E tests can be run using the following command:
```shell
npm run test.e2e
```
> [!NOTE]
> This command is a wrapper for `npx playwright test`. All data passed to `npm run test.e2e` can also be passed to `npx playwright test`.
### Running Specific Test Files
Specific test files can be run by passing the file paths or a directory that contains multiple test files. See [Managing Screenshots](#managing-screenshots) for generating ground truths before running screenshot tests.
**Specific Test Files**
```shell
npm run test.e2e src/components/button/test/basic/button.e2e.ts src/components/button/test/a11y/button.e2e.ts
```
**Test Directory with Multiple Files**
```shell
# Will run all the test files in the `test` directory
npm run test.e2e src/components/button/test
```
### Running Tests Inside Docker
While `npm run test.e2e` can be used to run tests in the same environment that you are developing in, `npm run test.e2e.docker` can be used to run tests in a Docker environment provided by the Ionic team. This command supports all the same features as `npm run test.e2e` detailed in the previous section.
This command builds a Docker image before tests run. It will also re-build the Docker image in the event that a Playwright update was merged into the repo.
Note that the Playwright report will not automatically open in your web browser when tests are complete because the tests were run in Docker. Run `npm run test.report` outside of Docker to open the most recent test report.
> [!NOTE]
> Additional setup is needed to run Playwright tests with headed mode in Docker. See [Configuring Docker for Headed Tests](#configuring-docker-for-headed-tests-optional) for more information.
### Headed vs. Headless Tests
Playwright tests in Ionic are run in headless mode by default. This means that a visual representation of the browser does not appear on your computer while running.
No additional steps are needed in order to run the tests in headless mode:
```shell
# Will run tests in headless mode
npm run test.e2e src/components/chip
```
Playwright supports the `--headed` flag to run in headed mode which causes the visual representation of the browser to appear:
```shell
# Will run tests in headed mode
npm run test.e2e src/components/chip -- --headed
```
## Managing Screenshots
If you are running a test that takes a screenshot, you must first generate the reference screenshot from your reference branch. This is known as generating a "ground truth screenshot". All other screenshots will be compared to this ground truth.
### Generating or Updating Ground Truths With Docker (Local Development)
We recommend generating ground truths inside of [Docker](https://www.docker.com). This allows anyone contributing to Ionic Framework to create or update ground truths.
To create or update ground truths, run the following command:
```shell
npm run test.e2e.docker.update-snapshots
```
Optionally, you can pass a directory to only update the ground truths for that directory & subdirectories. This is useful when working on a specific component.
```shell
npm run test.e2e.docker.update-snapshots src/components/alert/
```
The resulting screenshots should be committed and pushed to your branch.
### Generating or Updating Ground Truths Without Docker (Local Development)
While we recommend generating ground truths inside of Docker it is possible to generate ground truths without it. Note that these generated ground truths can only be used for local testing and will not update the ground truths stored in the repo.
If the reference branch has changed since the last time you generated ground truths you may need to update your local ground truths.
For most types of work the reference branch is typically `main`. Features are merged into a different branch, so developers should use that as the reference branch. For example, if branch `foo` will be merged into `bar`, then the reference branch is `bar`.
The examples provided in the [Running Tests](#running-tests) section also apply here, allowing you to update screenshots for a specific test file.
Note that since you are generating the reference branch ground truth screenshots, you must be on the reference branch locally. Don't forget to pull the latest reference branch changes and then re-build using `npm run build`.
```shell
npm run test.e2e.update-snapshots
```
Optionally, you can pass a directory to only update the ground truths for that directory & subdirectories. This is useful when working on a specific component.
```shell
npm run test.e2e.update-snapshots src/components/alert/
```
From here, you can switch back to your branch and run the tests.
> [!NOTE]
> Locally generated ground truths should not be committed to the repo. The `.gitignore` file prevents this from accidentally happening.
### Generating or Updating Ground Truths (CI)
> [!IMPORTANT]
> Only Ionic Team members can update ground truths on the main repo. Ground truths cannot be updated on forked versions of the repo. Instead, we recommend generating ground truths in Docker.
When making an intentional visual change, you will need to update the ground truth screenshots or add new ones. It is important that the ground truth and comparison screenshots are taken in the same environment, so do not update the ground truth screenshots locally and commit them to the repo.
Instead, use the [Update Reference Screenshots GitHub Action](https://github.com/ionic-team/ionic-framework/actions/workflows/update-screenshots.yml).
1. Click the **Run workflow** dropdown.
2. Select your branch.
3. Click **Run workflow**.
This workflow will re-run the screenshot tests. Instead of failing any tests with mismatched screenshots, it will take new ground truth screenshots. These ground truth screenshots will be pushed as a single commit to your branch once the workflow is completed.
### Verifying Screenshot Differences
When any of the screenshot tests fail, it means a potential regression was caught. Developers must manually verify the difference in the Playwright test report.
If the screenshots fail on CI then developers must download the build artifact. On the **Summary** page for a particular workflow, find the **Artifacts** section. Screenshot tests are currently parallelized across several test runners, and the results from each of those runners is included in an artifact with the following naming scheme:
```
test-results-[current shard]-[total shards]
Example:
test-results-2-5 --> Test results from job runner 2 out of 5.
```
Download the appropriate artifact and unzip the file.
In the newly created directory, open the `playwright-report/index.html` in your browser. From here, you will be able to see the tests that failed as well as the expected screenshot, the actual screenshot, and the pixel differences.
> [!WARNING]
> It is recommended to verify the screenshot difference within the Playwright test report first. If you choose to try and reproduce the difference in a browser manually, make sure you are using the **exact** same browser version that Playwright is using.
## Further Reading
For more info on how to use Playwright, please see the [Playwright documentation](https://playwright.dev/docs/intro).