javascript/ionic-framework
1
0
Fork 0
mirror of https://github.com/ionic-team/ionic-framework.git synced 2026-03-13 10:22:08 +08:00
Code Issues Packages Projects Releases Wiki Activity

test(e2e): port e2e tests to @ionic/core (#13438)

Browse Source 
* feat(e2e-tests) simplify e2e test structure

* test(badge) add basic e2e test

* test(button) update e2e test to new structure

* test(card) add basic e2e test

* test(checkbox) add basic e2e test

* chore(e2e-test) update path to e2e module in run-e2e

* fix(button) update toolbar e2e deps

* fix(e2e-test) update path in snapshot script

* feat(e2e-test) move e2e scripts into scripts/e2e

* test(chip) add basic e2e test

* test(content) add basic e2e test

* test(datetime) add basic e2e test

* style(e2e-test) use consistent title/header in e2e test pages

* test(fab) add basic e2e test

* fix(e2e-test) don't run e2e script when required

* test(grid) add basic e2e test

* test(icon) add basic e2e test

* test(input) add basic e2e test

* style(e2e-test) use consistent e2e test header titles

* test(list) add basic e2e test

* test(menu) add basic e2e test

* test(modal) add basic e2e test

* feat(e2e-test) add navigate export to e2e module

* feat(e2e-test) add present method to Page class

* test(popover) add basic e2e test

* test(menu) add present left menu e2e test

* test(radio) add basic e2e test

* test(range) add  basic e2e test

* test(searchbar) add basic e2e test

* test(segment) add basic e2e test

* test(select) add basic e2e test

* test(modal) add shows modal e2e test

* test(slides) add basic e2e test

* test(spinner) add basic/color e2e tests

* test(tabs) add basic e2e test

* test(toast) add basic e2e test

* test(toggle) add basic e2e test

* test(toolbar) add basic e2e test

* docs(e2e-test) update e2e readme to reflect simplest test

* test(card) update basic e2e test

* chore(e2e-test) remove run-e2e script

* test(components): move remaining component tests to index files

* chore(package): add mocha to devDependencies

* test(infinite-scroll) add basic e2e test

* test(item-sliding) add basic e2e test

* test(item) add basic/buttons e2e tests

* test(nav) add basic e2e test

* test(reorder) add basic e2e test

* test(split-pane) add basic e2e test

* chore() update declarations file

* refactor(toast): reduce border-radius

* chore(components): update components.d.ts
This commit is contained in:
Cam Wiegert
2017-11-16 12:19:23 -06:00
committed by Brandy Carney
parent 90b6e01a38
commit db475cd153
90 changed files with 846 additions and 509 deletions
Download Patch File Download Diff File Expand all files Collapse all files

84
packages/core/scripts/e2e/README.md Normal file
View File

@@ -0,0 +1,84 @@
# End-to-end Testing Scripts
The end-to-end testing scripts consist of the following modules:
1. `e2e` - the test controller and test utilities
1. `run-e2e` - the script that npm uses to kick this stuff off
1. `E2ETestPage` - a base class for end-to-end tests
1. `Snapshot` - the snapshot tool, copied from the `index.js` file in the [Snapshot Repo](https://github.com/ionic-team/snapshot) (private)
## Writing End-to-end Tests
Each end-to-end test file is NodeJS ES2015 script that contains at least one `describe` and registers at least one case.
In general, writing an end-to-end tests consists of the following steps:
1. create a `e2e.js` file
1. extend the `Page` class to perform the extra actions a page needs to do (if any)
1. register each test you would like to run using the `register` method from the `e2e` module, the `register` method takes two parameters: a test description and a callback function that contains the test, the callback is passed the selenium driver that is in use for the test
The most basic end-to-end test just navigates to the page in order to verify that it draws properly. In this case, it is not necessary to extend the E2ETestPage class. The base class contains a navigate method that goes to the page and waits for it to load. The test just needs to instantiate the page with the proper URL and call the navigate. Such a test looks like this:
```ts
const { register, navigate } = require('../../../../scripts/e2e');
describe('button: basic', () => {
register('navigates', navigate('http://localhost:3333/src/components/button/test/basic'));
});
```
For more complicated tests, it may be necessary to extend the base E2ETestPage class to add perform more actions that can then be used in the tests. Such a test may look like this:
```ts
const { By, until } = require('selenium-webdriver');
const { register, Page } = require('../../../../scripts/e2e');;
class ActionSheetE2ETestPage extends Page {
constructor(driver) {
super(driver, 'http://localhost:3333/src/components/action-sheet/test/basic');
}
present(buttonId) {
this.navigate();
this.driver.findElement(By.id(buttonId)).click();
this.driver.wait(until.elementLocated(By.css('.action-sheet-container')));
return this.driver.wait(until.elementIsVisible(this.driver.findElement(By.css('.action-sheet-container'))));
}
}
describe('action-sheet: basic', () => {
register('navigates', driver => {
const page = new ActionSheetE2ETestPage(driver);
return page.navigate();
});
describe('present', () => {
register('shows basic', driver => {
const page = new ActionSheetE2ETestPage(driver);
return page.present('basic');
});
register('shows noBackdropDismiss', (driver) => {
const page = new ActionSheetE2ETestPage(driver);
return page.present('noBackdropDismiss');
});
});
});
```
Note that you generally do not have to `await` any of the async actions in your tests. Selenium has a call-chain that handles all of that so long as the final promise is waiting upon, and the `register` function takes care of that for you.
## Running the Tests
To run the tests, just use npm from the `packages/core` directory under the `ionic` project.
* `npm run e2e`
* `npm run snapshot`
## TODO Items
1. this script could probably be used for other packages and should be moved back a directory, perhaps the same for the base class as well, that needs to be figured out as we go
1. turn off animations and then adjust the wait time accordingly
1. adjustments will likely be needed when the Snapshot tool has better reporting, for example the tool will likely have `start` and `finish` methods (or some such thing)
1. cycle through the various platforms (or at least iOS and Android) like the current `ionic-angular` does (I think that is currently handled via `gulp`, needs to be looked into)
1. the current Snapshots seem to have some funky boardering issues when uploaded, may need to look into that

23
packages/core/scripts/e2e/e2e-test-page.js Normal file
View File

@@ -0,0 +1,23 @@
const webdriver = require('selenium-webdriver');
const By = webdriver.By;
const until = webdriver.until;
module.exports = class E2ETestPage {
constructor(driver, url) {
this.url = url;
this.driver = driver;
}
navigate() {
this.driver.navigate().to(this.url);
this.driver.wait(until.elementLocated(By.css('.hydrated')));
return this.driver.wait(until.elementIsVisible(this.driver.findElement(By.css('.hydrated'))));
}
present(clickTarget, options) {
this.navigate();
this.driver.findElement(By.css(clickTarget)).click();
this.driver.wait(until.elementLocated(By.css(options.waitFor)));
return this.driver.wait(until.elementIsVisible(this.driver.findElement(By.css(options.waitFor))));
}
}

139
packages/core/scripts/e2e/index.js Normal file
View File

@@ -0,0 +1,139 @@
'use strict';
const fs = require('fs');
const glob = require('glob');
const Mocha = require('mocha');
const path = require('path');
const webdriver = require('selenium-webdriver');
const Page = require('./e2e-test-page');
const Snapshot = require('./snapshot');
let driver;
let snapshot;
let specIndex = 0;
let takeScreenshots = false;
function startDevServer() {
const server = require('@stencil/dev-server/dist'); // TODO: fix after stencil-dev-server PR #16 is merged
const cmdArgs = ['--config', path.join(__dirname, '../stencil.config.js'), '--no-open'];
return server.run(cmdArgs);
}
function generateTestId() {
let chars = 'abcdefghjkmnpqrstuvwxyz';
let id = chars.charAt(Math.floor(Math.random() * chars.length));
chars += '0123456789';
while (id.length < 3) {
id += chars.charAt(Math.floor(Math.random() * chars.length));
}
return id;
}
function getTestFiles() {
return new Promise((resolve, reject) => {
const src = path.join(__dirname, '../../src/**/e2e.js');
glob(src, (err, files) => {
if (err) {
reject(err);
} else {
resolve(files);
}
});
});
}
function processCommandLine() {
process.argv.forEach(arg => {
if (arg === '--snapshot') {
takeScreenshots = true;
}
});
}
function registerE2ETest(desc, tst) {
// NOTE: Do not use an arrow function here because: https://mochajs.org/#arrow-functions
it(desc, async function() {
await tst(driver);
if (takeScreenshots) {
await snapshot.takeScreenshot(driver, {
name: this.test.fullTitle(),
specIndex: specIndex++
});
}
return Promise.resolve(true);
});
}
function getTotalTests(suite) {
let ttl = suite.tests.length;
suite.suites.forEach(s => (ttl += getTotalTests(s)));
return ttl;
}
async function run() {
const mocha = new Mocha({
timeout: 5000,
slow: 2000
});
driver = new webdriver.Builder().forBrowser('chrome').build();
processCommandLine();
const devServer = await startDevServer();
const files = await getTestFiles();
files.forEach(f => mocha.addFile(f));
mocha.loadFiles(() => {
specIndex = 0;
snapshot = new Snapshot({
groupId: 'ionic-core',
appId: 'snapshots',
testId: generateTestId(),
domain: 'ionic-snapshot-go.appspot.com',
// domain: 'localhost:8080',
sleepBetweenSpecs: 750,
totalSpecs: getTotalTests(mocha.suite),
platformDefaults: {
browser: 'chrome',
platform: 'linux',
params: {
platform_id: 'chrome_400x800',
platform_index: 0,
platform_count: 1,
width: 400,
height: 800
}
},
accessKey: process.env.IONIC_SNAPSHOT_KEY
});
mocha.run(function(failures) {
process.on('exit', function() {
process.exit(failures); // exit with non-zero status if there were failures
});
if (takeScreenshots) {
snapshot.finish();
}
devServer.close();
driver.quit();
});
});
}
const navigate = url => driver => new Page(driver, url).navigate();
// Invoke run() only if executed directly i.e. `node ./scripts/e2e`
if (require.main === module) {
run();
}
module.exports = {
Page,
navigate,
register: registerE2ETest,
run: run
};

121
packages/core/scripts/e2e/snapshot.js Normal file
View File

@@ -0,0 +1,121 @@
'use strict';
const request = require('request');
class Snapshot {
constructor(options) {
this.appId = (options && options.appId) || 'test_app';
this.domain = (options && options.domain) || 'localhost:8080';
this.groupId = (options && options.groupId) || 'test_group';
this.sleepTime = (options && options.sleepBetweenSpecs) || 500;
this.totalSpecs = options && options.totalSpecs;
this.accessKey = options && options.accessKey;
this.platformId =
options && options.platformDefaults && options.platformDefaults.params && options.platformDefaults.params.platform_id;
this.platformIndex =
options && options.platformDefaults && options.platformDefaults.params && options.platformDefaults.params.platform_index;
this.platformCount =
options && options.platformDefaults && options.platformDefaults.params && options.platformDefaults.params.platform_count;
this.width =
(options && options.platformDefaults && options.platformDefaults.params && options.platformDefaults.params.width) || -1;
this.height =
(options && options.platformDefaults && options.platformDefaults.params && options.platformDefaults.params.height) || -1;
this.start(options && options.testId);
}
async finish() {
console.log('waiting for uploads to complete');
await Promise.all(this.queue);
console.log(`done processing ${this.queue.length} screenshots`);
console.log(`${this.mismatches.length} snapshots had significant mismatches`);
console.log(`Test Id: ${this.testId}`);
}
start(testId) {
this.testId = testId;
this.queue = [];
this.highestMismatch = 0;
this.mismatches = [];
this.results = {};
}
async takeScreenshot(driver, options) {
this._resizeWindow(driver);
await this._allowForAnnimation();
const screenshot = await this._takeScreenshot(driver, options);
return this._post(screenshot);
}
_allowForAnnimation() {
return new Promise((resolve, reject) => {
setTimeout(function() {
resolve();
}, this.sleepTime);
});
}
_post(screenshot) {
const p = new Promise((resolve, reject) => {
request.post(`http://${this.domain}/screenshot`, { form: screenshot }, (error, res, body) => {
if (error) {
console.error(error);
} else if (res.statusCode > 400) {
console.log('error posting screenshot:', response.statusCode, body);
} else {
const data = JSON.parse(body);
this.highestMismatch = Math.max(this.highestMismatch, data.Mismatch);
const resultKey = (data.Mismatch * 1000 + 1000000 + '').split('.')[0] + '-' + screenshot.spec_index;
this.results[resultKey] = {
index: screenshot.spec_index,
name: screenshot.description,
mismatch: Math.round(data.Mismatch * 100) / 100,
compareUrl: data.CompareUrl,
screenshotUrl: data.ScreenshotUrl
};
if (data.IsMismatch) {
this.mismatches.push(resultKey);
}
}
resolve(res);
});
});
this.queue.push(p);
return Promise.resolve(true);
}
_resizeWindow(driver) {
return driver
.manage()
.window()
.setSize(this.width, this.height);
}
async _takeScreenshot(driver, options) {
const capabilities = await driver.getCapabilities();
const png = await driver.takeScreenshot();
const url = await driver.getCurrentUrl();
return Promise.resolve({
app_id: this.appId,
group_id: this.groupId,
description: options.name,
spec_index: options.specIndex,
total_specs: this.totalSpecs,
test_id: this.testId,
url: url,
png_base64: png,
height: this.height,
width: this.width,
platform_count: this.platformCount,
platform_id: this.platformId,
platform_index: this.platformIndex,
browser: capabilities.get('browserName'),
platform: capabilities.get('platform'),
version: capabilities.get('version'),
access_key: this.accessKey
});
}
}
module.exports = Snapshot;