Florian Giroud
GitHub
parent
859828a0f0
commit
7003dd2d2d
@ -6,53 +6,22 @@ sidebar_label: Functional tests
|
||||
|
||||
import useBaseUrl from '@docusaurus/useBaseUrl';
|
||||
|
||||
You will need:
|
||||
## Introduction
|
||||
|
||||
- [Node.js 10 or 12 and above](https://nodejs.org)
|
||||
- [Yarn or NPM](https://yarnpkg.com/)
|
||||
- A Unix/Linux shell environment or the Windows command line
|
||||
OpenRefine interface is tested with the [Cypress framework](https://www.cypress.io/).
|
||||
With Cypress, tests are performing assertions using a real browser, the same way a real user would use the software.
|
||||
|
||||
## Installation
|
||||
Cypress tests can be ran
|
||||
|
||||
To install Cypress and dependencies, run :
|
||||
- using the Cypress test runner (development mode)
|
||||
- using a command line (CI/CD mode)
|
||||
|
||||
```
|
||||
cd ./main/tests/cypress
|
||||
yarn install
|
||||
```
|
||||
|
||||
Cypress tests can be started in two modes:
|
||||
|
||||
|
||||
### Development / Debugging mode
|
||||
|
||||
Dev mode assumes that OpenRefine is up and running on the local machine, the tests themselves do not launch OpenRefine, nor restarts it.
|
||||
|
||||
Run :
|
||||
|
||||
```shell
|
||||
yarn --cwd ./main/tests/cypress run cypress open
|
||||
```
|
||||
|
||||
It will open the Cypress test runner, where you can choose, replay, visualize tests.
|
||||
This is the recommended way to run tests when adding or fixing tests.
|
||||
The runners assumes
|
||||
|
||||
### Command-line mode
|
||||
|
||||
Command line mode will starts OpenRefine with a temporary folder for data
|
||||
|
||||
```shell
|
||||
./refine ui_test chrome
|
||||
```
|
||||
|
||||
It will run all tests in the command-line, without windows, displaying results in the standard output
|
||||
This is the way to run tests in CI/CD
|
||||
If you are writing tests, the cypress test runner is good enough, and the command-line is mainly used by the CI/CD platform (Github actions)
|
||||
|
||||
## Cypress brief overview
|
||||
|
||||
Cypress operates insides a browser, it's internally using NodeJS.
|
||||
That's a key difference with tools such as selenium.
|
||||
That's a key difference with tools such as selenium.
|
||||
|
||||
**From the Cypress documentation:**
|
||||
|
||||
@ -67,12 +36,132 @@ The general workflow of a Cypress test is to
|
||||
- Trigger user actions
|
||||
- Assert that the DOM contains expected texts and elements using selectors
|
||||
|
||||
## Browsers
|
||||
## Getting started
|
||||
|
||||
If that's the first time you use Cypress, it is recommended for you to get familiar with the tool.
|
||||
|
||||
- [Cypress overview](https://docs.cypress.io/guides/overview/why-cypress.html)
|
||||
- [Cypress examples of tests and syntax](https://example.cypress.io/)
|
||||
|
||||
### 1. Install Cypress
|
||||
|
||||
You will need:
|
||||
|
||||
- [Node.js 10 or 12 and above](https://nodejs.org)
|
||||
- [Yarn or NPM](https://yarnpkg.com/)
|
||||
- A Unix/Linux shell environment or the Windows command line
|
||||
|
||||
To install Cypress and dependencies, run :
|
||||
|
||||
```
|
||||
cd ./main/tests/cypress
|
||||
yarn install
|
||||
```
|
||||
|
||||
### 2. Start the test runner
|
||||
|
||||
The test runner assumes that OpenRefine is up and running on the local machine, the tests themselves do not launch OpenRefine, nor restarts it.
|
||||
|
||||
Start OpenRefine with
|
||||
|
||||
```shell
|
||||
./refine
|
||||
```
|
||||
|
||||
Then start Cypress
|
||||
|
||||
```shell
|
||||
yarn --cwd ./main/tests/cypress run cypress open
|
||||
```
|
||||
|
||||
### 3. Run the existing tests
|
||||
|
||||
Once the test runner is up, you can choose to run one or several tests by selecting them from the interface.
|
||||
Click on one of them and the test will start.
|
||||
|
||||
### 4. Add your first test
|
||||
|
||||
- Add a `test.spec.js` into the cypress/integration folder.
|
||||
- The test is instantly available in the list
|
||||
- Click on the test
|
||||
- Start to add some code
|
||||
|
||||
## Tests technical documentation
|
||||
|
||||
### A typical test
|
||||
|
||||
A typical OpenRefine test starts with the following code
|
||||
|
||||
```javascript
|
||||
it('Ensure cells are blanked down', function () {
|
||||
cy.loadAndVisitProject('food.mini')
|
||||
cy.get('.viewpanel-sorting a').contains('Sort').click()
|
||||
cy.get('.viewpanel').should('to.contain', 'Something')
|
||||
})
|
||||
```
|
||||
|
||||
The first noticeable thing about a test is the description (`Ensure cells are blanked down`), which describes what the test is doing.
|
||||
Lines usually starts with `cy.something...`, which is the main way to interact with the Cypress framework.
|
||||
|
||||
A few examples:
|
||||
|
||||
- `cy.get('a.my-class')` will retrieve the `<a class="my-class" />` element
|
||||
- `cy.click()` will click on the element
|
||||
- eventually, `cy.should()` will perform an assertion, for example that the element contains an expected text with `cy.should('to.contains', 'my text')`
|
||||
|
||||
On top of that, OpenRefine contributors have added some functions for common OpenRefine interactions.
|
||||
For example
|
||||
|
||||
- `cy.loadAndVisitProject` will create a fresh project in OpenRefine
|
||||
- `cy.assertCellEquals` will ensure that a cell contains a given value
|
||||
|
||||
See below on the dedicated section 'Testing utilities'
|
||||
|
||||
### Testing guidelines
|
||||
|
||||
- `cy.wait` should be used in the last resort scenario. It's considered a bad practice, though sometimes there is no other choice
|
||||
- Tests should remain isolated from each other. It's best to try one feature at the time
|
||||
- A test should always start with a fresh project
|
||||
- The name of the files should mirror the OpenRefine UI organization
|
||||
|
||||
### Testing utilities
|
||||
|
||||
OpenRefine contributors have added some utility methods on the top of the Cypress framework.
|
||||
Those methods perform some common actions or assertions on OpenRefine, to avoid code duplication.
|
||||
|
||||
Utilities can be found in `cypress/support/commands.js`.
|
||||
|
||||
The most important utility method is `loadAndVisitProject`.
|
||||
This method will create a fresh OpenRefine project based on a dataset given as a parameter.
|
||||
The fixture parameter can be
|
||||
|
||||
- An arbitrary array, the first row is for the column names, other rows are for the values
|
||||
Use an arbitrary array **only** if the test requires some specific grid values
|
||||
**Example:**
|
||||
|
||||
```javascript
|
||||
const fixture = [
|
||||
['Column A', 'Column B', 'Column C'],
|
||||
['0A', '0B', '0C'],
|
||||
['1A', '1B', '1C'],
|
||||
['2A', '2B', '2C'],
|
||||
]
|
||||
cy.loadAndVisitProject(fixture)
|
||||
```
|
||||
|
||||
- A referenced dataset: `food.small` or `food.mini`
|
||||
Most of the time, tests does not require any specific grid values
|
||||
Use food.mini as much as possible, it loads 2 rows a very few columns in the grid
|
||||
Use food.small if the test requires a few hundreds of rows in the grid
|
||||
|
||||
Those datasets live in `cypress/fixtures`
|
||||
|
||||
### Browsers
|
||||
|
||||
In terms of browsers, Cypress is using what is installed on your operating system.
|
||||
See the [Cypress documentation](https://docs.cypress.io/guides/guides/launching-browsers.html#Browsers) for a list of supported browsers
|
||||
|
||||
## Folder organization
|
||||
### Folder organization
|
||||
|
||||
Tests are located in main/tests/cypress.
|
||||
The test should not use any file outside the cypress folder.
|
||||
@ -83,7 +172,7 @@ The test should not use any file outside the cypress folder.
|
||||
- `/screenshots` and `/videos` contains the recording of the tests, Git ignored
|
||||
- `/support` is a custom library of assertion and common user actions, to avoid code duplication in the tests themselves
|
||||
|
||||
## Configuration
|
||||
### Configuration
|
||||
|
||||
Cypress execution can be configured with environment variables, they can be declared at the OS level, or when running the test
|
||||
|
||||
@ -93,11 +182,11 @@ Available variables are
|
||||
|
||||
Cypress contains and [exaustive documentation](https://docs.cypress.io/guides/guides/environment-variables.html#Setting) about configuration, but here are two simple ways to configure the execution of the tests:
|
||||
|
||||
### Overriding with a cypress.env.json file
|
||||
#### Overriding with a cypress.env.json file
|
||||
|
||||
This file is ignored by Git, and you can use it to configure Cypress locally
|
||||
|
||||
### Command-line
|
||||
#### Command-line
|
||||
|
||||
You can pass variables at the command-line level
|
||||
|
||||
@ -105,6 +194,17 @@ You can pass variables at the command-line level
|
||||
yarn --cwd ./main/tests/cypress run cypress open --env OPENREFINE_URL="http://localhost:1234"
|
||||
```
|
||||
|
||||
## CI/CD
|
||||
|
||||
In CI/CD, tests are runned headless, with the following command-line
|
||||
|
||||
```shell
|
||||
./refine ui_test chrome
|
||||
```
|
||||
|
||||
Results are displayed in the standard output
|
||||
|
||||
## Resources
|
||||
|
||||
[Cypress command line options](https://docs.cypress.io/guides/guides/command-line.html#Installation)
|
||||
[Lots of good Cypress examples](https://example.cypress.io/)
|
||||
|
||||
@ -35,6 +35,9 @@ Cypress.Commands.add('editCell', (rowIndex, columnName, value) => {
|
||||
cy.get('.menu-container button[bind="okButton"]').click();
|
||||
});
|
||||
|
||||
/**
|
||||
* Ensure a textarea have a value that id equal to the JSON given as parameter
|
||||
*/
|
||||
Cypress.Commands.add('assertTextareaHaveJsonValue', (selector, json) => {
|
||||
cy.get(selector).then((el) => {
|
||||
// expected json needs to be parsed / restringified, to avoid inconsitencies about spaces and tabs
|
||||
@ -42,6 +45,10 @@ Cypress.Commands.add('assertTextareaHaveJsonValue', (selector, json) => {
|
||||
cy.expect(JSON.stringify(present)).to.equal(JSON.stringify(json));
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Open OpenRefine
|
||||
*/
|
||||
Cypress.Commands.add('visitOpenRefine', (options) => {
|
||||
cy.visit(Cypress.env('OPENREFINE_URL'), options);
|
||||
});
|
||||
@ -78,6 +85,9 @@ Cypress.Commands.add('doCreateProjectThroughUserInterface', () => {
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Cast a whole column to the given type, using Edit Cell / Common transform / To {type}
|
||||
*/
|
||||
Cypress.Commands.add('castColumnTo', (selector, target) => {
|
||||
cy.get(
|
||||
'.data-table th:contains("' + selector + '") .column-header-menu'
|
||||
@ -90,6 +100,9 @@ Cypress.Commands.add('castColumnTo', (selector, target) => {
|
||||
cy.get('body > .menu-container').eq(2).contains(targetAction).click();
|
||||
});
|
||||
|
||||
/**
|
||||
* Return the td element for a given row index and column name
|
||||
*/
|
||||
Cypress.Commands.add('getCell', (rowIndex, columnName) => {
|
||||
const cssRowIndex = rowIndex + 1;
|
||||
// first get the header, to know the cell index
|
||||
@ -102,6 +115,9 @@ Cypress.Commands.add('getCell', (rowIndex, columnName) => {
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Make an assertion about the content of a cell, for a given row index and column name
|
||||
*/
|
||||
Cypress.Commands.add('assertCellEquals', (rowIndex, columnName, value) => {
|
||||
const cssRowIndex = rowIndex + 1;
|
||||
// first get the header, to know the cell index
|
||||
@ -121,25 +137,40 @@ Cypress.Commands.add('assertCellEquals', (rowIndex, columnName, value) => {
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Navigate to one of the entries of the main left menu of OpenRefine (Create Project, Open Project, Import Project, Language Settings)
|
||||
*/
|
||||
Cypress.Commands.add('navigateTo', (target) => {
|
||||
cy.get('#action-area-tabs li').contains(target).click();
|
||||
});
|
||||
|
||||
/**
|
||||
* Wait for OpenRefine to finish an Ajax load
|
||||
*/
|
||||
Cypress.Commands.add('waitForOrOperation', () => {
|
||||
cy.get('body[ajax_in_progress="true"]');
|
||||
cy.get('body[ajax_in_progress="false"]');
|
||||
});
|
||||
|
||||
/**
|
||||
* Delete a column from the grid
|
||||
*/
|
||||
Cypress.Commands.add('deleteColumn', (columnName) => {
|
||||
cy.get('.data-table th[title="' + columnName + '"]').should('exist');
|
||||
cy.columnActionClick(columnName, ['Edit column', 'Remove this column']);
|
||||
cy.get('.data-table th[title="' + columnName + '"]').should('not.exist');
|
||||
});
|
||||
|
||||
/**
|
||||
* Wait until a dialog panel appear
|
||||
*/
|
||||
Cypress.Commands.add('waitForDialogPanel', () => {
|
||||
cy.get('body > .dialog-container > .dialog-frame').should('be.visible');
|
||||
});
|
||||
|
||||
/**
|
||||
* Click on the OK button of a dialog panel
|
||||
*/
|
||||
Cypress.Commands.add('confirmDialogPanel', () => {
|
||||
cy.get(
|
||||
'body > .dialog-container > .dialog-frame .dialog-footer button[bind="okButton"]'
|
||||
@ -147,6 +178,9 @@ Cypress.Commands.add('confirmDialogPanel', () => {
|
||||
cy.get('body > .dialog-container > .dialog-frame').should('not.exist');
|
||||
});
|
||||
|
||||
/**
|
||||
* Will click on a menu entry for a given column name
|
||||
*/
|
||||
Cypress.Commands.add('columnActionClick', (columnName, actions) => {
|
||||
cy.get(
|
||||
'.data-table th:contains("' + columnName + '") .column-header-menu'
|
||||
@ -158,11 +192,20 @@ Cypress.Commands.add('columnActionClick', (columnName, actions) => {
|
||||
cy.get('body[ajax_in_progress="false"]');
|
||||
});
|
||||
|
||||
/**
|
||||
* Go to a project, given it's id
|
||||
*/
|
||||
Cypress.Commands.add('visitProject', (projectId) => {
|
||||
cy.visit(Cypress.env('OPENREFINE_URL') + '/project?project=' + projectId);
|
||||
cy.get('#project-title').should('exist');
|
||||
});
|
||||
|
||||
/**
|
||||
* Load a new project in OpenRefine, and open the project
|
||||
* The fixture can be
|
||||
* * an arbitrary array that will be loaded in the grid. The first row is for the columns names
|
||||
* * a file referenced in fixtures.js (food.mini | food.small)
|
||||
*/
|
||||
Cypress.Commands.add(
|
||||
'loadAndVisitProject',
|
||||
(fixture, projectName = Date.now()) => {
|
||||
|
||||
Loading…
Reference in New Issue
Block a user