doc: Enriched UX testing documentation, #3573 (#3583)

docs/docs/technical-reference/functional-tests.md

@@ -6,53 +6,22 @@ sidebar_label: Functional tests
 
 import useBaseUrl from '@docusaurus/useBaseUrl';
 
-You will need:
+## Introduction
 
--   [Node.js 10 or 12 and above](https://nodejs.org)
+OpenRefine interface is tested with the [Cypress framework](https://www.cypress.io/).  
--   [Yarn or NPM](https://yarnpkg.com/)
+With Cypress, tests are performing assertions using a real browser, the same way a real user would use the software.
--   A Unix/Linux shell environment or the Windows command line
 
-## Installation
+Cypress tests can be ran
 
-To install Cypress and dependencies, run :
+-   using the Cypress test runner (development mode)
+-   using a command line (CI/CD mode)
 
-```
+If you are writing tests, the cypress test runner is good enough, and the command-line is mainly used by the CI/CD platform (Github actions)
-cd ./main/tests/cypress
-yarn install
-```
-
-Cypress tests can be started in two modes:
-
-
-### Development / Debugging mode
-
-Dev mode assumes that OpenRefine is up and running on the local machine, the tests themselves do not launch OpenRefine, nor restarts it.
-
-Run :
-
-```shell
-yarn --cwd ./main/tests/cypress run cypress open
-```
-
-It will open the Cypress test runner, where you can choose, replay, visualize tests.
-This is the recommended way to run tests when adding or fixing tests.  
-The runners assumes
-
-### Command-line mode
-
-Command line mode will starts OpenRefine with a temporary folder for data
-
-```shell
-./refine ui_test chrome
-```
-
-It will run all tests in the command-line, without windows, displaying results in the standard output
-This is the way to run tests in CI/CD
 
 ## Cypress brief overview
 
 Cypress operates insides a browser, it's internally using NodeJS.
-That's a key difference with tools such as selenium.  
+That's a key difference with tools such as selenium.
 
 **From the Cypress documentation:**
 
@@ -67,12 +36,132 @@ The general workflow of a Cypress test is to
 -   Trigger user actions
 -   Assert that the DOM contains expected texts and elements using selectors
 
-## Browsers
+## Getting started
+
+If that's the first time you use Cypress, it is recommended for you to get familiar with the tool.
+
+-   [Cypress overview](https://docs.cypress.io/guides/overview/why-cypress.html)
+-   [Cypress examples of tests and syntax](https://example.cypress.io/)
+
+### 1. Install Cypress
+
+You will need:
+
+-   [Node.js 10 or 12 and above](https://nodejs.org)
+-   [Yarn or NPM](https://yarnpkg.com/)
+-   A Unix/Linux shell environment or the Windows command line
+
+To install Cypress and dependencies, run :
+
+```
+cd ./main/tests/cypress
+yarn install
+```
+
+### 2. Start the test runner
+
+The test runner assumes that OpenRefine is up and running on the local machine, the tests themselves do not launch OpenRefine, nor restarts it.
+
+Start OpenRefine with
+
+```shell
+./refine
+```
+
+Then start Cypress
+
+```shell
+yarn --cwd ./main/tests/cypress run cypress open
+```
+
+### 3. Run the existing tests
+
+Once the test runner is up, you can choose to run one or several tests by selecting them from the interface.  
+Click on one of them and the test will start.
+
+### 4. Add your first test
+
+-   Add a `test.spec.js` into the cypress/integration folder.
+-   The test is instantly available in the list
+-   Click on the test
+-   Start to add some code
+
+## Tests technical documentation
+
+### A typical test
+
+A typical OpenRefine test starts with the following code
+
+```javascript
+it('Ensure cells are blanked down', function () {
+    cy.loadAndVisitProject('food.mini')
+    cy.get('.viewpanel-sorting a').contains('Sort').click()
+    cy.get('.viewpanel').should('to.contain', 'Something')
+})
+```
+
+The first noticeable thing about a test is the description (`Ensure cells are blanked down`), which describes what the test is doing.  
+Lines usually starts with `cy.something...`, which is the main way to interact with the Cypress framework.
+
+A few examples:
+
+-   `cy.get('a.my-class')` will retrieve the `<a class="my-class" />` element
+-   `cy.click()` will click on the element
+-   eventually, `cy.should()` will perform an assertion, for example that the element contains an expected text with `cy.should('to.contains', 'my text')`
+
+On top of that, OpenRefine contributors have added some functions for common OpenRefine interactions.
+For example
+
+-   `cy.loadAndVisitProject` will create a fresh project in OpenRefine
+-   `cy.assertCellEquals` will ensure that a cell contains a given value
+
+See below on the dedicated section 'Testing utilities'
+
+### Testing guidelines
+
+-   `cy.wait` should be used in the last resort scenario. It's considered a bad practice, though sometimes there is no other choice
+-   Tests should remain isolated from each other. It's best to try one feature at the time
+-   A test should always start with a fresh project
+-   The name of the files should mirror the OpenRefine UI organization
+
+### Testing utilities
+
+OpenRefine contributors have added some utility methods on the top of the Cypress framework.
+Those methods perform some common actions or assertions on OpenRefine, to avoid code duplication.
+
+Utilities can be found in `cypress/support/commands.js`.
+
+The most important utility method is `loadAndVisitProject`.  
+This method will create a fresh OpenRefine project based on a dataset given as a parameter.  
+The fixture parameter can be
+
+-   An arbitrary array, the first row is for the column names, other rows are for the values  
+    Use an arbitrary array **only** if the test requires some specific grid values  
+    **Example:**
+
+    ```javascript
+    const fixture = [
+        ['Column A', 'Column B', 'Column C'],
+        ['0A', '0B', '0C'],
+        ['1A', '1B', '1C'],
+        ['2A', '2B', '2C'],
+    ]
+    cy.loadAndVisitProject(fixture)
+    ```
+
+-   A referenced dataset: `food.small` or `food.mini`  
+    Most of the time, tests does not require any specific grid values  
+    Use food.mini as much as possible, it loads 2 rows a very few columns in the grid  
+    Use food.small if the test requires a few hundreds of rows in the grid
+
+    Those datasets live in `cypress/fixtures`
+
+### Browsers
 
 In terms of browsers, Cypress is using what is installed on your operating system.
 See the [Cypress documentation](https://docs.cypress.io/guides/guides/launching-browsers.html#Browsers) for a list of supported browsers
 
-## Folder organization
+### Folder organization
 
 Tests are located in main/tests/cypress.
 The test should not use any file outside the cypress folder.
@@ -83,7 +172,7 @@ The test should not use any file outside the cypress folder.
 -   `/screenshots` and `/videos` contains the recording of the tests, Git ignored
 -   `/support` is a custom library of assertion and common user actions, to avoid code duplication in the tests themselves
 
-## Configuration
+### Configuration
 
 Cypress execution can be configured with environment variables, they can be declared at the OS level, or when running the test
 
@@ -93,11 +182,11 @@ Available variables are
 
 Cypress contains and [exaustive documentation](https://docs.cypress.io/guides/guides/environment-variables.html#Setting) about configuration, but here are two simple ways to configure the execution of the tests:
 
-### Overriding with a cypress.env.json file
+#### Overriding with a cypress.env.json file
 
 This file is ignored by Git, and you can use it to configure Cypress locally
 
-### Command-line
+#### Command-line
 
 You can pass variables at the command-line level
 
@@ -105,6 +194,17 @@ You can pass variables at the command-line level
 yarn --cwd ./main/tests/cypress run cypress open --env OPENREFINE_URL="http://localhost:1234"
 ```
 
+## CI/CD
+
+In CI/CD, tests are runned headless, with the following command-line
+
+```shell
+./refine ui_test chrome
+```
+
+Results are displayed in the standard output
+
 ## Resources
 
 [Cypress command line options](https://docs.cypress.io/guides/guides/command-line.html#Installation)
+[Lots of good Cypress examples](https://example.cypress.io/)

main/tests/cypress/cypress/support/commands.js

@@ -35,6 +35,9 @@ Cypress.Commands.add('editCell', (rowIndex, columnName, value) => {
   cy.get('.menu-container button[bind="okButton"]').click();
 });
 
+/**
+ * Ensure a textarea have a value that id equal to the JSON given as parameter
+ */
 Cypress.Commands.add('assertTextareaHaveJsonValue', (selector, json) => {
   cy.get(selector).then((el) => {
     // expected json needs to be parsed / restringified, to avoid inconsitencies about spaces and tabs
@@ -42,6 +45,10 @@ Cypress.Commands.add('assertTextareaHaveJsonValue', (selector, json) => {
     cy.expect(JSON.stringify(present)).to.equal(JSON.stringify(json));
   });
 });
+
+/**
+ * Open OpenRefine
+ */
 Cypress.Commands.add('visitOpenRefine', (options) => {
   cy.visit(Cypress.env('OPENREFINE_URL'), options);
 });
@@ -78,6 +85,9 @@ Cypress.Commands.add('doCreateProjectThroughUserInterface', () => {
   });
 });
 
+/**
+ * Cast a whole column to the given type, using Edit Cell / Common transform / To {type}
+ */
 Cypress.Commands.add('castColumnTo', (selector, target) => {
   cy.get(
     '.data-table th:contains("' + selector + '") .column-header-menu'
@@ -90,6 +100,9 @@ Cypress.Commands.add('castColumnTo', (selector, target) => {
   cy.get('body > .menu-container').eq(2).contains(targetAction).click();
 });
 
+/**
+ * Return the td element for a given row index and column name
+ */
 Cypress.Commands.add('getCell', (rowIndex, columnName) => {
   const cssRowIndex = rowIndex + 1;
   // first get the header, to know the cell index
@@ -102,6 +115,9 @@ Cypress.Commands.add('getCell', (rowIndex, columnName) => {
   });
 });
 
+/**
+ * Make an assertion about the content of a cell, for a given row index and column name
+ */
 Cypress.Commands.add('assertCellEquals', (rowIndex, columnName, value) => {
   const cssRowIndex = rowIndex + 1;
   // first get the header, to know the cell index
@@ -121,25 +137,40 @@ Cypress.Commands.add('assertCellEquals', (rowIndex, columnName, value) => {
   });
 });
 
+/**
+ * Navigate to one of the entries of the main left menu of OpenRefine (Create Project, Open Project, Import Project, Language Settings)
+ */
 Cypress.Commands.add('navigateTo', (target) => {
   cy.get('#action-area-tabs li').contains(target).click();
 });
 
+/**
+ * Wait for OpenRefine to finish an Ajax load
+ */
 Cypress.Commands.add('waitForOrOperation', () => {
   cy.get('body[ajax_in_progress="true"]');
   cy.get('body[ajax_in_progress="false"]');
 });
 
+/**
+ * Delete a column from the grid
+ */
 Cypress.Commands.add('deleteColumn', (columnName) => {
   cy.get('.data-table th[title="' + columnName + '"]').should('exist');
   cy.columnActionClick(columnName, ['Edit column', 'Remove this column']);
   cy.get('.data-table th[title="' + columnName + '"]').should('not.exist');
 });
 
+/**
+ * Wait until a dialog panel appear
+ */
 Cypress.Commands.add('waitForDialogPanel', () => {
   cy.get('body > .dialog-container > .dialog-frame').should('be.visible');
 });
 
+/**
+ * Click on the OK button of a dialog panel
+ */
 Cypress.Commands.add('confirmDialogPanel', () => {
   cy.get(
     'body > .dialog-container > .dialog-frame .dialog-footer button[bind="okButton"]'
@@ -147,6 +178,9 @@ Cypress.Commands.add('confirmDialogPanel', () => {
   cy.get('body > .dialog-container > .dialog-frame').should('not.exist');
 });
 
+/**
+ * Will click on a menu entry for a given column name
+ */
 Cypress.Commands.add('columnActionClick', (columnName, actions) => {
   cy.get(
     '.data-table th:contains("' + columnName + '") .column-header-menu'
@@ -158,11 +192,20 @@ Cypress.Commands.add('columnActionClick', (columnName, actions) => {
   cy.get('body[ajax_in_progress="false"]');
 });
 
+/**
+ * Go to a project, given it's id
+ */
 Cypress.Commands.add('visitProject', (projectId) => {
   cy.visit(Cypress.env('OPENREFINE_URL') + '/project?project=' + projectId);
   cy.get('#project-title').should('exist');
 });
 
+/**
+ * Load a new project in OpenRefine, and open the project
+ * The fixture can be
+ *   * an arbitrary array that will be loaded in the grid. The first row is for the columns names
+ *   * a file referenced in fixtures.js (food.mini | food.small)
+ */
 Cypress.Commands.add(
   'loadAndVisitProject',
   (fixture, projectName = Date.now()) => {