add UI testing section to contributor documentation (#10364)

* add UI testing section to contributor documentation

* add dependency install section

* use full link path

docs/source/developer/contributing.rst

@@ -525,6 +525,25 @@ the two reports:
      `Learn
      more <https://developers.google.com/web/tools/lighthouse/audits/dom-size>`__.
 
+Visual Regression and UI Tests
+------------------------------
+
+As part of JupyterLab CI workflows, UI tests are run with visual regression checks. `Galata <https://github.com/jupyterlab/galata>`__ is used for UI testing. Galata provides a high level API to control and inspect JupyterLab UI programmatically, testing tools and CLI to manage tests and other tasks.
+
+UI tests are run for each commit into JupyterLab project in PRs or direct commits. Code changes can sometimes cause UI tests to fail for various reasons. After each test run, Galata generates a user friendly test result report which can be used to inspect failing UI tests. Result report shows the failure reason, call-stack up to the failure and detailed information on visual regression issues. For visual regression errors, reference image and test capture image, along with diff image generated during comparison are provided in the report. You can use these information to debug failing tests. Galata test report can be downloaded from GitHub Actions page for a UI test run. Test artifact is named ``ui-test-output`` and once you extract it, you can access the report by opening ``test/report/index.html`` in a browser window.
+
+Main reasons for UI test failures are:
+
+1. **A visual regression caused by code changes**:
+
+   Sometimes unintentional UI changes are introduced by modifications to project source code. Goal of visual regression testing is to detect this kind of UI changes. If your PR / commit is causing visual regression, then debug and fix the regression caused. You can locally run and debug the UI tests to fix the visual regression. Follow the instructions in steps 5-7 of ``Adding a new UI test suite guide`` in `UI Testing documentation <https://github.com/jupyterlab/jupyterlab/blob/master/ui-tests/README.md#adding-a-new-ui-test-suite>`__ to locally debug and fix UI tests. Once you have a fix, you can push the change to your GitHub branch and test with GitHub actions.
+
+2. **An intended update to user interface**:
+
+   If your code change is introducing an update to UI which causes existing UI Tests to fail, then you will need to update reference image(s) for the failing tests. In order to do that, simply go to GitHub Actions page for the failed test and download test artifacts. It will contain test captures in directory ``test/screenshots``. You can copy the capture for the failed test and paste into reference screenshots directory in JupyterLab source code, replacing the failing test's reference capture. Reference captures are located in ``ui-tests/reference-output/screenshots`` in JupyterLab source code.
+
+For more information on UI Testing, please read the `UI Testing developer documentation <.https://github.com/jupyterlab/jupyterlab/blob/master/ui-tests/README.md>`__ and `Galata documentation <https://github.com/jupyterlab/galata/blob/main/README.md>`__.
+
 Contributing to the debugger front-end
 --------------------------------------
 

ui-tests/README.md

@@ -18,6 +18,9 @@ import { galata, describe, test } from '@jupyterlab/galata';
 ```bash
 # inside ui-tests directory:
 
+# install dependencies
+jlpm install --frozen-lockfile
+
 # launch JupyterLab
 #   create empty working directory
 mkdir jlab_root