End-to-End Testing

Overview

End-to-End tests simulate real user scenarios, interacting with applications through the UI. This ensures that all application components work together as expected, increasing confidence in your software's quality and reducing the risk of bugs reaching production. Instead of focusing on isolated components, E2E tests validate the entire software flow—from the user interface down to the backend and back—ensuring all parts work together seamlessly.

Getting Started

As an End-to-End testing solution, the Storefront Application integrates Playwright, an open-source automation framework for cross-browser testing developed by Microsoft. Playwright provides a stable environment for writing E2E tests that run smoothly across all major browsers using a single API. It features reliable cross-browser support and powerful tools for handling complex scenarios like multiple tabs, file uploads, mobile emulation, and native parallel test execution.

We recommend getting familiar with Playwright features and capabilities before proceeding with this guide.

Setup & Configuration

The Playwright integration is handled as a dedicated sub-package within the Storefront Application. It is located within the playwright directory. For Playwright to function properly, we need to manually install its dependencies, as this is not done in parallel with the Storefront Application. The following command should be executed within the playwright directory:

sh

Copy

Playwright setup features can be checked and modified in playwright.config.ts file. Some of the most important are:

• Tests directory: The directory within the codebase to store test files.
• Base URL: Should be defined via an environment variable and uses localhost as a fallback.
  const BASE_URL = process.env.BASE_URL ?? 'https://localhost:3000/de/
• Parallelism: By default, all tests are executed in a parallel mode.
• Workers: Defines how many worker threads will be used during parallel test execution. By default in the CI we only use a single thread, when executed locally it will use all cores of your CPU.
• Reporters: Depending on the desired output format, the reporter can be modified within this section. By default junit, list and playwright-testrail-reporter are configured.

• Projects: A project allows you to run your test suite against multiple configurations. For example, it allows you to run your End-to-End tests against multiple browsers to ensure consistent quality for all of your users, regardless of the browser they choose. By default, the Storefront Application runs the tests against Safari, Firefox and Chrome.
  
  More information on how to configure the project to use multiple browsers or to execute the tests on different environments can be found in the Playwright documentation.

typescript

Copy

Environment Variables

As a prerequisite to successfully running End-to-End tests that require user authentication, relevant search terms for testing search features, or any data-specific tests, environment variables should be set.

The list of required environment variables is available in templates/nuxt/.env.example file. They can be set in the working .env file, as well as in the respective CI tool, e.g. as environment variables in GitLab.

Test Users

Test users should be registered in the system, and their e-mail addresses and passwords should match the values of environment variables.

Search

As a prerequisite to successfully run the Search End-to-End tests, the following environment variables should be set.

Running the Tests

Tests can be executed locally from playwright directory. By default, tests stored within playwright/tests/are executed when the command is triggered.

To aid debugging, tests can be run in headed mode. Using headed mode, Playwright will start its UI with a list of tests and a configuration section to choose in which browsers you want to run the tests. After the tests are executed, UI allows you to check the execution step by step with the visual representation of every test step.

Headed mode can be executed using the command:

bash

Copy

Another way of local test execution is to use headless mode. In that case browser will not be launched, but you will still be able to check the execution progress and test results in the terminal.

Headless mode can be executed using the command:

bash

Copy

Test Structure

Every test file should be named as *.spec.ts and stored in playwright/tests/ directory.

Each .spec.ts file can contain one or more tests. It is recommended to group feature-related tests into one file. For example, all Search related tests are stored into e2e-Search.spec.ts file.

Tests follow the Arrange-Act-Assert pattern:

• Arrange:
  • await homePage.visitPage(): Navigate to the home page.
  • await header.clickLoginHeaderButton(): Click the login button.
• Act:
  • await signinPage.fillLoginData(...): Enter valid login credentials.
  • await signinPage.clickLoginButton(): Submit the login form.
• Assert:
  • await page.waitForURL('/de'): Verify successful login by checking the URL.
  • await toastMessage.assertToastInfoIsVisible(): Assert login notification is visible
  • await signinPage.assertLoginButtonIsVisible(): Verify the user is back to the login page.