🔗 End-to-End Testing

Overview

End-to-End (E2E) tests simulate real user scenarios, interacting with applications through the UI. This ensures that all application components work together as expected, increasing confidence in your software's quality and reducing the risk of bugs reaching production. Instead of focusing on isolated components, E2E tests validate the entire software flow—from the user interface down to the backend and back—ensuring all parts work together seamlessly.

Getting Started

As an E2E testing solution, the Storefront Application integrates Playwright, an open-source automation framework for cross-browser testing developed by Microsoft. Playwright provides a stable environment for writing E2E tests that run smoothly across all major browsers using a single API. It features reliable cross-browser support and powerful tools for handling complex scenarios like multiple tabs, file uploads, mobile emulation, and native parallel test execution.

We recommend getting familiar with Playwright features and capabilities before proceeding with this guide.

Setup & Configuration

The Playwright integration is handled as a dedicated sub-package within the Storefront Application. It is located in the templates/nuxt/testing/playwright/ directory. To ensure it functions properly, you must manually install its dependencies, as this isn't done automatically with the main application install. The following command should be executed within the templates/nuxt/testing/playwright/ directory:

sh

Copy

The playwright.config.ts file is the central place to configure how your tests are run. You can check and modify the following features:

• Tests Directory: This defines the directory where all your test files are stored within the codebase.
• Base URL: The base URL for your application should be defined via an environment variable. A fallback is provided, which defaults to localhost if no environment variable is set:
  const BASE_URL = process.env.BASE_URL ?? 'https://localhost:3000/de/
• Parallelism: By default, all tests are executed in a parallel mode to speed up test runs.
• Workers: This setting defines how many worker threads are used during a parallel test execution. By default, the CI uses a single thread, while local runs use all available CPU cores.
• Reporters: Playwright allows you to modify the output format of your test results. By default, the junit and list reporters are configured.
• Projects: Projects are used to run your test suite against multiple configurations simultaneously. For example, the Storefront Application runs tests against Safari, Firefox, and Chrome to ensure consistent quality across all major browsers.
  
  More information on how to configure the project to use multiple browsers or to execute the tests on different environments can be found in the Playwright documentation.

typescript

Copy

Environment Variables

As a prerequisite for running E2E tests that require specific data (e.g., for user authentication, search terms, or data-specific tests), you need to set up the appropriate environment variables.

The list of required environment variables is available in the templates/nuxt/.env.example file. They can be set in the working .env file, as well as in the respective CI tool, e.g. as environment variables in GitLab.

Storefront API credentials

Some end-to-end tests are dynamically fetching data directly from the backend. The prerequisite for them to run smoothly is to define the credentials environment variables as follows. These variables are usually set as part of the Storefront Application local setup, so they don't need to be set additionally for local end-to-end test execution. If the tests are running in the CI, then these variables should be set within the respective CI tool.

Test Users

Test users should be registered in the system, and their e-mail addresses and passwords should match the values of environment variables.

Search

As a prerequisite for successfully running the Search E2E tests, the following environment variables must be set.

Running Tests

Tests can be executed locally from playwright directory. By default, tests stored within templates/nuxt/testing/playwright/tests/ directory are executed when the command is triggered.

To aid debugging, you can run tests in headed mode. This launches a UI with a list of tests and configuration options, allowing you to run tests against specific browsers (e.g., Safari, Firefox, Chrome). After execution, the UI provides a visual, step-by-step breakdown of each test.

Headed mode can be executed using the command:

bash

Copy

Another way to run local tests is in headless mode. In this mode, a browser UI is not launched, but you can still view the test execution progress and results in the terminal.

Headless mode can be executed with this command:

bash

Copy

Best Practices

Tests Structured in Arrange-Act-Assert Pattern

Test files should be named with the *.spec.ts convention and stored in the playwright/tests/e2e directory. It's recommended to group all feature-related tests into a single file to maintain a clean and logical structure. For example, all tests for search functionality can be stored in a search.e2e.spec.ts file.

All tests should follow the Arrange-Act-Assert pattern, which clearly separates the setup, action, and validation steps. This structure is essential for clear and maintainable test code. The following example demonstrates this pattern for a test that verifies the display of search results.

• Arrange:

typescript

Copy

• Act:

typescript

Copy

• Assert:

typescript

Copy

Use the Page Object Model

The Page Object Model is a design pattern that makes your E2E tests more readable and maintainable. It works by abstracting away page-specific details from the test logic itself.

• What it is: It involves creating a class for each page or major component of your application. These classes contain the locators and methods needed to interact with that page.
• Why use it: The primary benefits are to prevent code duplication and improve test maintenance. If a UI element changes, you only have to update its locator in a single file—the Page Object file—instead of having to find and update it in every test that uses it.

In the Storefront Application's E2E tests, Page Object files for pages and components are stored within the /playwright/page-objects directory. This is where you can find and create the classes that represent your application's pages.

Creating a Custom End-to-End Test

Here's a step-by-step workflow for adding a new test to the Playwright suite.

Important note: the test used in the following example already exists in Storefront Application E2E suite.

• Define the Test Case: Clearly state what you want to test from a user's perspective. For example, "Verify that a user can add a product with multiple variants to the basket from the product details page."
• Create a new Spec File:
  • Test files are stored in the templates/nuxt/testing/playwright/tests/e2e directory.
  • Use the naming convention {page}.e2e.spec.ts. For our example, this would be pdp.e2e.spec.ts.
  • Group the tests related to the feature (e.g. PDP) into one .spec.ts file. In this case there is no need to create a new file, just add a new test to the existing pdp.e2e.spec.ts file.
  • If a test is for a newly introduced page, create a new spec file for it and all future related tests.
• Check for Existing Page Objects:
  • Look in the templates/nuxt/testing/playwright/page-objects directory for an existing Page Object file for the page or component you need to test. For our example, there is already a productDetailPage.ts file that can be used.
  • If you need a new Page Object for a different page or component, create it in the same directory and import it into your test.
• Write the Test:
  • Import test and expect from @playwright/test.
  • Use the test() function to define your test, giving it a descriptive title.
  • Use test.beforeEach to handle repetitive setup, such as navigating to the correct page before each test runs.
  • Follow the Arrange-Act-Assert pattern to structure your test logic.
• Run Your Test Locally:
  • To aid debugging, run your new test file in UI Mode to watch it execute step-by-step.
  • Command: npx playwright test tests/e2e/pdp.e2e.spec.ts --ui
  • Once it's passing reliably, run it headlessly to confirm.

Data-Driven Tests

Specific data-dependent tests should be designed to run on any environment or shop. This is achieved by utilizing dedicated Playwright fixtures that leverage the Storefront API to provide test data. This approach ensures your tests are reliable and not tied to specific, hardcoded data.

How Fixtures Work

The fixture availableMultiVariantProduct demonstrates this by using the @scayle/storefront-api package. It fetches a multi-variant product with an available stock quantity, ensuring the test can always run on a valid product, regardless of the environment.

typescript

Copy

StorefrontAPIClient is initialized in the /playwright/support/storefrontAPIClient.ts and it contains the function findProduct() which returns the first slice of 25 products. If the criteria are fulfilled (at least one multi-variant product with stock more than 1), then the search stops and the product list is returned.

The fixture is then used in the test as follows:

typescript

Copy

Benefits of fetching products in fixtures:

• Efficiency: The API call to get the products is now made only once per test worker. The result is then cached and reused for every test that runs on that worker.
• Cleaner Tests: Individual test files no longer need to handle the logic of fetching the products, checking for errors, or converting the data type. They simply declare products as a dependency and receive a pre-populated, validated product ID that meets the test criteria.
• Dynamic and realistic test data: Fixtures fetch data directly from the application's environment at the moment the test is run. This ensures that the test is always executing against the current state of the product catalog and validates the user journey with real, available products, just as a customer would. This avoids test failures caused by using hardcoded product IDs that may have been deleted, have gone out of stock, or have changed in a way that makes them unsuitable for the test.
• Decoupling and resilience: By fetching a product that meets a set of criteria (e.g., "is in stock and has multiple variants"), we decouple the test logic from specific data. The test's success becomes dependent on the Storefront Application functionality, rather than the existence of one particular product.
• Code reusability and maintainability: The logic for finding a suitable product (paging through results, checking stock, checking variant count) is complex. Fixtures allow you to define this logic in one central place. Any test that needs a specific type of product can simply request the fixture by name, without repeating the complex data-finding logic. This follows the "Don't Repeat Yourself" (DRY) principle, making the test files shorter, cleaner, and much easier to maintain.