End-to-End testing

End-to-End tests simulate real user scenarios, interacting with applications through the UI. This ensures that all application components work together as expected, increasing confidence in your software's quality and reducing the risk of bugs reaching production. Instead of focusing on isolated components, End-to-End tests validate the entire software flow, from the user interface down to the database and back, ensuring all components work together seamlessly.

Getting started

As End-to-End testing solution, the Storefront Boilerplate integrates Playwright, an open-source automation framework for cross-browser testing developed by Microsoft. It provides a stable environment for writing End-to-End tests that run smoothly across all major browsers using a single API. It features reliable cross-browser support and powerful tools for handling complex scenarios like multiple tabs, file uploads, mobile emulation, and native parallel test execution.

We recommend getting familiar with Playwright features and capabilities, before proceeding with this guide:

• Playwright Documentation

Setup & Configuration

The Playwright integration is handled as a dedicated sub-package within the Storefront Boilerplate. It is located within the playwright directory. For Playwright to function properly, we need to manually install its dependencies, as this is not done in parallel with the Storefront Boilerplate. The following command should be executed within the playwright directory:

sh

Copy

yarn playwright install

Playwright setup features can be checked and modified in playwright.config.ts file. Some of the most important are:

• Tests directory: The directory within the codebase to store test files.
• Base URL: Should be defined via an environment variable and uses localhost as a fallback.
  const BASE_URL = process.env.BASE_URL ?? 'https://localhost:3000/de/
• Parallelism: By default, all tests are executed in a parallel mode.
• Workers: Defines how many worker threads will be used during parallel test execution. By default in the CI we only use a single thread, when executed locally it will use all cores of your CPU.
• Reporters: Depending on the desired output format, the reporter can be modified within this section. By default junit, list and playwright-testrail-reporter are configured.

• Projects: A project allows you to run your test suite against multiple configurations. For example, it allows you to run your end-to-end tests against multiple browsers to ensure consistent quality for all of your users, regardless of the browser they choose. By default, the Storefront runs the tests against Safari, Firefox and Chrome.
  
  More information on how to configure the project to use multiple browsers or to execute the tests on different environments can be found in the Playwright documentation.

typescript

Copy

  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
...

Running the tests

Tests can be executed locally from playwrightdirectory. By default, tests stored within playwright/tests/are executed when the command is triggered.

To aid debugging, tests can be run in headed mode. Using headed mode, Playwright will start its UI with a list of tests and a configuration section to choose in which browsers you want to run the tests. After the tests are executed, UI allows you to check the execution step by step with the visual representation of every test step.

Headed mode can be executed using the command:

bash

Copy

yarn playwright test --ui

Another way of local test execution is to use headless mode. In that case browser will not be launched, but you will still be able to check the execution progress and test results in the terminal.

Headless mode can be executed using the command:

bash

Copy

yarn playwright test

Test structure

Every test file should be named as *.spec.ts and stored in playwright/tests/ directory.

Each .spec.ts file can contain one or more tests. It is recommended to group feature-related tests into one file. For example, all Search related tests are stored into e2e-Search.spec.ts file.

typescript

Copy

// ...
test.beforeEach(async ({ homePage, page }) => {
  await homePage.visitPage()
  // ...
})

test('C2139814: Verify Search no results page', async ({ search, page }) => {
  // ... test code ...
})

test('C2130650: Verify Search results page', async ({ search, page }) => {
  // ... test code ..  
})

test('C2130721: Verify Search suggestions', async ({ search }) => {
  // ... test code ...
})

test('C2132124: Verify Search suggestions "More" button', async ({
  search,
}) => {
  // ... test code ...
})

test('C2132173: Verify Search suggestions exact product match', async ({
  // ... test code ...
})

```

Page Object Model

Page Object Model is a design pattern that abstracts page-specific properties, locators, actions, and assertions.

We implemented it in our End-to-End tests to improve test maintainability and reduce code duplication. Every page has its page object file stored in playwright/page-objects/ directory. Page Objects group common actions and informations for a given page in a dedicated class to be used across the different tests. It is recommended that each page object file contains selectors and methods for only one page.

Page sections that can be used on various pages, and not specifically bonded to one page are stored in playwright/page-objects/components/ directory, such as breadcrumb.ts, header.ts, and more.

Directory structure for page objects and components example:

typescript

Copy

/page-objects
    /components
      breadcrumb.ts
      header.ts
      search.ts
      ...
  accountPage.ts
  basketPage.ts
  homePage.ts
  ...

The anatomy of a Page Object consists of three main sections:

• Properties
• Selectors
• Methods

Page object model example

Let’s take playwright/page-objects/components/search.ts as an example. Within this file, we have a class named Search.

Now we can check its anatomy:

• Properties
  • page: Stores a reference to the Page object from Playwright, providing access to browser interaction methods.
  • searchButton, searchInput, etc: Store the Locator objects for later use in methods.

typescript

Copy

import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
import { searchSuggestions } from '../../support/constants'

export class Search {
  readonly page: Page
  readonly searchButton: Locator
  readonly searchInput: Locator
  readonly searchCategoryListItem: Locator
  readonly searchResultsProductImage: Locator
  readonly searchResultsHeadline: Locator
  readonly searchResultsFlyout: Locator
  readonly searchMoreButton: Locator
  readonly searchExactProduct: Locator

  // ...

• Selectors
  • searchButton = page.getByTestId('header-search-button'): This selector locates DOM element search button to perform action in methods, such as clicking on it.
  • searchInput = page.getByTestId('header-search-input'): This selector locates DOM element search input field to perform action in methods, such as typing a search term in it.

typescript

Copy

  // ...
  constructor(page: Page) {
    this.page = page
    this.searchButton = page.getByTestId('header-search-button')
    this.searchInput = page.getByTestId('header-search-input')
    this.searchResultsFlyout = page.getByTestId('search-results-flyout')
    this.searchCategoryListItem = page.getByTestId('search-exact-product-item')
    this.searchResultsProductImage = page.getByTestId('product-image')
    this.searchResultsHeadline = page.getByTestId('headline')
    this.searchMoreButton = page.getByTestId('search-more-button')
    this.searchExactProduct = page.getByTestId('search-exact-product-item')
  }
  // ...

• Methods
  • clickSearchMoreButton(): checks if the button is visible and if yes, then clicks on it.
  • assertHeadlineSearchResults(): asserts if the headline contains search term and product count.

typescript

Copy

  // ...
  async executeSearch(searchTerm: string) {
    await this.searchInput.click({ force: true })
    await this.searchInput.fill(searchTerm)
    await this.searchInput.press('Enter')
  }

  async startTypingSearch(searchTerm: string) {
    await this.searchInput.click({ force: true })
    await this.searchInput.fill(searchTerm)
    await expect(this.searchResultsFlyout).toBeVisible()
  }

  async assertSearchCategorySuggestions(searchTerm: string) {
    await expect(this.searchCategoryListItem.first()).toBeVisible()
    await this.searchCategoryListItem.first().click()
    await this.page.waitForURL(searchSuggestions.plpUrl)
    await this.page.waitForLoadState('networkidle')

    const pageUrl = this.page.url()
    expect(pageUrl).toContain(searchTerm)
  }

  async assertHeadlineSearchResults(
    searchTerm: string,
    searchCount: string = '',
  ) {
    await expect(this.searchResultsHeadline.first()).toContainText(searchTerm)
    await expect(this.searchResultsHeadline.first()).toContainText(searchCount)
  }

  async clickSearchMoreButton() {
    expect(this.searchMoreButton).toBeVisible()
    await this.searchMoreButton.click()
  }

  async clickExactProductItem() {
    expect(this.searchExactProduct).toBeVisible()
    await this.searchExactProduct.click()
  }

  async assertPdpIsLoaded() {
    const pageUrl = this.page.url()
    expect(pageUrl).toContain(searchSuggestions.pdpUrl)
  }

In summary

This Search class encapsulates the logic for interacting with a search component. The selectors target specific elements, the methods provide reusable actions and assertions, and the properties manage the state of the page object. This promotes code reusability and maintainability in your test suite.

Recommended practices in Page Object Model

Separate actions and assertions

One of the best practices is to separate Action and Assertion methods. In theory, these actions and assertions from the following example could be written within one method, like this:

typescript

Copy

async executeSearch(searchTerm: string) {
    await this.searchInput.click({ force: true })
    await this.searchInput.fill(searchTerm)
    await this.searchInput.press('Enter')
    // ...
    await expect(this.searchResultsHeadline.first()).toContainText(searchTerm)
    await expect(this.searchResultsHeadline.first()).toContainText(searchCount)
    // ...
    expect(this.searchExactProduct).toBeVisible()
    // ...
}

This code would work, but it is better to separate actions from assertions. The reason is simple - separating them into their methods makes the code more reusable.

typescript

Copy

  async executeSearch(searchTerm: string) {
    // ...
  }

  async startTypingSearch(searchTerm: string) {
    // ...
  }

  async assertSearchCategorySuggestions(searchTerm: string) {
    // ...
  }

  async assertHeadlineSearchResults(
    // ...
  }

  async clickSearchMoreButton() {
    // ...
  }

  async clickExactProductItem() {
    // ...
  }

  async assertPdpIsLoaded() {
    // ...
  }

Keep Page Object as small as possible

Each page or page section should have its own page object, ensuring the code is organized and easy to maintain. This brings clear boundaries among different feature areas of the Storefront Boilerplate application.

Avoid extending Page Objects

Extending page objects can lead to complex pages with too many methods and properties, which might be a challenge to understand and maintain.

typescript

Copy

class AuthenticationPage {}
class SignInPage extends AuthenticationPage {}
class SignUpPage extends AuthenticationPage {}

Do not use state in Page Object

Although the assertion methods are OK to use in Page Object, as shown earlier, using state is not recommended. Avoiding this will ensure that the Page Object is reusable across multiple tests and it makes it more generic.

typescript

Copy

import { type Page, expect } from '@playwright/test'

class SignInPage {
  authenticated: boolean
  // ...
  async isSignedIn() {
    await expect(this.page.getByTestId('toast-info')).toHaveText('The login has been successful')
    this.authenticated = true
  }
}

Selectors and Locators

Selectors have been already mentioned in the previous chapter. In this part, some best practices on how to use them will be explained.

• Selector - search query used to match elements within DOM.
• Locator - abstraction built on top of selectors.

In general, it is recommended to use data attributes in selectors whenever possible. In the following example, emailInput selector uses the attribute placeholder to locate the element.

this.emailInput = page.getByPlaceholder('E-Mail-Adresse')

This is fine and resilient. The same element can be located by using XPath or CSS selector. But, there is one way that is the most recommended in Playwright. It uses a data attribute named data-testid. In case when your DOM element has this attribute, e.g. data-testid=input-email-address, you can easily design your selector as follows:

this.emailInput = page.getByTestId('input-email-address')

The prerequisite is to add these data attributes to all relevant DOM elements, so it requires a little bit more syncing with developers.

However your tests become more clear about which elements it targets and more resilient to changes in your codebase, e.g. changing labels or a changing markup structure.

Assertions

Assertions are checkpoints in the code that verify if the application is behaving the way as expected to. Instead of just passively running the code, assertions actively examine the outcome.

Playwright includes test assertions in the form of expect function. To make an assertion, call expect(value) and choose a matcher that reflects the expectation.

It is recommended to use Web-first assertions due to the clear benefits you can get in your tests. Some of the advantages of Web-First Assertions in Playwright are:

• Resilience to Change: Web-first assertions are less likely to break when you make minor UI updates. As long as the way a user interacts with the element (e.g., clicking a button, reading text) remains consistent, your tests will likely still pass.
• More User-Centric: Your tests become closer to real user flows. You're verifying the behavior and functionality that actually matter to your users, not just the underlying code.
• Improved Readability: Web-first assertions tend to be more descriptive and easier to understand. For instance, instead of checking if an element has a specific CSS class, you can directly assert if it's visible or enabled.

typescript

Copy

expect(await this.productInBasket.first().isVisible()).toBe(true)

typescript

Copy

await expect(this.productInBasket.first()).toBeVisible()

Fixtures

Fixtures are used to set up the test environment and provide access to the browser and page objects. Playwright Test is based on the concept of test fixtures. Test fixtures are used in every standard Playwright test, for example:

import { test, expect } from '@playwright/test'

To make our tests more organized, to reuse as much code as possible, and to keep our tests flexible and clean, it is recommended to create our fixtures.

Example of using fixtures in Playwright End-to-End tests in Storefront Boilerplate

Checking the file playwright/fixtures/fixtures.ts, we can see how the fixture can be created. The example shows how the SignInPage Page Object is setup and provided as a fixture to the tests.

typescript

Copy

import { test as base } from '@playwright/test'
// ...
import { SignInPage } from '../page-objects/signinPage'
// ...

interface Fixtures {
  // ...
  signinPage: SignInPage
  // ...
}

export const test = base.extend<Fixtures>({
  // ...
  signinPage: async ({ page }, use) => {
    const signinPage = new SignInPage(page)
    await use(signinPage)
  },
  // ...
})
export { expect } from '@playwright/test'

Later on, in our tests, in .spec.ts files, we can use the fixture as follows:

typescript

Copy

import { test } from '../fixtures/fixtures'

// ...

test('C2130648: Verify User login and log out', async ({
  signinPage,
  page,
}) => {
  // ... the rest of the test code
  await signinPage.clickLoginButton()
  // ... the rest of the test code
})

test('C2130649: Verify User login with wrong credentials', async ({
  homePage,
  signinPage,
}) => {
  await homePage.visitPage()
  // ... the rest of the test code
  await signinPage.assertLoginButtonIsVisible()
})

This allows us to easily use all page object models in all tests without having to construct them in every single test.

Handling Asynchronous Operations

• Await asynchronous actions: Ensure that asynchronous operations like page navigation or network requests are completed before proceeding with assertions.
• Use Playwright's built-in waiting mechanisms: Leverage waitForNavigation, waitForSelector, waitForURL, etc., to handle dynamic content and asynchronous actions effectively.

Example - in tests/verifyUserLogin.spec.ts you can find waitForURL() which ensures that the URL has been loaded after some time needed to load when clicking on the Login button.

typescript

Copy

  ...
  await signinPage.clickLoginButton()
  await page.waitForURL('/de')
  ...

Writing Storefront End-to-End Tests

Some of the recommended practices on how to write End-to-End tests are explained as follows:

• Keep tests concise and focus on a single scenario: Focus each test on a single user scenario or functionality.
• Use descriptive names: Test names should indicate the scenario being tested.
• Follow the Arrange-Act-Assert pattern:
  • Arrange: Set up the necessary preconditions for the test.
  • Act: Perform the actions that trigger the functionality under test.
  • Assert: Verify the expected outcome.

As an example, test file e2e-UserLogin.spec.ts has two tests. The first one checks user login and logout using the correct credentials. The second one checks the user login with incorrect credentials. That makes both tests focused on one single scenario. Furthermore, both tests have a descriptive and self-explanatory name.

typescript

Copy

import { test } from '../fixtures/fixtures'

const LOGGED_IN_USER_DATA = {
  email: '[email protected]', 
  password: 'T3st!Passw0rd',
}

const LOGIN_WRONG_CREDENTIALS = {
  email: '[email protected]',
  password: 'wrong-pass',
}

test('C2130648: Verify User login and log out', async ({
  homePage,
  signinPage,
  header,
  accountPage,
  toastMessage,
  page,
}) => {
  await homePage.visitPage()
  await header.clickLoginHeaderButton()
  await signinPage.fillLoginData(
    LOGGED_IN_USER_DATA.email,
    LOGGED_IN_USER_DATA.password,
  )
  await signinPage.clickLoginButton()
  await page.waitForURL('/de')

  await header.clickLoginHeaderButton()
  await toastMessage.assertToastInfoIsVisible()
  await toastMessage.clickToastMessageButton()
  await toastMessage.assertToastInfoNotVisible()
  await accountPage.assertLogoutButtonIsVisible()
  await accountPage.clickLogoutButton()

  await header.clickLoginHeaderButton()
  await signinPage.assertLoginButtonIsVisible()
})

test('C2130649: Verify User login with wrong credentials', async ({
  homePage,
  signinPage,
  header,
  toastMessage,
}) => {
  await homePage.visitPage()
  await header.clickLoginHeaderButton()
  await signinPage.fillLoginData(
    LOGIN_WRONG_CREDENTIALS.email,
    LOGIN_WRONG_CREDENTIALS.password,
  )

  await signinPage.clickLoginButton()
  await toastMessage.assertToastInfoIsVisible()
  await toastMessage.clickToastMessageButton()
  await toastMessage.assertToastInfoNotVisible()
  await signinPage.assertLoginButtonIsVisible()
})

Tests follow the Arrange-Act-Assert pattern:

• Arrange:
  • await homePage.visitPage(): Navigate to the home page.
  • await header.clickLoginHeaderButton(): Click the login button.
• Act:
  • await signinPage.fillLoginData(...): Enter valid login credentials.
  • await signinPage.clickLoginButton(): Submit the login form.
• Assert:
  • await page.waitForURL('/de'): Verify successful login by checking the URL.
  • await toastMessage.assertToastInfoIsVisible(): Assert login notification is visible
  • await signinPage.assertLoginButtonIsVisible(): Verify the user is back to the login page.