Search

How does it work?

SCAYLE’s search follows a category-first approach. This means we first attempt to resolve a search term to a specific shop category that contains relevant filterable attribute groups. We fall back to a generic text-based search across your entire product assortment if no matching category is found. Additionally, as the user types their search term, suggested matching categories are displayed.

Benefits of using categories for search

Leveraging categories for search offers several advantages, particularly when paired with Search Analytics, which allows you to track and understand what search terms users are entering. From our experience, most search terms fall under broader product categories. Utilizing your existing categories enhances the search experience in four key ways:

1. Exact Catalog & Sorting
   Category pages allow you to define the exact catalog of products for each category. You can also use sorting to control how the products in each category are ordered.
2. Specific Filters
   Category pages come with predefined, detailed filters in the SCAYLE Panel, enabling users to refine their search even further. These filters should be applied on the frontend to ensure users can narrow down search results effectively.
3. Customized Content
   Category pages can feature customized content from a CMS, which enhances both the user experience and SEO performance.
4. Efficient Maintenance
   Category pages, including their filters, content, and sorting, are regularly maintained, ensuring your search experience remains up-to-date and optimized.

Configuration & Customization

Optimizing your search performance starts with effective configuration and customization. Using the SCAYLE Panel, search managers can easily adjust shop categories, filterable attribute groups, and product attributes to ensure more accurate and relevant search results. Tailoring these settings directly impacts how users interact with your product catalog, leading to improved search efficiency and user satisfaction.

To dive deeper into specific search concepts and best practices, explore the following sections:

• Entities
• Exclusion
• Resolving
• Search Content
• Search Term
• Stemming
• Suggestions
• Synonyms
• Tokenization
• Typo Tolerance

Entities

We currently store the following entities in our Search Database:

Exclusion

Certain products and categories can be excluded from the search endpoints in the SCAYLE Panel. If a product or a category is excluded, it will not appear in any response of /v2/suggestions or /v2/resolve. Additionally, to exclude a specific product from the search, the system attribute isExcludedFromSearch is used.

You can find more information on how to do a category exclusion and product exclusion in our User Guide.

Stemming

Stemming is the practice of reducing words to their root form to improve accuracy and matching. For example, stemming converts plural forms to singular or removes tenses from words.

We apply stemming both to the searched content and the search term.

Synonyms

Synonyms are alternate spellings of the same meaning or word; a good example would be different sayings of "Hello" in a particular language or dialect.

For example, "Hallo" has the same meaning as "Moin" or "Servus" in German, so they are synonyms for the same word.

Using synonyms, you can help your users find the correct content if they use different wording or redirect specific terms to a more fitting search result.

SCAYLE supports four types of synonyms:

• Category synonyms
• Attribute synonyms
• Word synonyms
• Navigation item synonyms

See the User Guide to learn more about how you can configure your synonyms.

Examples

Imagine you have a category called "Sweaters"; however, in your Search Tracking, you can see a lot of unresolved searches for the term “Pullover“.
Since these are technically two different terms, the SCAYLE Search by default won’t be able to correctly resolve “Pullover“ to the “Sweaters“ category.

However, setting the synonym “Pullover“ on the category will resolve the searches to the correct category, leading to an improved search result.

Tokenization

Tokenization refers to splitting a term into multiple searchable and standalone tokens.

A token is usually a single word; for example, in the term Blue Pants, there are two tokens: Blue and Pants.

We apply tokenization to the searched content and the search term.

The tokenization happens based on a set of separating characters; in our case, these are only whitespace where we split our search term.

We use different tokenizers to optimize our results depending on the use case of the API endpoint.

Typo Tolerance

Typo tolerance, also known as "fuzziness", returns results similar to the search term but not precisely matched.

The Levenshtein Edit Distance defines the difference between two words; it is the number of one-character changes needed to turn the search term (what the user typed in) into the searched content (what is stored in the database).

These changes include changing a character, removing a character or inserting a character.

The search term must match the first character, allowing a specified number of mistakes thereafter:

Suggestions

When the searched term returns a match on the resolve endpoint, that match will also rank highest on the suggestions endpoint. Refer to the resolve documentation for a more detailed explanation of that logic.

The endpoint also works as an autocomplete and will match categories while a user is still typing. The autocomplete matching supports stemming (in supported languages) and supports synonyms as well as phonetic matching. It does not have any typo tolerance, but can match mistyped terms due to phonetic matching and stemming. The more accurately the searched term matches the category, the higher the category will rank.

Example:

• Category 1:
  • name: "sweatpants"
  • filters: "color"
  • product count: 100
• Category 2:
  • name: "sweaters"
  • filters: "color"
  • product count: 50

When searching for "blue swet" the returned results would be:

• Category 1 with filter "blue" applied
• Category 2 with filter "blue" applied

Both categories would match due to phonetic matching ("swet" and "sweat" match phonetically), but Category 1 will rank higher due to a higher product count.

Examples

Typescript

Copy

const response = await client.searchv2.suggestions(
  "red Pants",
  {
    with: {
      categories: {
        parents: 'all'
      }
    }
  }
);

console.log(`Suggestions count: ${response.suggestions.length}`);

Restrict the search to a subset of categories

Restricting the results to a specific category can be helpful when you, for example, sell products for women and men; however, in your search, you might only want to get results for women or men, depending on the user's choice.

Typescript

Copy

const response = await client.searchv2.suggestions(
  "red Pants",
  {
    categoryId: 20201,
    with: {
      categories: {
        parents: 'all'
      }
    }
  }
);

console.log(`Suggestions count: ${response.suggestions.length}`);

Resolve

While clicking on a suggestion is one action from a user, it’s not the only one.

Some users will just press "Enter" when they finish typing in their search terms and expect to be redirected to the correct content.
To satisfy that need, we have the resolve API. It should be called with the user's search term, once the user presses "Enter".

The resolve API is designed to return a single result that is the best match to the search term. It should be used when the user types in the full search term and presses "Enter". To be as accurate and correct as possible, the API only works with complete search terms.

The result is based on your categories and filterable attribute groups, and your searchable navigation items.
A matching product is only returned if there is an exact match between the search term and the product identifiers.

This endpoint will provide insights to the Search Analytics in the SCAYLE Panel. There, you will find insights on the different ways in which user searches are resolved, the ratio or percentage of searches that successfully led to a resolution and how these factors are developing over time. Find more information on how that dashboard works in the User Guide.

Logic

The endpoint will resolve to a category when at least one of the typed words matches a category name or its synonyms. When the searched term also matches a filter on that category and the combination of that category and applied filter results in at least one product, then the filter is added to the response.

If multiple categories match, the categories that have a higher product count or the ones that are positioned higher on the category tree will rank higher.

The endpoint supports stemming and will have a tolerance of 1 typo on words longer than 5 characters.

Example:

• Category 1:
  • name: "sweaters"
  • filters: "color"
  • product count: 100
• Category 2:
  • name: "sweaters"
  • filters: "color"
  • product count: 50

When searching for "blue sweater", the returned results would be Category 1 with the filter "blue" applied. Both categories would match, but Category 1 has a higher product count and so will be returned.

Example

Typescript

Copy

const response = await client.searchv2.resolve(
  "red Pants",
  {
    with: {
      categories: {
        parents: 'all'
      }
    }
  }
);

console.log(`Resolved Entity: ${response?.type}`);

Restrict the search to a subset of categories

Restricting the results to a specific category can be helpful. For example, when you sell products for women and men. In your search, you might only want to get results for women or men, depending on the user's choice.

Typescript

Copy

const response = await client.searchv2.resolve(
  "red Pants",
  {
    categoryId: 20201,
    with: {
      categories: {
        parents: 'all'
      }
    }
  }
);

console.log(`Resolved Entity: ${response?.type}`);

Text Search

For certain search terms, the resolve endpoint cannot match the search term to a specific category or product.

In this case, we offer a simple text-based search using all of your assortment.

We leverage the term parameter of the products endpoint for that.

Logic

For example, where name and attribute1 are set as searchable through the SCAYLE Panel:

• Product 1:
  • name: "Christmas sweater"
  • attribute 1: "light blue"
• Product 2:
  • name: "Pants"
  • attribute 1: "blue"
• Product 3:
  • name: "Cardigan sweater"
  • attribute 1: "red"
• Product 4:
  • name: "Pants"
  • attribute 1: "red"

When searching for "blue sweater" the returned results would be:

• Product 1 - it will rank the highest as both "blue" and "sweater" match
• Product 3 - it will be returned because the searched word "sweater" matches the name
• Product 2 - it will be returned because the searched word "blue" matches the attribute 1. By default, it will rank lower than Product 3 as the name is given higher relevance than attribute 1. However, this can be adjusted through the SCAYLE Panel.

Product 4 will not be returned as the searched term does not match the name or attribute 1.

Search Analytics and Tracking

Our Storefront API provides basic search tracking out of the box, which captures information on whether searches are resolved or unresolved and whether results are found, based on categories and search terms.

For complete tracking, including features like search suggestions and detailed user interactions, you will need to implement a dedicated tracking tool, as the built-in tracking only covers core metrics.

SCAYLE Search Analytics

The tracked data is displayed in SCAYLE Panels' Search Analytics feature, where you can view insights into user searches and their results. It analyzes requests made through the /v2/resolve and v1/products?filters[term] endpoints. To ensure proper tracking of search terms, it is crucial to use trackSearchAnalyticsEvent parameter for the v1/products?filters[term] endpoint as shown in the example below. Without it, search terms will not be tracked.

Search Flow:

1. Use the /v2/resolve endpoint to find matching results for the search term.
2. If no results are found, proceed with the products search endpoint (v1/products?filters[term]) and include the trackSearchAnalyticsEvent parameter.

You can use the insights gained to improve your search configuration. For example, if you find a lot of not resolved search terms related to a category that is named differently in your shop, you should consider implementing synonyms for that category.

Example: Text Search with Search Analytics Tracking

In this example, we are performing a product search for the term "shirt" by requesting up to 100 results per page and tracking the search in the Search Analytics. As the trackSearchAnalyticsEvent parameter would count each request as a new search attempt, it is set only when fetching the first page. When requesting further pages, the parameter should be left out.

Typescript

Copy

let currentPage = 1

while (true) {
  const products = await client.products.query({
    trackSearchAnalyticsEvent: currentPage == 1,
    pagination: {
      page: currentPage,
      perPage: 100,
    },
    where: {
      term: "shirt",
    },
    with: {
      attributes: {
        withKey: ["name"],
      },
    },
  });

  if (products.entities.length == 0) {
    break
  }

  currentPage++

  console.log(products.entities[0].attributes.name.values.label); 
}