Mapping the Data

General

In SCAYLE, master feeds are your go-to for exporting product information. They cover everything from a product's name, attributes, and images, to its availability and pricing. These details are all available in the Storefront API, ready for you to use and shape into a master feed. This introduction will show you how to leverage the SCAYLE Storefront API to effectively craft these feeds.

Use cases

Master feeds in SCAYLE play a crucial role in several scenarios. Whether you’re integrating with marketplaces, enhancing your CRM integrations, or even optimizing BI tools for sorting product details, master feeds have got you covered. Imagine you’re selling products via SCAYLE in different online stores, and now you plan to expand into marketplace platforms. In this context, you’ll need a product master feed that provides the necessary details to the marketplace or a third-party tool managing marketplace interactions. This feed is essential, and the data within SCAYLE often serves as the primary source, determining whether a product should be available for sale or not.

CRM integrations

Consider also CRM integrations, where you aim to engage customers with newsletters highlighting products they might find interesting. Master feeds ensure that your CRM is updated with the latest and most relevant product information.

Shops in multiple countries

When operating internationally, you might need to create distinct feeds for each country where your shop is present. For example, updates on stock levels might be necessary every few hours, while other details may only require daily updates. This tutorial will guide you through the process of crafting these master feeds, focusing on essential aspects for their successful creation.

Create a master feed

To create a master feed, you’ll need to build a middleware. This middleware will retrieve data from SCAYLE, transform it as needed, and then publish it to any third-party. The diagram below shows the basic idea of creating a master feed:

As you can see the data is fetched from SCAYLE’s Storefront API using middleware. The diagram doesn't illustrate the transformation process of this data or its final destination.

Next, we’ll provide a deeper exploration into developing your master feed for any planned third-party integration.

Before you start

Ensure you’re familiar with SCAYLE’s Storefront API and how to get information using the /products endpoint.

Determine feed requirements

Before diving into master feed creation, it's crucial to determine the specific needs of this feed. Different systems might have varying requirements. While gathering these requirements you should also check the required data format and how the feed should be provided. Additionally, consider if the feeds need to be localized or internationalized, as this will most likely result in multiple feeds being generated. Feeds are typically stored on an SFTP server, a bucket, or provided directly via API. Once you've gathered all the requirements, you're ready to begin building your master feed.

Typical questions to address beforehand include:

• What specific data is needed?
• In what format should the data be?
• How and where should the data be provided?
• Will you require multiple files for different scenarios?
• Is there a need to localize or internationalize the data?
• How often should the feed be generated?

Generate a master feed

Generating a master feed should be based on SCAYLE’s Storefront API, which offers the products endpoint to extract all available product data. With this endpoint, you can add further information using the with= parameter to get a more detailed response.

For instance, let's say you want product data that includes attributes, advanced attributes, variants, and variant attributes for a shop with the ID 10001. Here's how your API request to SCAYLE might appear:

http

Copy

{{baseUrl}}/v1/products?with=attributes, advancedAttributes,variants,variant.attributes&shopId=10001

Note that adding multiple values to the with= parameter can increase the size of the response drastically. We recommend limiting the request to only receiving the necessary data .

For example, adding with=attributes can particularly increase the size significantly. To limit this, you can fetch only the relevant attributes you need, like so:

with=attributes:key(attributeKey1|attributeKey2|...|attributeKeyn).

It's important to evaluate if both attributes and advanced attributes need to be included in your requests. As these can be queried as separate parameters, it's crucial to tailor your request based on your specific needs. For instance, to access only certain relevant attributes, your request could be structured as follows:

http

Copy

{{baseUrl}}/v1/products?with=attributes:key(color|material),variants,variant.attributes&shopId=10001

For comprehensive guidance on what can be included within the with= parameter, check our Developer Guide.

Keep in mind that our Storefront API is tailored to be shop-country specific. This means that attributes and other translatable data you receive will be in the language specific to each country. To effectively provide localized product data across various countries, you will likely need to create separate master feeds for each shop country. This ensures that your data remains accurate and relevant for each shop country.

Handling pagination

SCAYLE’s storefront API uses pagination to limit the response size. You’ll need to iterate over the paginated responses to compile complete data. Specify the number of items per page using the perPage parameter and the page number with the page parameter. When increasing items per page, monitor response times to avoid long loading times.

Mapping the data

Once you have identified the data requirements and familiarized yourself with SCAYLE's Storefront API, the next step is mapping SCAYLE data to the format required by the third party. We recommend developing a detailed mapping chart. This chart should include each required data element, its type, an illustrative example, the corresponding element in SCAYLE's API response, and a brief description. Here's an example to guide you:

Providing your feed

After successfully creating your master feeds, the next crucial phase is making them accessible to your third-party partners. Each partner may have specific requirements regarding how they receive the data. Common methods include uploading files to an SFTP server, or a cloud storage bucket, or pushing data directly through an API. Key considerations include understanding not just how to provide the feed, but also how frequently it needs to be updated and delivered. Some common questions to address are:

• How should the data be provided to each third-party system?
• How often is data refresh required by each third-party?

Tailoring the delivery and update frequency to meet third-party requirements ensures the data remains relevant, accurate, and valuable for its intended purpose.