Bulk Requests

Admin API Bulk Requests

The Admin API Bulk Request feature is designed to enable asynchronous processing of slow or complex operations in bulk. Instead of submitting the entire bulk request at once, it allows you to compose the request incrementally by sending multiple smaller requests to add more operations. Once all operations have been added, it is possible to mark the bulk request as complete so that the system starts processing it.

Whitelisting And Limits

Currently, there are the following restrictions in place:

Only the following Admin API operations are allowed for bulk operations:
- POST /products and PUT /product/{productIdentifier}
- POST /products/composite and PUT /products/composite/{productIdentifier}
- POST /variants/{variantIdentifier}/prices
Operation-Level Limits:
- POST /products and PUT /product/{productIdentifier}: 10000
- POST /products/composite and PUT /products/composite/{productIdentifier}: 10000
- POST /variants/{variantIdentifier}/prices: 100000
Global Limit:
- A global total number of running operations across all scopes.
- Default: 100000

When receiving a new bulk request, Admin API validates the following thresholds:

the number of running operations (existing running operations + those in the new request) per scope (if the scope-specific limit is set)
the total number of running operations

Bulk Request Statuses

Bulk Request Statuses describe the different stages in the lifecycle of a bulk request. These include:

NEW: The bulk request has been created but is not yet ready for processing, as some of the request's operation chunks are still pending.
PENDING: All chunks of the bulk request have been received and validated. The request is queued, waiting to be processed.
PROCESSING: At least one operation in the bulk request is currently being processed by a worker.
PROCESSED: All operations in the bulk request have been processed, either successfully or with failures.
FAILED: An error occurred during the bulk request's lifecycle, preventing it from being processed fully.
CANCELLED: The client has explicitly cancelled the bulk request before completion.
ABORTED: The system has terminated the bulk request because it exceeded the maximum allowed processing time (default: 12 hours).

Operation Statuses

Operation Statuses describe the stages of an individual operation within a bulk request:

NEW: The operation is newly created and not yet assigned to a worker.
PROCESSING: A worker has started executing the operation.
PROCESSED: The operation has been completed successfully.
FAILED: An error occurred during the operation’s execution.

Lifecycle Flow

Lifecycle Flow outlines the progression of a bulk request and its individual operations:

Bulk Request: NEW → PENDING → PROCESSING → PROCESSED or FAILED.

Individual Operation: NEW → PROCESSING → PROCESSED or FAILED

Webhook Events

Whenever a request or operation transitions to a new state (except CANCELLED), the client is notified through the webhook callback URLs provided during the request creation.

bulk-request-status-changed

The callback will be triggered when request status is changed.

{
  "key": "string",
  "meta": {
    "xRequestId": "string"
  },
  "occurredAt": "datetime",
  "type": "string",
  "payload": {
    "key": "integer",
    "status": "string",
    "progress": {
      "totalOperations": "integer",
      "processedOperations": "integer",
      "failedOperations": "integer"
    }
  }
}

bulk-operation-status-changed

The callback will be triggered when operation status is changed.

{
  "key": "string",
  "meta": {
    "xRequestId": "string"
  },
  "occurredAt": "datetime",
  "type": "string",
  "payload": {
    "key": "string",
    "status": "string",
    "response": {
      "body": "object",
      "statusCode": "integer"
    },
    "bulkRequest": {
      "key": "integer",
      "status": "string",
      "progress": {
        "totalOperations": "integer",
        "processedOperations": "integer",
        "failedOperations": "integer"
      }
    }
  }
}

Entities

CreateBulkRequest

field name	type	description
`callbacks`	`BulkRequestCallbacks`	Defines the structure for callback URLs and related data associated with bulk request processing
`operations`	`Array<CreateBulkOperation> required`	Defines the structure for creating an individual operation within a bulk request

BulkRequestCallbacks

field name	type	description
`requestUrl`	`string`	Callback url, which will be triggered when request status is changed
`operationUrl`	`string`	Callback url, which will be triggered when operation status is changed
`requestStatus`	`string enum(complete, incomplete)`	Status of the bulk request creation
`bulkRequestKey`	`integer`	Key indicating the bulk request id. By passing this `bulkRequestKey` with requestStatus=incomplete, the operations of that bulk request can be extended. When extending the bulk operations, same operation key that may exists in the previous bulk requests will not be checked. Extending the operations of an already completed request is not possible. When not provided a new bulk request is created

CreateBulkOperation

field name	type	description
`key`	`string required`	Operation key, which is unique within the given bulk request
`url`	`string required`	Operation relative url
`method`	`string enum (GET, HEAD, POST, PUT, PATCH, DELETE) required`	Operation HTTP method
`headers`	`object`	Operation additional HTTP headers
`body`	`object`	Operation request body

BulkRequest

field name	type	description
`key`	`integer`	Bulk request identifier

BulkRequestStatus

field name	type	description
`key`	`integer required`	Bulk request identifier
`status`	`string enum(new, pending, processed, failed, cancelled, aborted) required`	Status of the bulk request
`progress`	`BulkRequestProgress`	Current progress of the bulk request

BulkRequestProgress

field name	type	description
`totalOperations`	`integer required`	Total number of operations
`processedOperations`	`integer required`	Number of processed operations
`failedOperations`	`integer required`	Number of failed operations

BulkOperationStatus

field name	type	description
`key`	`string required`	Operation key, that is unique within the given bulk request
`status`	`string enum(new, processing, processed, failed) required`	Status of the bulk operation
`response`	`BulkOperationResponse`	Operation response data. Can be null if operation hasn't been processed yet
`bulkRequest`	`BulkRequestStatus`	Status of the bulk request

BulkOperationResponse

field name	type	description
`body`	`object required`	Operation response body. Can be null if operation doesn't return any response (e.g. 204 status code)
`statusCode`	`integer required`	Operation response status code.

Create Product Bulk Request

Initiate a bulk operation to create or update multiple products.

Example

const newBulkRequest = {
    "callbacks": {
        "requestUrl": "https://example.com/request",
        "operationUrl": "https://example.com/operation",
        "requestStatus": "complete"
    },
    "operations": [
        {
            "key": "o1",
            "url": "/products",
            "method": "POST",
            "body": {
                "referenceKey": "M0001-black",
                "name": {
                    "de_DE": "Blaues Shirt",
                    "en_GB": "Blue Shirt"
                },
                "state": "draft",
                "master": {
                    "referenceKey": "M0001",
                    "categories": {
                        "paths": [
                            ["Fashion", "Men", "Shirts"]
                        ]
                    }
                }
            }
        }
    ]
  };

const response = await client.apis.Products.createProductBulkRequest({}, {requestBody: newBulkRequest});
const bulkRequest = response.body;

Create Composite Product Bulk Request

Initiate a bulk operation to create or update multiple composite products.

Example

const newBulkRequest = {
    "callbacks": {
        "requestUrl": "https://example.com/request",
        "operationUrl": "https://example.com/operation",
        "requestStatus": "complete"
    },
    "operations": [
        {
            "key": "o1",
            "url": "/products/composite",
            "method": "POST",
            "body": {
                "referenceKey": "M0001-black",
                "name": {
                    "de_DE": "Blaues Shirt",
                    "en_GB": "Blue Shirt"
                },
                "state": "draft",
                "master": {
                    "referenceKey": "M0001",
                    "categories": {
                        "paths": [
                            ["Fashion", "Men", "Shirts"]
                        ]
                    }
                }
            }
        }
    ]
  };

const response = await client.apis.Products.createCompositeProductBulkRequest({}, {requestBody: newBulkRequest});
const bulkRequest = response.body;

Create Product Variant Price Bulk Request

Initiate a bulk operation to set prices for product variants.

Example

const newBulkRequest = {
    "callbacks": {
        "requestUrl": "https://example.com/request",
        "operationUrl": "https://example.com/operation",
        "requestStatus": "complete"
    },
    "operations": [
        {
            "key": "o1",
            "url": "/variants/123/prices",
            "method": "POST",
            "body": {
              "price": 80,
              "oldPrice": 100,
              "recommendedRetailPrice": 90,
              "buyingPrice": 70,
              "tax": 19.0,
              "countryCode": "DE",
              "currencyCode": "EUR",
              "groupKey": "myGroupKey",
              "promotionKey": "myPromotionKey",
              "unitPrice": {
                "unit": "ml",
                "amount": 100,
                "price": 399
              },
              "validFrom": "2021-09-23T11:30:58+00:00",
              "validTo": null,
              "merchantReferenceKey": "merchant-1"
            }
        }
    ]
  };

const response = await client.apis.Prices.createProductVariantPriceBulkRequest({}, {requestBody: newBulkRequest});
const bulkRequest = response.body;

Get List of Bulk Request Statuses

Retrieve the statuses of all bulk requests to monitor their progress and outcomes.

Example

const response = await client.apis.BulkRequests.getBulkRequestStatuses({
      "with": "progress",
  });

const bulkRequestStatuses = response.body.entities;

bulkRequestStatuses.forEach(
  bulkRequestStatus => console.log(bulkRequestStatuses.key)
);

Get Status Of Bulk Request

Fetch detailed information about the status of a specific bulk request using its unique identifier.

Example

const response = await client.apis.BulkRequests.getBulkRequestStatus({requestKey: 123}, {
      "with": "progress",
  });

const bulkRequestStatus = response.body;

console.log(bulkRequestStatus.key);

Get Status Of Bulk Operation

Retrieve the status of a specific operation within a bulk request.

Example

const response = await client.apis.BulkRequests.getBulkOperationStatus({requestKey: 123, operationKey: "operation-1"}, {
      "with": "response,bulkRequest,bulkRequest.progress",
  });

const bulkOperationStatus = response.body;

console.log(bulkOperationStatus.key);

Cancel Bulk Request

Cancel a bulk request that is in progress.

Note, that the result of already processed operations will not be reverted.

Example

await adminApi.apis.BulkRequests.cancelBulkRequest({requestKey: 123});