# Shopify Scraper (`autofacts/shopify`) Actor Shopify online store collection and product data extractor. Supports realtime price/stock monitor. Crawl product list or single product in a structured form, including title,description,price,sku, etc. - **URL**: https://apify.com/autofacts/shopify.md - **Developed by:** [Richard Feng](https://apify.com/autofacts) (community) - **Categories:** Developer tools, E-commerce - **Stats:** 2,235 total users, 24 monthly users, 99.8% runs succeeded, 70 bookmarks - **User rating**: 4.91 out of 5 stars ## Pricing from $2.00 / 1,000 products This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Shopify Scraper **Shopify Scraper** is a professional-grade crawling tool designed to extract high-fidelity product data from any e-commerce store built on the Shopify platform. It can optionally capture **real per-variant inventory counts** for stores that expose them. ### Why Use Shopify Scraper? - **📊 Market Research**: Analyze competitor pricing, inventory, product variants, and descriptions to sharpen your strategy. - **📈 Trend Monitoring**: Track new product launches and stock status changes across multiple brands in real-time. - **🗂️ Data Aggregation**: Build comprehensive product catalogs by aggregating data from various Shopify-based stores. - **🎯 Marketing Insights**: Understand how successful brands structure their metadata, tags, and categories. > 💳 You pay per result, not per minute. See [Pricing](#pricing) for what each event costs and how to cap a run's spend. ### Key Features - **🔍 Search Support**: Find products by keyword using the built-in search functionality. - **🎁 Product Recommendations**: Optionally fetch recommended products for each item (configurable limit). - **🚀 Store-wide Extraction**: Capable of crawling an entire store's catalog, from collections to individual products. - **🎯 Precision Targeting**: Fetch specific collections or singular product URLs with surgical precision. - **💱 Currency Normalization**: Automatically handles currency selection and formats prices (multiplied by 100) to ensure financial accuracy without floating-point errors. - **📦 Comprehensive Data**: Extracts everything: titles, descriptions, images, options (size/color), variants, and real-time stock status. - **🛡️ Resilience**: Includes logic to bypass common gateway restrictions and handle "InStock"/"OutOfStock" parsing robustly. - **📊 Real Inventory (experimental)**: Optionally read true per-variant stock counts (`quantityAvailable`) — not just in/out-of-stock — for stores that expose inventory in their storefront JSON. See `grabRealInventory` below. ### Pricing This actor uses Apify's **pay-per-event** model: you are charged for the results it produces, not for how long it runs. | Event | Charged | Notes | | :--- | :--- | :--- | | `product` | Once per product saved to the dataset | The main unit of value. | | `collection` | Once per collection saved to the dataset | Only `/collections` crawls emit these — they list a store's categories rather than its products, and a single request can return up to 250 of them. | | `recommends` | Once per recommended product | Recommendations are nested inside the parent product record rather than saved as separate results, so they are billed here. Only applies when `maxRecommendationsPerProduct` is above `0`. | | `real-inventory` | Once per product that comes back with a real stock count | Only applies when `grabRealInventory` is enabled **and** the store actually exposes inventory. Stores that don't expose it are never billed for it. | #### Controlling your spend - **`maxRequestsPerCrawl`** caps how many records are saved, so it caps `product`/`collection` charges directly. Default `100`; `0` means unlimited. - **Max total charge** (set per run or per task in the Apify UI) is a hard ceiling. When it is reached the actor stops of its own accord and the run finishes with everything it had already saved — it is not cut off mid-fetch. - **Duplicates are never billed twice.** A product reached through several start URLs or through overlapping collections, or re-visited after a failed request is retried, is saved and charged once per run. ### Important Note On certain Shopify websites, multiple SKUs may be visually merged into a single product page. Please be aware that this scraper treats each SKU as an individual product entry and does not perform any merging of these SKUs. ### How it Works #### 1. Verification Before running the scraper, verify your target is a Shopify store: 1. Navigate to `https:///admin` (e.g., `https://gymshark.com/admin`). 2. If you see a Shopify login page, the site is compatible. #### 2. Configuration The scraper accepts a JSON input defining the target URLs and scraping behavior. | Parameter | Type | Required | Description | | :--- | :--- | :--- | :--- | | `startUrls` | Array | Yes | A list of URLs to scrape (Homepage, Collection, or Product URLs). | | `proxy` | Object | Yes | Proxy configuration. **Residential proxies are highly recommended** to avoid blocks. | | `maxRequestsPerCrawl` | Integer | No | Limit the number of records saved — and therefore billed. Set to `0` for unlimited (default `100`). | | `maxRecommendationsPerProduct` | Integer | No | Number of recommended products to fetch per product. Billed per recommended product — see [Pricing](#pricing). Set to `0` to disable (default `0`). Max `20`. | | `grabRealInventory` | Boolean | No | Fetch real per-variant inventory counts from each product's JSON. Billed per product that returns a real count; adds one request per product on collection crawls. Only stores that expose inventory in their storefront JSON are supported — others report it as unknown and are not billed for it (default `false`). | | `maxInventoryFetches` | Integer | No | Cap on the extra per-product inventory fetches during collection crawls (only when `grabRealInventory` is enabled). `0` = unlimited (default `200`). | | `query` | String | No | Search query to find specific products. | ##### Input Examples | Use Case | Input JSON | | :--- | :--- | | Scrape all products from a store | `{ "startUrls": [{ "url": "https://kith.com/" }], "maxRequestsPerCrawl": 0, "proxy": { "useApifyProxy": true } }` | | Scrape products from a collection | `{ "startUrls": [{ "url": "https://kith.com/collections/kith-tops" }], "maxRequestsPerCrawl": 0, "proxy": { "useApifyProxy": true } }` | | Scrape a single product page | `{ "startUrls": [{ "url": "https://kith.com/products/kith-williams-iii-hoodie-black" }], "proxy": { "useApifyProxy": true } }` | | Search products by keyword | `{ "startUrls": [{ "url": "https://kith.com/" }], "query": "hoodie", "proxy": { "useApifyProxy": true } }` | | Fetch product with recommendations | `{ "startUrls": [{ "url": "https://kith.com/products/kith-williams-iii-hoodie-black" }], "maxRecommendationsPerProduct": 10, "proxy": { "useApifyProxy": true } }` | | Grab real inventory counts | `{ "startUrls": [{ "url": "https://kith.com/collections/kith-tops" }], "grabRealInventory": true, "proxy": { "useApifyProxy": true } }` | | Limit total scraped results | `{ "startUrls": [{ "url": "https://kith.com/" }], "maxRequestsPerCrawl": 100, "proxy": { "useApifyProxy": true } }` | | Scrape with custom currency | `{ "startUrls": [{ "url": "https://kith.com/collections/kith-tops?currency=EUR" }], "proxy": { "useApifyProxy": true } }` | #### 3. Output Data is stored in the default dataset. The scraper outputs either **Product** or **Collection** objects. > When `grabRealInventory` is enabled, variants of supported stores also carry `quantityAvailable` (real stock count), `inventoryTracked` (whether the store tracks stock), and `inventoryPolicy` (`deny`/`continue`). Stores that don't expose inventory simply omit these fields.
View Product Output Example ```json { "source": { "id": "8286037409920", "canonicalUrl": "https://kith.com/products/dm41758001", "retailer": "Dr. Martens", "language": "en", "currency": "USD", "createdUTC": 1765831308000, "updatedUTC": 1766980752000, "publishedUTC": 1765903710000 }, "title": "Dr. Martens Penton Brogue Emboss Leather Loafers - Black", "description": "

Penton Brogue Emboss Leather Loafers silhouette

\n

Leather upper

\n

Goodyear Welted upper

\n

Wax coating 

\n

Round toe

\n

Reinforced toe cap

\n

All-over croc embossed

\n

Structured heel 

\n

Bex sole 

\n

Cemented rubber outsole 

\n

 

\n

Style: dm41758001

\n

Color: Black 

\n

Material: Leather

", "brand": "Dr. Martens", "categories": [ "Loafers" ], "tags": [ "121525", "121525inlinefw", "dm41758001", "dr. martens", "Dr. Martens Penton Brogue Emboss Leather Loafers - Black", "flow_tagged:types", "footwear", "inlinefw121525", "limit-quantity-two", "loafer", "loafers", "mens", "primary", "square-footwear" ], "variants": [ { "id": "45137286234240", "title": "3", "sku": "15089867", "options": [ "3" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286267008", "title": "3.5", "sku": "15089868", "options": [ "3.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286299776", "title": "4", "sku": "15089869", "options": [ "4" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286332544", "title": "4.5", "sku": "15089870", "options": [ "4.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286365312", "title": "5", "sku": "15089871", "options": [ "5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "InStock" } }, { "id": "45137286398080", "title": "5.5", "sku": "15089872", "options": [ "5.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286430848", "title": "6", "sku": "15089873", "options": [ "6" ], "price": { "current": 18000, "previous": 0, "stockStatus": "InStock" } }, { "id": "45137286463616", "title": "6.5", "sku": "15089874", "options": [ "6.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286496384", "title": "7", "sku": "15089875", "options": [ "7" ], "price": { "current": 18000, "previous": 0, "stockStatus": "InStock" } }, { "id": "45137286529152", "title": "7.5", "sku": "15089876", "options": [ "7.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286561920", "title": "8", "sku": "15089877", "options": [ "8" ], "price": { "current": 18000, "previous": 0, "stockStatus": "InStock" } }, { "id": "45137286594688", "title": "8.5", "sku": "15089878", "options": [ "8.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286627456", "title": "9", "sku": "15089879", "options": [ "9" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286660224", "title": "9.5", "sku": "15089880", "options": [ "9.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286692992", "title": "10", "sku": "15089881", "options": [ "10" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286725760", "title": "10.5", "sku": "15089882", "options": [ "10.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286758528", "title": "11", "sku": "15089883", "options": [ "11" ], "price": { "current": 18000, "previous": 0, "stockStatus": "InStock" } }, { "id": "45137286791296", "title": "11.5", "sku": "15089884", "options": [ "11.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286824064", "title": "12", "sku": "15089885", "options": [ "12" ], "price": { "current": 18000, "previous": 0, "stockStatus": "InStock" } }, { "id": "45137286856832", "title": "12.5", "sku": "15089886", "options": [ "12.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286889600", "title": "13", "sku": "15089887", "options": [ "13" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286922368", "title": "13.5", "sku": "15089888", "options": [ "13.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286955136", "title": "14", "sku": "15089889", "options": [ "14" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137286987904", "title": "14.5", "sku": "15089890", "options": [ "14.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137287020672", "title": "15", "sku": "15089891", "options": [ "15" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137287053440", "title": "15.5", "sku": "15089892", "options": [ "15.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137287086208", "title": "16", "sku": "15089893", "options": [ "16" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137287118976", "title": "16.5", "sku": "15089894", "options": [ "16.5" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137287151744", "title": "17", "sku": "15089895", "options": [ "17" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } }, { "id": "45137287184512", "title": "18", "sku": "15089896", "options": [ "18" ], "price": { "current": 18000, "previous": 0, "stockStatus": "OutOfStock" } } ], "medias": [ { "id": "41601988165760", "type": "Image", "url": "https://cdn.shopify.com/s/files/1/0094/2252/files/41758001_3.jpg?v=1765831319", "variantIds": [], "alt": "Dr. Martens Penton Brogue Emboss Leather Loafers - Black" }, { "id": "41601988198528", "type": "Image", "url": "https://cdn.shopify.com/s/files/1/0094/2252/files/41758001.jpg?v=1765831318", "variantIds": [], "alt": "Dr. Martens Penton Brogue Emboss Leather Loafers - Black" }, { "id": "41601988231296", "type": "Image", "url": "https://cdn.shopify.com/s/files/1/0094/2252/files/41758001_1.jpg?v=1765831319", "variantIds": [], "alt": "Dr. Martens Penton Brogue Emboss Leather Loafers - Black" }, { "id": "41601988264064", "type": "Image", "url": "https://cdn.shopify.com/s/files/1/0094/2252/files/41758001_2.jpg?v=1765831319", "variantIds": [], "alt": "Dr. Martens Penton Brogue Emboss Leather Loafers - Black" }, { "id": "41601988296832", "type": "Image", "url": "https://cdn.shopify.com/s/files/1/0094/2252/files/41758001_4.jpg?v=1765831319", "variantIds": [], "alt": "Dr. Martens Penton Brogue Emboss Leather Loafers - Black" } ], "options": [ { "type": "Size", "values": [ { "id": "3", "name": "3" }, { "id": "3.5", "name": "3.5" }, { "id": "4", "name": "4" }, { "id": "4.5", "name": "4.5" }, { "id": "5", "name": "5" }, { "id": "5.5", "name": "5.5" }, { "id": "6", "name": "6" }, { "id": "6.5", "name": "6.5" }, { "id": "7", "name": "7" }, { "id": "7.5", "name": "7.5" }, { "id": "8", "name": "8" }, { "id": "8.5", "name": "8.5" }, { "id": "9", "name": "9" }, { "id": "9.5", "name": "9.5" }, { "id": "10", "name": "10" }, { "id": "10.5", "name": "10.5" }, { "id": "11", "name": "11" }, { "id": "11.5", "name": "11.5" }, { "id": "12", "name": "12" }, { "id": "12.5", "name": "12.5" }, { "id": "13", "name": "13" }, { "id": "13.5", "name": "13.5" }, { "id": "14", "name": "14" }, { "id": "14.5", "name": "14.5" }, { "id": "15", "name": "15" }, { "id": "15.5", "name": "15.5" }, { "id": "16", "name": "16" }, { "id": "16.5", "name": "16.5" }, { "id": "17", "name": "17" }, { "id": "18", "name": "18" } ] } ] } ```
View Collection Output Example ```json { "id": "448305135744", "title": "&Kin Winter 2025", "handle": "kin-winter-2025", "description": "

An exclusive partnership with Giorgio Armani as the official suiting partner for &Kin, reintroducing the Traveller and the Artist silhouettes. Discover unbranded staples made from ultra-soft cashmere, pony hair leather, extra-fine merino wool, and luxe French terry, as well as textured finishes such as Basketweave knit and Tile Herringbone. 

", "productsCount": 30, "updatedUTC": 1766923230000, "publishedUTC": 1761312567000 } ```
*** ### Troubleshooting | Issue | Possible Cause | Solution | | :--- | :--- | :--- | | **0 Results Found** | The site may not be Shopify-based or has strong anti-bot protection. | Verify with the `/admin` trick. Try using residential proxies. | | **Access Denied / 403** | Your IP has been flagged. | Enable `useApifyProxy` and ensure you have sufficient proxy quota. | | **Incorrect Price** | Currency mismatch or raw integer format. | Remember prices are x100 (e.g., 2000 = 20.00). Ensure `?currency=XYZ` param is used if supported. | | **Run stopped early** | The run's **Max total charge** limit was reached. | Expected behaviour — the actor stops cleanly and keeps everything it saved. Raise the limit, or lower `maxRequestsPerCrawl` to fit the budget. | *** ### TODO - \[ ] Review fetch support - \[x] Real stock monitoring - \[ ] 7/30 day sales estimation # Actor input Schema ## `startUrls` (type: `array`): Start URLs of Shopify site to start the actor. Site root url, category page urls, product page urls are all supported. ## `maxRequestsPerCrawl` (type: `integer`): The maximum number of records saved to the dataset. Each record is one billed result, so this is also the run's cost cap. The scraper stops when the limit is reached.

If set to 0, there is no limit — use the run's Max total charge setting to bound spend instead. ## `maxRecommendationsPerProduct` (type: `integer`): The maximum number of recommended products to fetch per product. Recommendations are returned nested inside the product record and are billed per recommended product, so a value of 20 can cost up to 20 extra events per product. Set to 0 to disable. Default is 0, max is 20. ## `grabRealInventory` (type: `boolean`): Fetch each product's real per-variant inventory count from its product JSON. Billed as a separate event for every product that comes back with a real stock count. On collection crawls it also adds one request per product. Only works for stores that expose inventory in their storefront JSON — others report inventory as unknown, and are not billed for it. Off by default. ## `maxInventoryFetches` (type: `integer`): Hard cap on the extra per-product detail fetches used to read inventory on collection crawls. Only applies when 'Grab real inventory' is enabled. Set to 0 for unlimited. ## `proxy` (type: `object`): Select proxies to be used by your crawler. ## `query` (type: `string`): Search query to find products. ## `maxResults` (type: `integer`): Maximum number of results to return. Hidden parameter, overrides maxRequestsPerCrawl if set. ## Actor input object example ```json { "startUrls": [ { "url": "https://kith.com/collections/kith-tops" } ], "maxRequestsPerCrawl": 100, "maxRecommendationsPerProduct": 0, "grabRealInventory": false, "maxInventoryFetches": 200, "proxy": { "useApifyProxy": true }, "maxResults": 0 } ``` # Actor output Schema ## `products` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://kith.com/collections/kith-tops" } ], "proxy": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("autofacts/shopify").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://kith.com/collections/kith-tops" }], "proxy": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("autofacts/shopify").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://kith.com/collections/kith-tops" } ], "proxy": { "useApifyProxy": true } }' | apify call autofacts/shopify --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,autofacts/shopify" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/Us2dCgQWZ0A8L9prQ/builds/JWaiZmbzNdvovnEw6/openapi.json