Lidl Product Scraper

Lidl Product Scraper creates a structured dataset of Lidl products collected from Lidl country shops such as lidl.de. Each dataset item represents one product and can include identifiers, title and brand, price and currency, category, rating, availability, and optional product-page details (description, EANs, variants, media, and delivery information). Query the source with search terms (searchQueries) or Lidl category, section, search, or product URLs (startUrls), scoped by countryCode. Control the result limit with maxItems, and retrieve records through the Apify Dataset API or export them as JSON, CSV, Excel, XML, or another supported format.

Dataset at a glance

Property	Value
Source	Lidl country shops (lidl.de, lidl.at, lidl.ch, lidl.fr, lidl.es, lidl.it, lidl.nl, lidl.be, lidl.pt, lidl.pl, lidl.cz, lidl.sk, lidl.hu, lidl.ro, lidl.ie, lidl.co.uk)
Record unit	One Lidl product
Input methods	Search terms (searchQueries) and/or Lidl category, section, search, or product URLs (startUrls)
Main identifiers	productId, product URL (url)
Delivery	Apify Dataset and API
Export formats	JSON, CSV, Excel, XML, HTML (Apify dataset exports)
Update model	Fresh records per Actor run
Pricing	$1 / 1,000 products; +$2 / 1,000 product details; Actor start $0.00005

Coverage and available records

The Actor collects public Lidl catalog products from one or both entry points. When both are set, the seeds are combined.

• Search-based: searchQueries (for example heissluftfritteuse) resolve against the shop selected by countryCode (default DE).
• URL-based: startUrls accepts Lidl category, section, search, or product pages. The host on each URL selects the country shop and overrides countryCode for that seed.

Record types and limits:

• Listing-level records are always collected: identifiers, title, brand, price, currency, image, category, rating, availability, country, and source metadata.
• Product-page fields are conditional: description, EANs, variants, media, and delivery information are returned only when fetchProductDetails is enabled and detailsFetched is true.
• Result cap: maxItems limits the number of products collected (0 means unlimited, the default). Pagination continues until the source returns no further products or the cap is reached.

Supported countryCode values: DE, AT, CH, FR, ES, IT, NL, BE, PT, PL, CZ, SK, HU, RO, IE, GB.

Known exclusions: content Lidl only shows behind login is not collected; each run captures catalog state at run time (no historical snapshots); pages outside the supported Lidl country shops are not collected. Assortment and prices differ by country shop.

Data dictionary

Field names below match dataset record JSON properties exactly. Fields marked conditional appear only when fetchProductDetails is enabled. Values come from Lidl for the selected shop; the Actor does not calculate prices or ratings. scrapedAt and detailsFetched are set by the Actor.

Product listing fields

Field	Type	Nullable	Description	Example
productId	string	No	Lidl product identifier; recommended deduplication key	276353
erpNumber	string	Yes	Article / ERP number	276353
title	string	Yes	Product title	SilverCrest® Heißluftfritteuse
fullTitle	string	Yes	Full listing title	SilverCrest® Heißluftfritteuse XXL 6,5 l
brand	string	Yes	Brand name	SilverCrest
price	number	Yes	Numeric price in currency	49.99
currency	string	Yes	ISO currency code	EUR
currencySymbol	string	Yes	Currency symbol as shown by Lidl	€
url	string	Yes	Product page URL; secondary deduplication key	https://www.lidl.de/p/silvercrest-heissluftfritteuse-xxl-6-5-l/p276353
image	string	Yes	Primary product image URL	https://www.lidl.de/media/product/...
category	string	Yes	Category label	Küche & Haushalt
ratingAverage	number	Yes	Average rating	4.5
ratingCount	number	Yes	Number of ratings	1287
availability	string	Yes	Availability text as shown by Lidl	Online verfügbar
onlineAvailable	boolean	Yes	Whether the product is marked available online	true
countryCode	string	No	Lidl shop country code	DE
sourceQuery	string	No	Search term or start URL that produced the record	heissluftfritteuse
scrapedAt	string	No	UTC timestamp when the record was written (ISO 8601)	2026-07-26T12:00:00.000000+00:00
detailsFetched	boolean	No	Whether product-page fields were attached	false

Product detail fields (conditional — fetchProductDetails)

These fields appear when fetchProductDetails is true and detailsFetched is true. Nested shapes follow what Lidl renders; they are omitted when the source does not supply them.

Field	Type	Nullable	Description	Example
description	string	Yes	Product description from the product page	Product-page description text
eans	string[]	Yes	European Article Numbers when listed	EAN strings from the product page
variants	object[]	Yes	Variant records: {erpNumber, title, price, currency}	See source when variants exist
deliveryInfo	object	Yes	Delivery information when Lidl renders it	Object as returned by the source

Additional product-page media is included when present. The Actor does not invent missing EANs, variants, media, or delivery values.

Example dataset record

Listing-level record for the Germany shop search heissluftfritteuse (fetchProductDetails off). Product-page fields are absent because details were not requested.

{
  "productId": "276353",
  "erpNumber": "276353",
  "title": "SilverCrest® Heißluftfritteuse",
  "fullTitle": "SilverCrest® Heißluftfritteuse XXL 6,5 l",
  "brand": "SilverCrest",
  "price": 49.99,
  "currency": "EUR",
  "currencySymbol": "€",
  "url": "https://www.lidl.de/p/silvercrest-heissluftfritteuse-xxl-6-5-l/p276353",
  "category": "Küche & Haushalt",
  "ratingAverage": 4.5,
  "ratingCount": 1287,
  "availability": "Online verfügbar",
  "onlineAvailable": true,
  "countryCode": "DE",
  "sourceQuery": "heissluftfritteuse",
  "scrapedAt": "2026-07-26T12:00:00.000000+00:00",
  "detailsFetched": false
}

The record above was produced with this input:

{
  "searchQueries": ["heissluftfritteuse"],
  "fetchProductDetails": false,
  "countryCode": "DE",
  "maxItems": 25
}

Query and input reference

Input	Type	Required	Default	Accepted values	Description
searchQueries	array (string)	No*	— (prefill heissluftfritteuse)	Product search terms	Search entry point. Uses the countryCode shop
startUrls	array (requestListSources)	No*	—	Lidl category, section, search, or product URLs	URL entry point. Country is taken from the host
fetchProductDetails	boolean	No	false	true / false	Attach product-page fields (charged as product details)
countryCode	string	No	DE	DE, AT, CH, FR, ES, IT, NL, BE, PT, PL, CZ, SK, HU, RO, IE, GB	Default shop for searchQueries. Ignored when a start URL sets the host
maxItems	integer	No	0	0 or any positive integer	Maximum products to collect; 0 = unlimited (console prefill 25)
proxyConfiguration	object	No	Apify proxy, RESIDENTIAL group	Apify proxy groups or custom proxies	Residential proxies are recommended

* Provide searchQueries and/or startUrls — at least one source is required.

Minimal request:

{ "searchQueries": ["heissluftfritteuse"], "countryCode": "DE" }

Advanced request (combined seeds with product-page fields):

{
  "searchQueries": ["heissluftfritteuse"],
  "startUrls": [{ "url": "https://www.lidl.de/h/garten-balkon/h10067558" }],
  "fetchProductDetails": true,
  "countryCode": "DE",
  "maxItems": 25,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}

Retrieve the data through the API

Records are delivered through the Apify Actor and Dataset APIs — not an official Lidl API.

1. Start the Actor with a JSON input (console or API).
2. Wait for the run to finish, or use a synchronous endpoint if you want the response inline.
3. Retrieve items from the run's default dataset.
4. Paginate or export the dataset.

Python example:

from apify_client import ApifyClient

client = ApifyClient("YOUR-APIFY-TOKEN")

run_input = {
    "searchQueries": ["heissluftfritteuse"],
    "countryCode": "DE",
    "maxItems": 10,
    "fetchProductDetails": True,
}

run = client.actor("datascrapers/lidl-scraper").call(run_input=run_input)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["productId"], item["title"], item["price"])

Apify generates ready-to-run Python, JavaScript, and cURL examples on the Actor's API tab. Do not put a real API token in shared code or URLs.

Data quality and record handling

• Conditional fields: product-page fields are present only when fetchProductDetails is enabled and detailsFetched is true. Listing-only runs return a leaner record.
• Source changes: Lidl page structure and values can change; unreadable fields are omitted or returned as null rather than fabricated.
• Deduplication: within a run, products are de-duplicated by shop and product identifier. Across runs, records are appended. Use productId as the stable key, or url as a secondary key, and filter repeated runs against previously stored IDs.
• Failures: one failed product or seed does not stop the run. The run stops if a spending limit is reached.
• Normalization: price and ratingAverage are numeric; scrapedAt is an ISO 8601 UTC timestamp written by the Actor. Currency, availability text, and ratings come from Lidl.

Export and pipeline examples

Destination	Recommended method	Typical use
PostgreSQL / Supabase	Dataset API poll or webhook consumer	Store catalog prices and article numbers
Google Sheets	Apify Google Sheets integration	Share shortlists and price checks
PIM / catalog systems	Dataset API or completion webhook	Product-feed refresh
S3 / cloud storage	Scheduled export via Apify scheduler + integration	Archival of market snapshots

Pricing and cost examples

The Actor uses pay-per-event pricing. Each run also charges a one-time Actor start. Residential proxies are recommended. Proxy and compute are billed by your Apify plan.

Event	Trigger	Rate
Actor start	Once per run	$0.00005
dataset-item (Product)	Each product written to the dataset	$0.001 / product ($1 / 1,000)
product-details	Product-page fields collected (fetchProductDetails)	$0.002 / product ($2 / 1,000)

Example costs (Actor start negligible; plan-tier discounts not applied):

Records	Configuration	Estimated base cost
1,000	Listing only	$1.00
10,000	Listing only	$10.00
1,000	Listing + product details	$3.00
10,000	Listing + product details	$30.00

Estimates depend on the verified pricing model and the options selected for the run.

Limitations and responsible data use

• The Actor collects publicly accessible data from supported Lidl country shops only.
• Field availability depends on what Lidl renders at run time; some values can be null or missing, and site changes can alter fields.
• The Actor does not provide historical snapshots unless you store them yourself. Assortment and prices differ by country shop.
• Large runs should use residential proxies; without them, coverage may degrade.
• You are responsible for compliance with Lidl's terms of service, applicable privacy law, and any contractual obligations before using the data.

Dataset questions

What does one dataset item represent?

One Lidl product. Variant data, when requested, is nested on that product (variants), so a product with several size or option variants still produces a single dataset item.

Which field should I use as a unique identifier?

productId is the recommended deduplication key. url is a reasonable secondary key. The same product can exist in more than one country shop; combine productId with countryCode when you merge multi-country runs.

Are fields nullable or conditional?

Yes. Product-page fields (description, EANs, variants, media, delivery information) exist only when fetchProductDetails is enabled and detailsFetched is true. Within listing records, fields that Lidl does not render for a product are omitted or null.

How do I select a Lidl country shop?

Set countryCode for searchQueries (default DE). For startUrls, the hostname selects the shop (lidl.de, lidl.fr, lidl.co.uk, and the other supported hosts) and overrides countryCode for that seed.

Can I retrieve the records as CSV or JSON?

Yes. The dataset can be exported as JSON, CSV, Excel, XML, or HTML from the Apify Console, and queried through the Dataset API.

Does the Actor return historical data?

No. Each run captures the state of the catalog at run time. To track price or availability changes, schedule repeated runs and store the outputs yourself.

What counts as a billable result?

Three pay-per-event charges apply: Actor start once per run ($0.00005), a dataset-item charge for every product record ($0.001, or $1 per 1,000), and a product-details charge for each product whose product-page fields are collected ($0.002, or $2 per 1,000). A listing-only run incurs Actor start plus dataset-item.

Related datasets from Data Scrapers

• IKEA Product Scraper — European home and household products that can be compared with Lidl assortment and prices.
• Target.com Product Scraper — US retail catalog records for the same price-monitoring pipelines.
• Amazon Search Scraper — Marketplace search products for cross-retailer comparison with Lidl items.
• Costco Product Scraper — Warehouse-club grocery and household products in overlapping categories.
• Instacart Scraper — Grocery catalog products for broader retail and availability analysis.

Data Scrapers support

Need an additional field, record type, or export workflow? Contact Data Scrapers at stardustspotlight@gmail.com. Include a sample source URL, required fields, expected record volume, and preferred delivery format.