# Structured Extract (`gastronomic_desk/structured-extract`) Actor

Only pay when it works. $0.05 per verified extraction — nothing charged on failure or retries. Extract structured JSON from any webpage using your own schema. AJV-validated output guaranteed. Compatible with Groq, OpenAI, Together AI, and Ollama.

- **URL**: https://apify.com/gastronomic\_desk/structured-extract.md
- **Developed by:** [Herbert Yeboah](https://apify.com/gastronomic_desk) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 3 total users, 0 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

$50.00 / 1,000 structured extractions

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Structured Data Extractor

**Extract structured JSON from any webpage using a Groq-compatible LLM.**

Provide a URL + a JSON Schema → get back validated, structured data. Works with Groq (free), OpenAI, Together AI, Fireworks AI, and Ollama.

[![Apify Actor](https://img.shields.io/badge/Apify-Actor-brightgreen)](https://apify.com/store)
[![PPE Pricing](https://img.shields.io/badge/Price-%240.05%2Fextraction-blue)](https://docs.apify.com/platform/actors/monetization/pay-per-event)

***

### What It Does

1. **Scrapes** the page at your URL using a real browser-grade crawler (CheerioCrawler)
2. **Strips** all HTML, navigation, scripts, and boilerplate → clean plain text
3. **Prompts** a Groq-compatible LLM to extract data matching your schema
4. **Validates** the response with AJV (JSON Schema validator)
5. **Retries** up to 3 times if the LLM returns invalid JSON, injecting the error back into the prompt
6. **Returns** validated structured data in the Apify dataset

**Charge:** `$0.05` per successful extraction. Nothing charged on failure.

***

### Input Schema

| Field | Type | Required | Default | Description |
|---|---|---|---|---|
| `url` | string | ✅ | — | Page to scrape |
| `output_schema` | object | ✅ | — | JSON Schema defining the data to extract |
| `groq_api_key` | string | ✅ | — | API key (Groq, OpenAI, Together AI, etc.) |
| `model` | string | ❌ | `llama-3.3-70b-versatile` | Model name |
| `base_url` | string | ❌ | Groq endpoint | For OpenAI-compatible providers |

***

### Usage Examples

#### Example 1: Groq (default, free tier)

Get a free API key at [console.groq.com](https://console.groq.com/).

```json
{
    "url": "https://example.com/product/widget-pro",
    "groq_api_key": "gsk_YOUR_GROQ_KEY_HERE",
    "output_schema": {
        "type": "object",
        "required": ["name", "price"],
        "properties": {
            "name":        { "type": "string" },
            "price":       { "type": "number" },
            "description": { "type": "string" },
            "in_stock":    { "type": "boolean" }
        }
    }
}
```

**Output:**

```json
{
    "url": "https://example.com/product/widget-pro",
    "extracted": {
        "name": "Widget Pro",
        "price": 29.99,
        "description": "The best widget on the market.",
        "in_stock": true
    },
    "model": "llama-3.3-70b-versatile",
    "attempts": 1
}
```

***

#### Example 2: OpenAI-compatible endpoint (Together AI, Fireworks AI)

Use any OpenAI-compatible provider by setting `base_url`:

```json
{
    "url": "https://jobs.lever.co/anthropic/engineer",
    "groq_api_key": "YOUR_TOGETHER_AI_KEY",
    "base_url": "https://api.together.xyz/v1",
    "model": "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    "output_schema": {
        "type": "object",
        "required": ["title", "company", "location", "salary_range"],
        "properties": {
            "title":        { "type": "string" },
            "company":      { "type": "string" },
            "location":     { "type": "string" },
            "salary_range": { "type": "string" },
            "remote":       { "type": "boolean" },
            "requirements": {
                "type": "array",
                "items": { "type": "string" }
            }
        }
    }
}
```

Other compatible endpoints:

- **Fireworks AI:** `https://api.fireworks.ai/inference/v1`
- **OpenAI:** `https://api.openai.com/v1`

***

#### Example 3: Ollama (local, completely free)

Run models locally at zero cost with [Ollama](https://ollama.com/):

```bash
## Start Ollama with a model
ollama serve
ollama pull llama3.3
```

```json
{
    "url": "https://news.ycombinator.com/item?id=12345",
    "groq_api_key": "ollama",
    "base_url": "http://localhost:11434/v1",
    "model": "llama3.3",
    "output_schema": {
        "type": "object",
        "required": ["title", "score", "comments_count"],
        "properties": {
            "title":          { "type": "string" },
            "score":          { "type": "integer" },
            "comments_count": { "type": "integer" },
            "author":         { "type": "string" },
            "url":            { "type": "string" }
        }
    }
}
```

> **Note:** When running the Actor on Apify cloud, Ollama requires a remote endpoint. For local testing, use `apify run` with `localhost`.

***

### Common Use Cases

| Use Case | Schema Fields |
|---|---|
| **Product extraction** | name, price, description, in\_stock, SKU |
| **Job postings** | title, company, location, salary, requirements |
| **News articles** | headline, author, published\_date, summary, tags |
| **Real estate listings** | address, price, bedrooms, bathrooms, sqft |
| **Restaurant menus** | restaurant\_name, items (name, price, description) |
| **Resume parsing** | name, email, skills, experience, education |
| **Event listings** | name, date, venue, ticket\_price, organizer |

***

### How Retry Logic Works

The actor uses the same retry-with-feedback pattern as [`constrained.py`](https://github.com/devilsfave/dagpipe) from the DagPipe core library:

1. **Attempt 1:** Send text + schema → LLM responds → AJV validates
2. **On failure:** Inject the exact AJV error message into the next prompt → retry
3. **Attempt 2:** LLM receives error and corrects → validate again
4. **After 3 failures:** Throw with a descriptive error message

This approach reliably extracts valid structured data even from smaller/cheaper models.

***

### Pricing

- **`$0.05` per successful extraction** (Pay-Per-Event)
- **Free if extraction fails** — you're never charged for failed attempts
- Groq's free tier provides 30 requests/minute at zero cost to you

***

### Technical Details

- **Scraper:** CheerioCrawler (zero-JS, fast, reliable)
- **Validation:** AJV v8 + ajv-formats (JSON Schema Draft-07/2019/2020 compatible)
- **LLM client:** OpenAI SDK (works with any OpenAI-compatible endpoint)
- **Retry strategy:** Error-feedback prompting (same pattern as DagPipe constrained.py)
- **Language:** TypeScript, Node.js 20+
- **Tests:** 9 vitest tests (100% passing)

***

### Built With

[DagPipe](https://github.com/devilsfave/dagpipe) — Zero-cost, crash-proof LLM pipeline orchestrator.

```bash
pip install dagpipe-core
```

# Actor input Schema

## `start_urls` (type: `array`):

List of URLs to scrape and extract data from. Each URL is processed independently.

## `output_schema` (type: `object`):

A JSON Schema object describing the structure of the data to extract. The LLM will return data validated against this schema.

## `groq_api_key` (type: `string`):

Your Groq API key (or API key for any OpenAI-compatible provider). Get a free Groq key at console.groq.com.

## `model` (type: `string`):

Model to use for extraction. Defaults to llama-3.3-70b-versatile (Groq free tier).

## `base_url` (type: `string`):

Base URL of the OpenAI-compatible API. Defaults to Groq. Use https://api.openai.com/v1 for OpenAI, https://api.together.xyz/v1 for Together AI, or http://localhost:11434/v1 for Ollama.

## Actor input object example

```json
{
  "start_urls": [
    {
      "url": "https://example.com/product"
    }
  ],
  "output_schema": {
    "type": "object",
    "properties": {
      "title": {
        "type": "string"
      },
      "price": {
        "type": "number"
      },
      "description": {
        "type": "string"
      }
    },
    "required": [
      "title",
      "price"
    ]
  },
  "model": "llama-3.3-70b-versatile",
  "base_url": "https://api.groq.com/openai/v1"
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("gastronomic_desk/structured-extract").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("gastronomic_desk/structured-extract").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call gastronomic_desk/structured-extract --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,gastronomic_desk/structured-extract"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/gUamfVozD1cWUEHAh/builds/vE8y091TNQ1SqATOl/openapi.json