# jq Helper – transform JSON with jq (`rl1987/jq-helper`) Actor

Run jq programs over inline JSON or a linked Apify dataset. Built for chaining into data-processing and enrichment workflows.

- **URL**: https://apify.com/rl1987/jq-helper.md
- **Developed by:** [R.L.](https://apify.com/rl1987) (community)
- **Categories:** Developer tools
- **Stats:** 23 total users, 11 monthly users, 47.5% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.01 / successful conversion

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## jq Helper – Transform & Filter JSON with jq (no code)

**Run any [jq](https://jqlang.github.io/jq/) program over your JSON and get clean, structured results back.** Reshape, filter, flatten, enrich, and aggregate JSON — from a pasted snippet or straight from another Apify Actor's dataset. The fast, no-code way to add a JSON transformation step to any scraping or data pipeline.

> Map · filter · select · flatten · rename keys · merge fields · group & aggregate — all with battle-tested jq 1.8 syntax.

***

### 🧩 What is jq Helper?

jq Helper is a tiny, blazing-fast Apify Actor that puts the full power of **jq** — the most popular command-line JSON processor — into a hosted, chainable step. Instead of writing a throwaway script every time a scraper returns messy JSON, drop in a jq filter and let this Actor do the transformation in the cloud.

It's built to slot directly into the **Apify ecosystem**: point it at the output dataset of any scraper and use it as a JSON **post-processing, cleanup, and enrichment** stage in your automation.

### ✨ Key features

- **Full jq 1.8 support** — the complete language: `select`, `map`, `group_by`, `reduce`, `sort_by`, `unique`, string ops, math, conditionals, and more.
- **Two input sources** — paste **inline JSON / JSONL**, or read items from a **linked Apify dataset** by ID (perfect for chaining Actors).
- **Two apply modes** — transform **each item** independently (mapping/filtering/enrichment) or process the **whole array at once** (aggregations, grouping, dedupe).
- **Pipeline-ready** — writes results to its own dataset so the next Actor or integration (Zapier, Make, Google Sheets, webhooks…) can pick them up.
- **Robust by default** — errors on a single bad record are skipped and logged (or fail-fast if you prefer), and results are pushed in efficient batches.
- **Zero setup** — no servers, no `jq` install, no dependencies to manage.

### 🚀 Use cases

- **Clean up scraped data** — drop nulls, rename fields, normalize values, flatten nested objects.
- **Filter datasets** — keep only the records that match your criteria (`select(.price < 100)`).
- **Reshape for export** — turn a verbose API response into a tidy, flat table for CSV/Excel/Sheets.
- **Enrich records** — derive new fields (domains from emails, full location strings, computed flags).
- **Aggregate & summarize** — group by a field, count, average, sum across an entire dataset.
- **Glue Actors together** — sit between a scraper and your destination as a no-code transformation step.

### 🔧 How to use

1. Choose your data source: paste JSON into **`jsonInput`**, or set **`datasetId`** to an existing dataset (e.g. an upstream scraper's run).
2. Write your transformation in the **`filter`** field using jq syntax.
3. Pick an **`applyMode`** (`perItem` or `wholeInput`).
4. Run it — transformed records land in the Actor's default dataset, ready to export or pass downstream.

#### ⚠️ Per-item vs. whole-input (read this first)

This is the one thing to get right:

| Mode | What the filter receives | Reach a field with |
|------|--------------------------|--------------------|
| **`perItem`** (default) | each array element, one at a time, as a single object | `.websiteUrl` ✅ |
| **`wholeInput`** | the entire array as one value | `.[].websiteUrl` ✅ |

In **per-item** mode the array is iterated for you, so **don't** prefix with `.[]` — `.[].websiteUrl` will fail with *"Cannot index string with string"*. Use a leading `.[]` only in **whole-input** mode.

### 📥 Input

| Field | Type | Description |
|-------|------|-------------|
| `filter` | string (required) | The jq program. jq 1.8 syntax. May emit 0, 1, or many values per input — each becomes one output item. |
| `applyMode` | `perItem` | `wholeInput` | `perItem` (default): run on each element. `wholeInput`: feed the whole array as a single value. |
| `datasetId` | string | Apify dataset to read items from (provide either this **or** `jsonInput`, not both). **Map this field to chain Actors in an integration.** Paid: $0.01 per successful run. Your API token must have access to the dataset (owned by you, or explicitly shared) — otherwise the run fails with an insufficient permissions error. |
| `jsonInput` | string | Raw JSON (object/array/scalar) or JSONL (one value per line). Free — for prototyping. |
| `wrapKey` | string | Non-object outputs are wrapped as `{ <wrapKey>: value }` (dataset items must be objects). Default `value`. |
| `failOnError` | boolean | `true` aborts on any jq runtime error; default `false` skips + logs the bad record. |

### 📤 Output

Each value emitted by your filter becomes one item in the Actor's default dataset. Object outputs are stored as-is (their keys become columns); scalars and arrays are wrapped under `wrapKey`. Export the dataset as **JSON, CSV, Excel, HTML, or RSS**, or hand it to the next step in your workflow.

### 💡 Examples

**Reshape & build a location string (per-item):**

```jq
{fullName, currentEmployer, location: ([.city, .state, .country] | map(select(.)) | join(", "))}
```

**Keep only verified records (per-item):**

```jq
select(.clearVerified == true)
```

**Pull one field as a clean column (per-item):**

```jq
{websiteUrl}
```

**Group and summarize the whole dataset (whole-input):**

```jq
group_by(.state) | map({state: .[0].state, count: length}) | .[]
```

**Deduplicate by a key (whole-input):**

```jq
unique_by(.profileUrl) | .[]
```

### 🔗 Chaining after another Actor (integrations)

Run **jq Helper** automatically whenever an upstream scraper finishes, and pass it that run's dataset.

**In the Console (no code):**

1. Open the **upstream** Actor's run config → **Integrations** tab → **Connect Actor or Task**.
2. Choose **jq Helper** to run on success (event `ACTOR.RUN.SUCCEEDED`).
3. In jq Helper's input, set the **`datasetId`** field to the upstream run's dataset using the variable:
   ```
   {{resource.defaultDatasetId}}
   ```
   Leave `jsonInput` empty (the two sources are mutually exclusive), then set your `filter` and `applyMode`.

The transformed items land in jq Helper's own dataset — ready for the next Actor, a webhook, or an export to Sheets/Make/Zapier.

> Other variables from the same run you can drop into string fields: `{{resource.id}}` (run ID), `{{resource.actId}}`, `{{resource.defaultKeyValueStoreId}}`, `{{resource.status}}`.

**In code (parent orchestrator):**

```python
run = await Actor.call(actor_id='you/scraper', run_input={...})
await Actor.call(
    actor_id='rl1987/jq-helper',
    run_input={
        'filter': '.websiteUrl',
        'applyMode': 'perItem',
        'datasetId': run.default_dataset_id,
    },
)
```

> Chaining always uses the `datasetId` path, so each chained run is a paid **$0.01** conversion. Inline `jsonInput` stays free for prototyping.

### ❓ FAQ

**Do I need to install jq?** No. The jq engine is bundled — just write filters.

**Which jq version?** jq 1.8, the full language.

**My filter returns a string/number — why is it `{"value": ...}`?** Dataset items must be JSON objects, so scalars and arrays are wrapped. Set `wrapKey` to rename the key, or emit an object (e.g. `{websiteUrl}`) for named columns.

**How do I aggregate across all records?** Use `applyMode: wholeInput` and reach elements with `.[]`.

**What if one record is malformed?** By default it's skipped and logged; set `failOnError: true` to stop the run instead.

### n8n integration

Use the [`n8n-nodes-jq-helper`](https://www.npmjs.com/package/n8n-nodes-jq-helper) community node ([source](https://github.com/rl1987/n8n-nodes-jq-helper)) to run this Actor directly from an n8n workflow.

### 🛠️ Local development

```bash
pip install -r requirements.txt
apify run   # reads storage/key_value_stores/default/INPUT.json
```

***

Built with the [Apify SDK for Python](https://docs.apify.com/sdk/python/) and the [`jq` bindings](https://github.com/mwilliamson/jq.py) (bundles libjq — no system `jq` required).

> **Disclaimer:** This is an independent, unofficial project. It is **not affiliated with, endorsed by, or otherwise associated with the [jq project](https://jqlang.github.io/jq/) or any of its developers.** "jq" is used here only to describe the JSON-processing language this Actor runs; all rights to jq belong to its respective authors.

### Data pipeline toolkit

Part of the **Data pipeline toolkit** — small, chainable Actors for cleaning, transforming, and generating data inside a larger pipeline:

- [DuckDB Helper – SQL over CSV, JSON, Parquet, Excel, SQLite](https://apify.com/rl1987/duckdb-wrapper) — Run a DuckDB SQL query over remote/local files, push results to a dataset.
- [Regex Helper](https://apify.com/rl1987/regex-helper) — Apply named regular expressions to strings, extract structured matches.
- [URL Wrangler](https://apify.com/rl1987/url-wrangler) — Join, decompose, and rewrite URLs and query params in batch.
- [ZIP Code Helper](https://apify.com/rl1987/zip-code-helper) — Resolves US ZIP codes into city, state, county, and more.
- [Postal Address Normaliser](https://apify.com/rl1987/postal-address-normaliser) — Parses and normalises postal addresses using libpostal.
- [Phone Number Wrangler](https://apify.com/rl1987/phone-number-wrangler) — Validate, format, and parse phone numbers using libphonenumber.
- [UUID Generator](https://apify.com/rl1987/uuid-generator) — Generate bulk UUIDs (v1, v3, v4, v5, v7) on demand.
- [Secure Password & Passphrase Generator](https://apify.com/rl1987/password-generator) — Generate secure passwords and diceware passphrases per NIST guidance.
- [Thumbnail Maker](https://apify.com/rl1987/thumbnail-maker) — Generates thumbnails from image URLs using ImageMagick.
- [Katana Web Crawler (ProjectDiscovery)](https://apify.com/rl1987/pd-katana) — Crawl websites with Katana, stream results as JSONL.
- [ProjectDiscovery Notify](https://apify.com/rl1987/pd-notify) — Stream records to Slack, Discord, Telegram, Email, and more.

### Did you find this useful?

⭐ Rate this actor on Apify! Your feedback helps other users find it and helps us keep improving it.

# Actor input Schema

## `filter` (type: `string`):

The jq program to run (standard jq 1.8 syntax). Compiled once and applied per the apply mode below. A filter may emit zero, one, or many values per input — each becomes one output item; non-object values are wrapped under the scalar wrap key. IMPORTANT — do NOT prefix with `.[]` in per-item mode: each item is already a single object, so `.websiteUrl` is correct and `.[].websiteUrl` errors with 'Cannot index string with string'. Use a leading `.[]` only in whole-input mode, where the whole array is fed in at once.

## `applyMode` (type: `string`):

How the filter maps over the input. Per-item (default): the array is iterated for you and the filter runs on each element as a single object — write `.field`, NOT `.[].field`. Best for mapping/filtering/enriching rows. Whole-input: the entire array is fed to the filter as one value, so use `.[]` to reach elements. Best for group\_by, reduce, sort, unique, and other cross-record aggregations.

## `jsonInput` (type: `string`):

Raw JSON (object, array, or scalar) or JSONL (one JSON value per line) to transform. Provide either this or a Dataset ID, not both. In per-item mode, a top-level array is iterated element by element; a single object/scalar is treated as one item. Runs on inline input are FREE — prototype your jq snippets here at no cost.

## `datasetId` (type: `string`):

ID of an Apify dataset to read items from (e.g. the default dataset of an upstream actor run). Provide either this or inline JSON, not both. This is the field to map when wiring this actor into an integration/workflow. Processing a dataset is a paid conversion ($0.01 per successful run, regardless of size). Make sure the API token running this actor has access to the dataset (own dataset, or one explicitly shared with you) - otherwise the run fails with an insufficient permissions error.

## `wrapKey` (type: `string`):

Dataset items must be JSON objects. When the filter emits a non-object value (string, number, array, boolean, null), it is wrapped as { <wrapKey>: value }.

## `failOnError` (type: `boolean`):

If enabled, a jq runtime error on any input aborts the run. If disabled (default), the erroring input is skipped, logged as a warning, and counted in the run summary.

## Actor input object example

```json
{
  "filter": "{name, isAdult: (.age >= 18)}",
  "applyMode": "perItem",
  "jsonInput": "[{\"name\": \"alice\", \"age\": 30}, {\"name\": \"bob\", \"age\": 16}]",
  "wrapKey": "value",
  "failOnError": false
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "filter": "{name, isAdult: (.age >= 18)}",
    "jsonInput": "[{\"name\": \"alice\", \"age\": 30}, {\"name\": \"bob\", \"age\": 16}]"
};

// Run the Actor and wait for it to finish
const run = await client.actor("rl1987/jq-helper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "filter": "{name, isAdult: (.age >= 18)}",
    "jsonInput": "[{\"name\": \"alice\", \"age\": 30}, {\"name\": \"bob\", \"age\": 16}]",
}

# Run the Actor and wait for it to finish
run = client.actor("rl1987/jq-helper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "filter": "{name, isAdult: (.age >= 18)}",
  "jsonInput": "[{\\"name\\": \\"alice\\", \\"age\\": 30}, {\\"name\\": \\"bob\\", \\"age\\": 16}]"
}' |
apify call rl1987/jq-helper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rl1987/jq-helper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/KK10Q0gjD9WIY7yGF/builds/rHwJcHWzrJIy3fHpj/openapi.json