# X (Twitter) Posts Search (`scraper-engine/x-twitter-posts-search`) Actor

Scrapes posts from Twitter X profiles, hashtags, or searches, capturing tweet text, media, timestamps, metrics, authors, hashtags, and URLs. Ideal for trend research, competitor analysis, sentiment studies, influencer insights, and large-scale Twitter X post extraction.

- **URL**: https://apify.com/scraper-engine/x-twitter-posts-search.md
- **Developed by:** [Scraper Engine](https://apify.com/scraper-engine) (community)
- **Categories:** Agents, Lead generation, Social media
- **Stats:** 17 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### X (Twitter) Posts Search — Text, Likes, Views and Author Data

X (Twitter) Posts Search turns a keyword, hashtag, x.com search URL or @handle into a structured post dataset: full post text including long-form notes, canonical post URL, UTC timestamp, likes, reposts, replies, quotes, views, bookmarks, language, posting client, media URLs and the author's handle, bio and follower counts. One typed JSON row per post, pushed live as it is collected, exportable to JSON, CSV, Excel or XML. Pick Top or Latest, add your targets, press Start.

⚠️ **Read the cookie requirement below before your first run.** X answers keyword, hashtag and search-URL requests from logged-out clients with an empty HTTP 404, so search targets need your own `auth_token` and `ct0` cookies. Profile and @handle targets work fully logged out. Without cookies, a keyword target is skipped and the run writes one unbilled accounting row explaining why.

### What is X (Twitter) Posts Search?

X (Twitter) Posts Search is an Apify Actor that reads X's own GraphQL timeline responses — `SearchTimeline` for queries, `UserTweets` for profiles — and flattens each result into a fixed 23-key post record. It accepts keywords, `#hashtags`, `x.com/search?q=…` URLs, `x.com/hashtag/<tag>` URLs, profile URLs and bare handles in the same input list, and writes every post to the dataset the moment it is normalised.

Whether a login is needed depends entirely on the target type. Profile timelines are fetched with a guest token the Actor obtains itself, so no credentials are required. Keyword search is different: X serves logged-out search requests an HTTP 404 with a zero-byte body, so you supply two cookies from your own X session. That is a platform behaviour, not an Actor limitation.

It is built for social listening and PR teams tracking a topic, research teams building post-level datasets, and developers piping X posts into a dashboard, a warehouse or an LLM pipeline.

### What X post data is publicly available to scrape?

Everything this Actor returns is what a visitor sees in X's own search results and on a public profile timeline: the post text, its engagement counters, the attached media and the author's public profile numbers. What X does not put in a timeline payload — who individually liked or reposted something, anything from a protected account — is not available at any price.

| Data Category | Available without a login | Needs your own logged-in X session |
| ----- | ----- | ----- |
| Public profile timeline posts (`@handle`, profile URL) | ✅ Yes — guest token, no cookies | — |
| Keyword, hashtag and `x.com/search` results | ❌ HTTP 404 with an empty body | ✅ `auth_token` + `ct0` cookies |
| Post text, canonical URL, post and conversation IDs, UTC timestamp | ✅ Yes | — |
| Likes, reposts, replies, quotes, views, bookmarks | ✅ Yes | — |
| Author display name, handle, bio, followers, following, total likes | ✅ Yes | — |
| Media URLs and media type per attachment | ✅ Yes | — |
| Reply threads under a post | ❌ Not fetched by this Actor | Covered by Twitter X Reply Scraper |
| Individual liker and reposter lists | ❌ Not in the timeline payload | Not available |
| Protected accounts, DMs, deleted posts | ❌ Never | Not available |
| Author email or phone number | ❌ Never in a post payload | Not available |

X (Twitter) Posts Search only returns publicly visible posts — what any visitor sees in search results or on a public timeline. Nothing from a protected account, and nothing behind a follow request.

### ⚠️ Keyword search needs your own X session cookies

This is the one prerequisite, and it belongs up front rather than buried in the input table.

X's `SearchTimeline` endpoint answers an authenticated session only. A logged-out POST comes back as HTTP 404 with a zero-byte body — there is no guest tier for search. The Actor detects that exact response and reports it rather than pretending the query matched nothing. Profile lookups, profile timelines and the runtime query-id discovery all still work logged out, and no credential is baked into the Actor.

Supply two cookie values from a browser where you are already signed in to X:

- `xAuthToken` — your `auth_token` cookie, a roughly 40-character hex string
- `xCt0` — your `ct0` cookie, sent onward as the `x-csrf-token` header

Get both from DevTools → **Application** → **Cookies** → `https://x.com`, or from a cookie-export extension. Both are needed: the Actor only switches into authenticated mode when `xAuthToken` **and** `xCt0` are non-empty, so one alone does nothing. Both fields are declared `isSecret` in the input schema, so the values are stored encrypted and masked in the Console and in run history, and the code never writes them to the dataset or the log — the log only states whether cookies were supplied. The Actor also reads `X_AUTH_TOKEN` and `X_CT0` environment variables as a fallback when the input fields are empty.

Neither field is marked `required` in the schema, deliberately. A run without them still succeeds: profile targets are collected normally, and every keyword, hashtag or search-URL target is skipped with an unbilled accounting row carrying `status` set to `login_required`.

One thing to weigh before you paste anything in: `auth_token` **is** a login credential for the account it came from, and automation replaying a live session is exactly the pattern platforms challenge. Use a secondary account you can afford to lose rather than your main one, rotate the value when you are done, and expect that X may require re-authentication on that account at any point.

### What data can I extract with X (Twitter) Posts Search?

Every row carries the same 23 keys in the same order: what the post says, when it was posted, how it performed, who wrote it, what was attached, and which target produced it.

| Field Name | Description |
| ----- | ----- |
| `postText` | Full post text — long-form note text when X sends one, otherwise the standard full text |
| `postUrl` | Canonical post URL, built as `https://x.com/<screenName>/status/<postId>`; `null` if either part is missing |
| `timestamp` | Post creation time as Unix milliseconds; `null` if X's date string cannot be parsed |
| `conversationId` | ID of the thread the post belongs to — the same value as `postId` on a thread's first post |
| `postId` | Numeric post ID as a string |
| `media` | Array of attachment objects: `mediaUrlHttps`, `type`, `id`; `[]` when the post has no media |
| `author` | Author object: `name`, `screenName`, `followersCount`, `favouritesCount`, `friendsCount`, `description` |
| `replyCount` | Replies on the post |
| `quoteCount` | Quote posts |
| `repostCount` | Reposts (X's `retweet_count`) |
| `favouriteCount` | Likes (X's `favorite_count`) |
| `postedAt` | The same instant as `timestamp`, rendered as a UTC ISO string `YYYY-MM-DDTHH:MM:SS.000Z` |
| `lang` | X's own detected language code for the post, e.g. `en`, `es`, `und` |
| `viewCount` | View count as an integer; `null` when X omits it for that post |
| `bookmarkCount` | Bookmark count |
| `source` | Posting client with HTML tags stripped, e.g. `Twitter Web App`; `null` when empty |
| `isRetweet` | `true` when the collected entry was a repost |
| `retweetedBy` | Handle of the account that reposted it, on repost rows only; `null` otherwise |
| `rowType` | `post` on real post rows, `run_status` on unbilled accounting rows — the one reliable filter |
| `target` | The normalised target that produced the row: the search query, or the handle without `@` |
| `targetType` | `search` or `profile` |
| `status` | `ok` on post rows; the failure or empty reason on `run_status` rows |
| `errorReason` | `null` on post rows; the same string as `status` on `run_status` rows |

#### Post, timestamp and media fields

`postText` reads X's `note_tweet` result first, so long-form posts arrive complete instead of truncated at the classic character limit. `postId`, `conversationId` and `postUrl` are the join keys: `conversationId` groups a thread, and `postUrl` is safe to hand to a human or to another Actor.

Time is given twice on purpose. `timestamp` is Unix milliseconds for sorting and diffing; `postedAt` is the same moment as a UTC ISO string, so the value does not shift with your container's time zone.

`media` is an array of three-key objects. `mediaUrlHttps` is X's own `media_url_https` value — for a video entry that is the thumbnail image, not the video file — `type` is X's label for the attachment, and `id` is the media ID. Video variant URLs, playback duration, hashtag and mention arrays, verified badges and author profile images are **not** in this output; the Actor emits what X already ships in the timeline payload and makes no extra call per post.

Reposts are handled explicitly. The row's post data and `author` describe the **original** post, while `isRetweet` is `true` and `retweetedBy` names the account whose timeline or search hit surfaced it. Two naming details worth knowing before you map fields: likes are `favouriteCount` with the British spelling, and the author's own total likes given are `author.favouritesCount` — different fields, similar names. And while the count key is `repostCount`, the repost flags kept X's original wording as `isRetweet` / `retweetedBy`. Every output key is camelCase, including the ones read from snake\_case X fields.

#### Author and engagement fields

`author` is a nested object with six keys, always present. `name` and `screenName` identify the account, `description` is the profile bio, and `followersCount`, `friendsCount` (accounts followed) and `favouritesCount` (likes given) are the profile-level numbers X includes alongside the post. Any of them can be `null` when X omits the user block.

`replyCount`, `quoteCount`, `repostCount`, `favouriteCount`, `viewCount` and `bookmarkCount` are the six engagement integers. `viewCount` depends on X returning the view block for that post and comes back `null` when it does not — plan for that rather than assuming zero.

#### 🤖 Add-on: Need additional X (Twitter) data?

Search is one entry point into X, and three sibling Scraper-Engine Actors cover the rest. Pick by what you have and what you are willing to supply:

| Actor | What it returns | X session cookies |
| ----- | ----- | ----- |
| **X (Twitter) Posts Search** (this one) | Posts from keywords, hashtags, search URLs and profiles, with Top/Latest and a time window | ✅ Needed for keyword, hashtag and search-URL targets |
| **Twitter X Posts Scraper** | Posts from profile targets — `startUrls`, `sortOrder`, `maxTweets` | ❌ No auth parameters in its schema at all |
| **Twitter X Reply Scraper** | Replies under posts — `startUrls`, `maxSearchResults`, `maxReplies` | ✅ Takes `authToken` + `ct0` |
| **Twitter Trends Scraper** | Trending topics with board rank, by country and hour | ❌ Its listing states no login, cookie or API key is required |

If all you need is a named account's recent posts, **Twitter X Posts Scraper** does that with no credential at all. Choose this Actor when the job starts with a query rather than a handle — that is what `searchType` and the time window exist for, and it is why search here costs a cookie.

### How does X (Twitter) Posts Search differ from the official X API?

X publishes an official API (v2) with documented search endpoints, and if your use case fits a paid tier, use it — it is the supported route with terms you can rely on. This Actor exists for the case where the paid tier, the approval process or the tier-dependent access to recent search does not fit the job.

| Feature | Official X API v2 | X (Twitter) Posts Search |
| ----- | ----- | ----- |
| Credential you provide | X developer account with a paid tier and app keys | Nothing for profile targets; your own `auth_token` + `ct0` for search |
| Access to recent-search endpoints | Tier-dependent — verify against X's current published API documentation | Search runs through the same web endpoint your browser uses |
| Setup before the first result | Developer account, app, project, plan selection | Paste a target, press Start |
| Input flexibility | Query syntax per the API reference, per endpoint | Keywords, `#hashtags`, `x.com/search` URLs, profile URLs and handles mixed in one list |
| Output shape | X's documented API response objects | 23 flat, camelCase keys, identical on every row |
| Failed or empty targets | Handled by your own code | Unbilled `run_status` row naming the reason |
| Rate limiting | Plan-based quotas per X's documentation | Whatever the quota is on the account whose cookies you supply, read live from X's response headers |

No price, quota or rate-limit figure from X's official plans is reproduced here — check X's current published API documentation, because those terms change. Use the official API for contractual guarantees and production workloads on an approved tier. Use this Actor for ad-hoc research, monitoring and one-off datasets where standing up a developer project is more work than the question is worth.

### How to use X (Twitter) Posts Search

The Actor runs on Apify. Start it from the Apify Console or call it through the Apify API.

1. Open X (Twitter) Posts Search on Apify and click **Try for free**
2. Add your targets to **Targets to scrape** (`startUrls`) — one row per keyword, hashtag, search URL, profile URL or handle. This is one of the two schema-required inputs
3. Pick **Search mode** (`searchType`) — `top` or `latest`. This is the other required input, and it defaults to `top`
4. Paste your `auth_token` and `ct0` cookies if any of your targets is a keyword, hashtag or search URL — required in practice, see the cookie section above
5. Set **Max posts per target** (`maxTweets`) — per target, not per run
6. Optionally set **last N hours** (`timeWindowHours`) or **last N days** (`timeWindow`), and turn on **Apify Proxy** if X blocks the direct connection
7. Click **Start**, watch posts stream into the Output tab live, then export as JSON, CSV, Excel or XML

Both `startUrls` and `searchType` are declared required in the schema, so a run that omits either is rejected by Apify's input validation before the Actor starts — no run, no rows, no charge. If `startUrls` is present but resolves to nothing usable (an empty list, or only entries the Actor skips), the run logs `No targets provided` and finishes with zero dataset rows and no accounting row. A caller that reaches the code with a missing or unrecognised `searchType` falls back to `top`.

The log prints one line per saved post with the author, likes, reposts, replies, views and a text snippet, plus the number of X search calls left in the current window, so you can see whether a query is worth a bigger run before committing to one.

#### How to scale to bulk post extraction

`startUrls` is a list, and every entry is processed in turn into the same dataset. Mix types freely — `@NASA`, `climate change`, `#Artemis`, `https://x.com/search?q=james%20webb&f=live` and `https://x.com/elonmusk` in one run is a normal input. Entries may be plain strings or objects; an object is read from its `url`, `query` or `value` key, which makes it easy to pipe another Actor's output straight in.

`maxTweets` applies **per target**, so ten targets at `maxTweets: 100` is a target of 1,000 rows, not 100. Deduplication is run-wide: the Actor tracks every `postId` it has already saved, so a post matched by two different queries is written once, and the log reports how many were dropped as already saved. A target that fails or matches nothing does not stop the run — the loop moves to the next one.

### What can you do with X post data?

- 📡 A **social listening analyst** tracking a product launch runs the same keyword daily with `searchType: "latest"` and `timeWindowHours: 24`, then ranks new rows by `favouriteCount` and `viewCount` to see which mentions actually travelled.
- 📰 A **newsroom researcher** verifying a claim pulls a hashtag plus the accounts involved in one run and uses `postedAt`, `conversationId` and `postUrl` to reconstruct the order in which a story spread.
- 📊 A **brand manager** benchmarking share of voice runs several competitor keywords in one pass and compares median `repostCount` and `replyCount` per `target` to see which message is being carried rather than just seen.
- 🌍 A **market researcher** studying a region filters rows on `lang` and reads `author.followersCount` alongside `favouriteCount` to separate genuine local reach from one large account's amplification.
- 🤖 An **AI engineer** building a monitoring agent indexes `postText`, `author.screenName` and `postedAt` into a vector store and keeps the six engagement counters plus `lang` as metadata filters, so the agent answers "what is being said about this topic today" against live posts rather than a stale export.

Every one of these is callable from an agent framework over the Apify API, since the Actor is a standard HTTP-triggered run.

### How does X (Twitter) Posts Search handle rate limits and blocking?

No CAPTCHA solving is involved and no browser is launched. The Actor speaks HTTP directly to X's GraphQL endpoints with a desktop Chrome header set — `POST` for `SearchTimeline`, `GET` for the profile operations — which is what those endpoints expect.

Two things are discovered at runtime instead of being hardcoded, because both rot. The public web bearer token and the GraphQL query IDs are harvested from x.com's live JavaScript bundle at the start of every run; pinned fallbacks exist for the two profile operations, deliberately not for search. If a search call comes back 404 with a zero-byte body, the Actor re-harvests the query ID once and retries with the new one before giving up. If the harvest yields no search query ID at all, affected targets end with `status` set to `qid_unavailable` rather than a silent zero.

Quota handling is read from X, not guessed. Every response's `x-rate-limit-limit`, `x-rate-limit-remaining` and `x-rate-limit-reset` headers are recorded for two separate buckets — the POST search bucket and the GET bucket the profile operations use — and the remaining count is logged as the run progresses. When X reports two or fewer calls left, the Actor stops that target early instead of burning your last calls. X sets these limits per account; the input schema records a measured figure of roughly 187 search calls per 15-minute window on the search bucket, but treat that as an observation from one account's headers rather than a promise, and note that failed calls consume quota too and concurrent runs share it. Because X returns 20 posts per search call regardless of the count requested, a 100-post target costs at least five calls.

Failures are typed and never fail the run. Network errors and HTTP 5xx are retried up to three times with exponential backoff; HTTP 429 stops the target immediately as `rate_limited` rather than retrying into a deeper block; 401 and 403 become `login_required`; an unexpected redirect, a non-JSON body or a missing response container each get their own reason. Proxy use is optional — the Actor holds one Apify Proxy session URL for the whole run when you enable it, and if proxy setup fails it logs a warning and continues on a direct connection rather than downgrading silently.

### ⬇️ Input

Eight parameters, two of them required. Everything below matches the Actor's input schema exactly.

| Parameter | Required | Type | Description | Example Value |
| ----- | ----- | ----- | ----- | ----- |
| `startUrls` | Yes | array | Targets, one per row: keyword, `#hashtag`, `x.com/search?q=…` URL, `x.com/hashtag/<tag>` URL, profile URL, `@handle`, bare handle, or an explicit `search: <query>` string. Strings, or objects with a `url` / `query` / `value` key. Prefilled with `["@NASA", "climate change"]`. | `["@NASA", "climate change", "#Artemis"]` |
| `maxTweets` | No | integer | Maximum posts collected **per target**. Minimum 1, maximum 10000, default `10`. Clamped into range at runtime. | `100` |
| `searchType` | Yes | string | `top` (X's popularity and relevance ranking) or `latest` (newest first). Default `"top"`. | `"latest"` |
| `timeWindow` | No | integer | Keep only posts newer than N days, UTC. Minimum 1, no default — leave empty for no date cut-off. Enforced on both sort modes. | `7` |
| `timeWindowHours` | No | integer | Keep only posts newer than N hours, UTC. Minimum 1, no default. Takes precedence over `timeWindow` when both are set. | `24` |
| `proxyConfiguration` | No | object | Apify Proxy settings. Prefilled with `{"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}`. Leave it off for a direct connection. | `{"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}` |
| `xAuthToken` | No | string | Your X `auth_token` cookie. Declared `isSecret`, so it is stored encrypted and masked. Not schema-required, but required in practice for every keyword, hashtag and search-URL target. Falls back to the `X_AUTH_TOKEN` environment variable. | `"a1b2c3d4e5f60718293a4b5c6d7e8f9012345678"` |
| `xCt0` | No | string | Your X `ct0` cookie, sent as the `x-csrf-token` header. Declared `isSecret`. Must be supplied together with `xAuthToken` — one alone does nothing. Falls back to the `X_CT0` environment variable. | `"7f3c9ab1e5d24680bc13fe57a90d2c48"` |

Five honest notes on how these behave at runtime:

- **`searchType` means something different on profile targets.** X's profile timeline API has no server-side sort. For a `@handle` or profile URL the Actor collects up to **twice** your `maxTweets` from the account's recent posts and then keeps either the most-liked (`top`) or the newest (`latest`) of what it collected. That ranking is this Actor's, over a bounded window — not X's ranking of the whole account. On search targets `searchType` maps straight to X's own `Top` / `Latest` product.
- **The time window is a post-fetch filter, not part of the query.** No `since:` or `until:` operator is added. Posts are fetched and then dropped if they are older than the cut-off, which means filtered-out posts still consume pages and X quota, and the scanned pool is capped at roughly twice `maxTweets`. A narrow window on a busy query can therefore return fewer rows than `maxTweets` even when older matches exist. The log reports how many rows were dropped for being outside the window.
- **`timeWindowHours` wins.** If both time fields are set, hours are used and `timeWindow` is ignored entirely. Both cut-offs are computed from the run's own clock in UTC.
- **Some entries are skipped with no row at all.** Single-post links containing `/status/<id>`, X's own system routes (`/explore/…`, `/i/…`, `/home`, `/messages`, `/notifications` and similar) and empty rows are logged and dropped — no post row, no accounting row, no charge. A non-x.com URL is not skipped: it is treated as a **search for that URL string**, which is rarely what you want.
- **What counts as a search versus a profile.** A multi-word entry, a `#hashtag` or anything containing a space is a search; a single word matching X's handle pattern, an `@handle` or an `x.com/<name>` URL is that account's timeline. Prefix an entry with `search:` to force search interpretation, for example `search: python from:NASA`.

#### Example input

```json
{
  "startUrls": [
    "@NASA",
    "james webb telescope",
    "#Artemis",
    "https://x.com/search?q=mars%20sample%20return&f=live",
    "https://x.com/elonmusk"
  ],
  "maxTweets": 100,
  "searchType": "latest",
  "timeWindowHours": 24,
  "timeWindow": 7,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  },
  "xAuthToken": "a1b2c3d4e5f60718293a4b5c6d7e8f9012345678",
  "xCt0": "7f3c9ab1e5d24680bc13fe57a90d2c4891bd6e07f2a3c5d81904e6b7a2c3d4f56"
}
```

With both time fields present as above, the run uses the last 24 hours and ignores the 7-day value.

### ⬆️ Output

Every row has the same 23 keys in the same order, run after run — no key is ever dropped, and unavailable values arrive as `null` (or `[]` for `media`) rather than going missing. Rows are pushed live as each post is normalised, so a run you stop early still leaves you everything collected up to that point. Export as JSON, CSV, Excel or XML, or read the dataset through the Apify API.

Billing is per row: each post row is pushed with the `row_result` charged event, one event per post. **Accounting rows are pushed with no charged event at all** — they are free. Because every key is always present, `status` is not the marker here; `rowType` is:

```python
posts = [row for row in items if row["rowType"] == "post"]
diagnostics = [row for row in items if row["rowType"] == "run_status"]
```

An accounting row is written for any target that ends with zero saved posts, and its `status` names the reason. `empty` means X answered normally and nothing matched. `login_required` means either the target was a search and no cookies were supplied, or X answered 401/403 or a zero-byte 404. The rest come straight from the failure that stopped the target: `rate_limited`, `not_found`, `unavailable` (a handle that could not be resolved — missing, suspended or renamed), `redirected`, `server_error`, `http_error`, `network_error`, `unparseable`, `api_error`, `container_missing`, `qid_unavailable`, or the catch-all `error`. `errorReason` repeats the same string; the longer detail — the HTTP status, the body size, the resolved URL — goes to the run log, not the row.

Two limits of that mechanism, stated plainly. A target that saved at least one post gets **no** accounting row, so a run cut short mid-target by a 429 or by the stall guard shows the reason only in the log. And targets skipped at parse time produce no row of any kind.

#### Example output

A post row from a search target:

```json
{
  "postText": "Webb just resolved the inner disk of Beta Pictoris in the mid-infrared. Three separate dust belts, and the innermost one is not where the models put it. Full data release and images in the thread below.",
  "postUrl": "https://x.com/NASAWebb/status/1948273615902847109",
  "timestamp": 1784989362000,
  "conversationId": "1948273615902847109",
  "postId": "1948273615902847109",
  "media": [
    {
      "mediaUrlHttps": "https://pbs.twimg.com/media/Gx7QpL2XwAAf3Rk.jpg",
      "type": "photo",
      "id": "1948273544012931072"
    }
  ],
  "author": {
    "name": "NASA Webb Telescope",
    "screenName": "NASAWebb",
    "followersCount": 2184903,
    "favouritesCount": 4127,
    "friendsCount": 118,
    "description": "The largest, most powerful space telescope ever built."
  },
  "replyCount": 412,
  "quoteCount": 296,
  "repostCount": 3841,
  "favouriteCount": 21507,
  "postedAt": "2026-07-25T08:22:42.000Z",
  "lang": "en",
  "viewCount": 1284617,
  "bookmarkCount": 1902,
  "source": "Twitter Web App",
  "isRetweet": false,
  "retweetedBy": null,
  "rowType": "post",
  "target": "james webb telescope",
  "targetType": "search",
  "status": "ok",
  "errorReason": null
}
```

An unbilled accounting row, written here for a keyword target on a run with no cookies. It carries the same 23 keys with empty defaults:

```json
{
  "postText": null,
  "postUrl": null,
  "timestamp": null,
  "conversationId": null,
  "postId": null,
  "media": [],
  "author": {
    "name": null,
    "screenName": null,
    "followersCount": null,
    "favouritesCount": null,
    "friendsCount": null,
    "description": null
  },
  "replyCount": null,
  "quoteCount": null,
  "repostCount": null,
  "favouriteCount": null,
  "postedAt": null,
  "lang": null,
  "viewCount": null,
  "bookmarkCount": null,
  "source": null,
  "isRetweet": false,
  "retweetedBy": null,
  "rowType": "run_status",
  "target": "james webb telescope",
  "targetType": "search",
  "status": "login_required",
  "errorReason": "login_required"
}
```

### How does it work?

Each run starts by loading x.com's own search page and reading its JavaScript bundles to harvest the public web bearer token and the current GraphQL query IDs for `SearchTimeline`, `UserByScreenName` and `UserTweets`. Those IDs change with X's build, which is why they are discovered per run rather than pinned.

Then each target is routed. A search target goes to `SearchTimeline` as a POST with `product` set to `Top` or `Latest`, and the Actor walks the bottom cursor page by page, deduplicating within the query and stopping when your limit is met, when the cursor stops advancing, after three consecutive pages arrive with no new posts, or when X's remaining-call header drops to two. A profile target resolves the handle to a numeric ID with `UserByScreenName`, then pages `UserTweets`, collecting standalone posts and self-thread items.

Every entry is normalised into the same 23 keys: reposts are unwrapped to the original post and flagged, long-form note text is preferred over truncated text, X's date string becomes both Unix milliseconds and a UTC ISO string, and the posting client is stripped of its HTML anchor. Only publicly visible posts are collected. Because the Actor reads X's structured GraphQL payload rather than rendered HTML, an X front-end redesign does not change your field names.

### Integrations

X (Twitter) Posts Search is an Apify Actor, so it works with anything that can call the Apify API or consume a dataset.

#### Calling X (Twitter) Posts Search from Python

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("<YOUR_USERNAME>/x-twitter-posts-search").call(run_input={
    "startUrls": ["james webb telescope", "@NASAWebb", "#Artemis"],
    "searchType": "latest",
    "maxTweets": 100,
    "timeWindowHours": 24,
    "xAuthToken": "<YOUR_X_AUTH_TOKEN_COOKIE>",
    "xCt0": "<YOUR_X_CT0_COOKIE>",
})

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["rowType"] != "post":
        print("skipped:", row["target"], row["status"])
        continue
    print(row["postedAt"], row["author"]["screenName"], row["favouriteCount"], row["postUrl"])
```

Works in Go, Ruby, Node.js, cURL — any language that can make an HTTP request. Branch on `rowType` to keep accounting rows out of your post pipeline.

#### Scheduling and webhooks

Because session cookies expire, keyword monitoring works best on a schedule you can see failing. Use an Apify schedule to run the same `startUrls` with `searchType: "latest"` and a `timeWindowHours` matching the interval, and attach an Actor webhook on run success to POST the dataset to your endpoint. A dataset whose only rows carry `status` set to `login_required` is your signal to refresh the two cookie values — worth alerting on explicitly.

#### No-code tools (n8n, Make, LangChain)

In n8n, use the Apify node — or an HTTP Request node pointed at the Apify run endpoint with your token — and pass the same JSON input shown above; an IF node on `rowType` splits real posts from accounting rows. In Make, the Apify module supports run-and-wait, so a daily keyword sweep can feed a Google Sheets or Airtable step directly. In LangChain, wrap the run endpoint as a tool and hand the typed rows to the agent as-is; no parsing step is needed.

### Is it legal to scrape X posts?

Scraping publicly visible posts is broadly treated as permissible where the content is accessible without circumventing an access restriction, and this Actor returns only public posts — nothing from a protected account, nothing from DMs. But this output **is personal data**, and that changes your obligations rather than removing them.

The fields that carry personal data are specific and worth naming: `author.name`, `author.screenName` and `author.description` identify and describe a real person, `author.followersCount`, `author.friendsCount` and `author.favouritesCount` are attributes of that person's account, `postText` is content they authored and can itself contain other people's names and details, `retweetedBy` names a second identifiable account, `postId`, `conversationId` and `postUrl` link straight back to the individual, and `media[].mediaUrlHttps` can point at images of identifiable people. Under GDPR and the UK GDPR you need a lawful basis before you store or process any of it. Legitimate interests is the usual route for research, monitoring and journalism, and it is not a formality: document the balancing test — your purpose, why the processing is necessary, the effect on the individual, and the safeguards you applied — before the first bulk run, not after. CCPA/CPRA treats the same fields as personal information about California residents.

Data minimisation is the practical control, and this output makes it easy. Drop `author.description` first — it is a free-text profile bio and the least likely field to be necessary — then `author.name` and the three author counters, then `postText` itself if you only need engagement patterns. What is left, the six counters plus `timestamp`, `lang`, `target` and `postId`, supports most trend analysis with far less exposure. Set a retention period and delete on it, keep a record of what you collected and why so you can answer a transparency request, and have a workable process for access, objection and deletion requests — including deleting rows for a post that has since been removed from X.

Separately, keyword search here uses your own X session cookies, which is governed by X's Terms of Service — read them and decide for yourself. Consult legal counsel if your use case involves bulk storage of personal data, profiling of identified individuals, or moving these rows outside your jurisdiction.

### ❓ Frequently asked questions

#### What X post fields does X (Twitter) Posts Search return?

The five most used are `postText`, `postUrl`, `favouriteCount`, `viewCount` and `author`. Every row carries the same 23 keys — including `postedAt`, `lang`, `bookmarkCount`, `source`, `isRetweet` and the `rowType` / `target` / `status` accounting trio. See the data fields table above for all of them, plus the nested `media` and `author` shapes.

#### Does X (Twitter) Posts Search require an X account or login?

It depends on the target. Profile URLs and `@handles` need nothing — the Actor obtains a guest token itself. Keyword, hashtag and search-URL targets need your own X session, supplied as `xAuthToken` (`auth_token`) and `xCt0` (`ct0`), because X answers logged-out search requests with an empty HTTP 404. Both cookie fields are declared `isSecret`, so they are stored encrypted and masked in the Console and run history, and the Actor never writes them to the dataset or the log. Neither is schema-required: without them, profile targets still run and search targets get an unbilled `login_required` row.

#### How many posts can I extract per target?

`maxTweets` accepts 1 to 10000 and applies per target, so ten targets at 100 is a 1,000-row run. The real ceilings come from X, not the schema: X returns 20 posts per search call, your account's search quota is finite per 15-minute window, and a query ends when X's cursor stops producing new posts — the Actor gives up after three consecutive pages with nothing new. For volume, add more targets rather than pushing `maxTweets` very high on one.

#### What happens if a search returns zero results?

The run still succeeds, and the empty target is visible in your dataset instead of being an absence you have to notice. The Actor writes one unbilled accounting row with `rowType` set to `run_status` and `status` set to `empty`, meaning X answered normally and the query matched nothing — usually a too-narrow keyword or too tight a time window. If the target failed rather than matched nothing, the same row carries the real reason instead: `login_required`, `rate_limited`, `unavailable`, `network_error`, `container_missing`, `qid_unavailable` and so on. In code, filter with `row["rowType"] == "post"` and branch on `row["status"]` for the rest. If every target came back empty, the final log line says so explicitly and points at those rows.

#### Can I search multiple keywords and profiles at once?

Yes. `startUrls` is a list and each entry is processed in turn into the same dataset, with keywords, hashtags, search URLs, profile URLs and handles freely mixed. Deduplication is run-wide by `postId`, so a post that two queries both match is saved once, and the log reports how many duplicates were dropped. `target` and `targetType` on every row tell you which entry produced it.

#### Does the time window filter on X's side or after collection?

After collection. No `since:` or `until:` operator is added to the query — posts are fetched and then discarded if they are older than the cut-off, which means the discarded ones still cost pages and X quota, and the pool scanned per target is bounded at roughly twice `maxTweets`. Expect fewer rows than `maxTweets` on a narrow window over a busy query. `timeWindowHours` takes precedence when both time fields are set, and both cut-offs are enforced on `top` and `latest` alike.

#### Does X (Twitter) Posts Search work with Claude, ChatGPT and other AI agent tools?

Yes. It is callable as a standard HTTP endpoint through the Apify API, so LangChain, CrewAI, n8n or a hand-written tool definition can invoke it and receive typed JSON with no parsing step. Give the agent the same run input shown above and have it skip rows where `rowType` is not `post`.

#### How does X (Twitter) Posts Search compare to other X post search scrapers?

Checked on the Apify Store on 25 July 2026: `scraper_one/x-posts-search` is the most used of the alternatives and returns the same 11 core keys this Actor returns — `postText`, `postUrl`, `timestamp`, `conversationId`, `postId`, `media`, `author`, `replyCount`, `quoteCount`, `repostCount`, `favouriteCount`. Its README documents a single `query` string plus `resultsCount`, `timeWindow`, `timeWindowHours` and `searchType`, states that its time windows work only with the Latest search type, and notes that free users can make only a few requests per day; `lang`, `viewCount`, `bookmarkCount`, `source`, repost flags, profile targets and cookie authentication are not documented on that listing. `scrapebase/x-twitter-posts-search` documents a `startUrls` / `sortOrder` / `maxTweets` / `maxComments` input and a snake\_case output shape (`id`, `url`, `user_posted`, `likes`, `views` as a string, `tagged_users`, `quoted_post`), and no cookie requirement is documented there either. What this Actor documents that neither does: mixed keyword, hashtag, search-URL and profile targets in one list, a time window enforced on both sort modes, the `auth_token` / `ct0` requirement stated as a prerequisite rather than omitted, unbilled accounting rows with a named `status` vocabulary, and run-wide deduplication across targets.

#### Does X (Twitter) Posts Search return data in a format LLMs can use directly?

Yes. Every row is typed, normalized JSON with the same 23 field names on every run. No HTML parsing, no selectors. Pass a row straight into an LLM context window, index it into a vector store, or hand it to an agent tool. `postText` already contains the full long-form note text where X sends one, so nothing needs re-assembling.

#### What happens when X changes its API or anti-bot system?

The scraper is maintained, and it is built so that X's most frequent change breaks nothing: the bearer token and GraphQL query IDs are harvested from x.com's live bundle on every run rather than hardcoded, and a stale search query ID triggers a single automatic re-harvest and retry. Because it reads X's GraphQL payloads rather than rendered markup, a front-end redesign does not affect your field names or types.

#### Can I use X (Twitter) Posts Search without managing proxies or browser infrastructure?

Yes. There is no browser to run and no proxy account to create — enable Apify Proxy in `proxyConfiguration` and the Actor holds one session for the whole run; leave it off and it connects directly. It does not solve CAPTCHAs, because the endpoints it uses do not present one. What it handles for you is typed block detection, retries with backoff on network and 5xx errors, an immediate stop on HTTP 429, and reading X's rate-limit headers so it stops before your account's quota is gone. The one thing you manage yourself is your X cookies, which expire.

#### Which X post fields work best for AI training data and RAG indexing?

For RAG indexing: `postText` is the high-information field and chunks cleanly on its own, with `author.screenName`, `target` and `lang` as metadata filters and `postedAt` for recency ranking. For training data and analytics: `favouriteCount`, `repostCount`, `replyCount`, `quoteCount`, `bookmarkCount` and `timestamp` are the most structurally consistent fields across records, with `viewCount` the one to null-check because X does not always send it. All values are returned as typed primitives, arrays or plain nested objects, so no normalization pass is needed before indexing.

### 🔗 Related scrapers

| Scraper Name | What it extracts |
| ----- | ----- |
| Twitter X Posts Scraper | Posts from X profile targets, with no session cookies required |
| Twitter X Reply Scraper | Replies under X posts, using your `authToken` and `ct0` |
| Twitter Trends Scraper | X trending topics by country and hour, with board rank |
| Threads User Posts Scraper | Posts from Threads profiles with keyword and date filters |
| Reddit Posts Scraper | Reddit posts by keyword with score filtering |
| Facebook Hashtag Search Scraper | Facebook posts published under a hashtag |

### 💬 Your feedback

Found a bug, or need a field that is in X's timeline payload but not in the output? Open an issue on the Actor's Issues tab and it will be looked at. Reports that include the exact `startUrls` entry, the `searchType` you used and the run ID are the fastest to reproduce and fix.

# Actor input Schema

## `startUrls` (type: `array`):

📋 **Add one row per target.** Mix and match freely!

✅ **What works:**
• 🔗 Profile URL — `https://x.com/elonmusk`  *(works logged-out)*
• 👤 Handle — `@NASA` or `BarackObama`  *(works logged-out)*
• 🔤 Keyword(s) — `climate change` · `world cup final`  *(needs cookies)*
• #️⃣ Hashtag — `#NASA` or `https://x.com/hashtag/NASA`  *(needs cookies)*
• 🔗 X search URL — `https://x.com/search?q=james%20webb&f=live`  *(needs cookies)*
• 🧠 Explicit search — `search: python from:NASA`

🚫 **Skipped:** single-post links (`…/status/123…`) and X's own system pages (`/explore/…`, `/i/…`).

💡 **Tip:** a multi-word entry is a search · a single word or profile URL is that account's timeline.

## `maxTweets` (type: `integer`):

🎯 **Cap collection per target** (URL / user / keyword). Range **1–10 000**.

📄 X returns **20 posts per search call**, so a limit of 100 costs 5 calls against your X account's search quota.

⚖️ Higher = more data + longer runs. Start small (50–100) to validate, then scale up.

## `searchType` (type: `string`):

🏆 **Top** → X's popularity/relevance ranking (great for trends and analysis).
🆕 **Latest** → newest first (perfect with a **time window** for monitoring).

🧑 **On profile targets** X's timeline API has **no server-side sort**, so the actor collects up to 2× your limit from the account's recent posts and then keeps the **most-liked** (Top) or the **newest** (Latest). That ranking is done by this actor over the posts it collected — not by X.

## `timeWindow` (type: `integer`):

🗓️ **Keep only posts newer than N days (UTC).**

✅ Enforced on **both** sort modes (Top and Latest).
🔕 Leave empty to collect up to *Max posts* regardless of date.

## `timeWindowHours` (type: `integer`):

⏰ **Keep only posts newer than N hours (UTC).**

✅ Enforced on **both** sort modes (Top and Latest).
⚡ Use this for breaking-news / real-time monitoring.
👑 Takes precedence over days if both are set.

## `proxyConfiguration` (type: `object`):

🔒 **Off** → direct connection (fast, but X may block datacenter IPs).
✅ **On** → Apify Proxy (pick your groups in the UI).

⭐ RESIDENTIAL is the most forgiving choice on X; BUYPROXIES94952 datacenter also returned full pages in testing.

## `xAuthToken` (type: `string`):

🔐 Your X `auth_token` cookie value (a ~40-character hex string).

⚠️ This is a **login credential** for the X account it came from — prefer a secondary account.

📉 Rate limit: roughly **187 search calls per 15 minutes** on the measured bucket (the actor reads the real number from X's response headers and logs it). Failed calls burn quota too, and concurrent runs can share it.

## `xCt0` (type: `string`):

🔐 Your X `ct0` cookie value (a long hex string). It is sent as the `x-csrf-token` header.

🤝 **Both** `auth_token` and `ct0` must be filled in — one alone does nothing.

## Actor input object example

```json
{
  "startUrls": [
    "@NASA",
    "climate change"
  ],
  "maxTweets": 10,
  "searchType": "top",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `results` (type: `string`):

All scraped items in the Actor's default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "@NASA",
        "climate change"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("scraper-engine/x-twitter-posts-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [
        "@NASA",
        "climate change",
    ],
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("scraper-engine/x-twitter-posts-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "@NASA",
    "climate change"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call scraper-engine/x-twitter-posts-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scraper-engine/x-twitter-posts-search"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/pSvtw4dKjNAUpBVR0/builds/VMgOWYdcYSHzujGkl/openapi.json