# Twitter Search Scraper (`epctex/twitter-search-scraper`) Actor
Scrape any keyword or hashtag from Twitter. Extract tweets, replies, favorites, retweets, and conversation threads without limit. Gather user-related information such as verification, location, profile image, friends, followers, following, and much more! Get everything with no limits on Twitter!!
- **URL**: https://apify.com/epctex/twitter-search-scraper.md
- **Developed by:** [epctex](https://apify.com/epctex) (community)
- **Categories:** Social media
- **Stats:** 1,063 total users, 1 monthly users, 0.0% runs succeeded, 19 bookmarks
- **User rating**: 5.00 out of 5 stars
## Pricing
$20.00/month + usage
To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period. You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have.
Learn more: https://docs.apify.com/actors/running/actors-in-store.md#rental-actors
## What's an Apify Actor?
Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Actor - Twitter Search Scraper
### 🐦 Twitter Search Scraper
The Twitter Search Scraper provides powerful tools to extract data from Twitter's search functionality, allowing you to find tweets based on keywords, phrases, and advanced search parameters.
### 🚀 Features and Functionality
Extremely detailed tweet information for data mining.
Multi-language support on Twitter. Globally working Twitter search actor. No restrictions!
Geo-target the tweets with advanced search capabilities. Integrate into your system with ease.
Lightning speed. Extreme efficiency. Suitable for integration into any platform.
**Query Wizard**: Easy-to-use Input for non-technical users. Create [Twitter Advanced Search](https://twitter.com/search-advanced) effortlessly.
A list of filters that you can easily use. Target your tweet search with precision!
#
- **Custom Start URLs**: Begin scraping from search query URLs.
- Examples:
- https://x.com/search?q=real%20madrid\&src=typed\_query
- https://x.com/search?q=from%3AKMbappe\&src=typed\_query
- **Search Queries**: Provide a list of keywords or phrases to find tweets that match specific topics, such as "barca" or "atletico madrid"
- **Authentication Support**: Use Twitter authentication tokens for improved reliability and access to more data.
- **Search Modes**:
- Choose between:
- **Latest**: Most recent tweets
- **Top**: Most popular tweets
- **Date Filters**:
- Define specific date ranges for scraping:
- Tweets newer than a specific date (YYYY-MM-DD)
- Tweets older than a specific date (YYYY-MM-DD)
- **Content Filters**:
- Customize the type of tweets to scrape:
- Only tweets with media (images, videos, or GIFs)
- Only tweets from verified users
- Exclude replies, retweets, or links
- **Tweet Stats Filters**: Filter tweets based on minimum likes, replies, or retweets.
- **Account Filters**:
- Scrape tweets:
- From a specific account.
- To a specific account.
- Mentioning a specific account.
- **Language and Location**: Limit results to specific languages or find tweets near a defined location.
- **Proxy Configuration**: Use advanced proxy settings to ensure anonymous and efficient scraping.
- **Limitations**: Set limits on the number of tweets or pages to scrape for optimized performance.
These features allow for a highly customizable and efficient search process, suitable for various data collection needs.
### 😎 Who needs this?
Twitter Search Scraper is designed to provide enriched search data with high speed in a cost-effective manner. It serves a wide range of users:
- **Cryptocurrency Analysts and Investors** - can use Twitter Search Scraper to track sentiment, key influencers' opinions, and market-moving announcements related to cryptocurrencies. Catch the panic buy or panic sell right away.
- **Market Research Analysts** can use Twitter Search Scraper to gather public opinion and trends on different products, services, or topics. By analyzing tweets, they can identify consumer behavior patterns, preferences, and market trends, helping companies make informed decisions.
- **Academic Researchers** can use Twitter Search Scraper to analyze social behaviors, communication patterns, and public opinion on various topics. This can aid in understanding societal trends, public reactions to events, and the spread of information or misinformation.
- **Human Resources and Recruitment Specialists** can use Twitter Search Scraper to monitor industry trends, understand job market sentiments, and identify potential candidates for recruitment, enhancing their talent acquisition strategies.
### 🔧 Bugs, fixes, updates, and changelog
This scraper is under active development. If you have any feature requests you can create an issue from [here](https://github.com/epctex/twitter-search-scraper/issues).
### ⚙️ Input and Definitions
| Field | Type | Description
| ----- | ---- | ----------- |
startUrls | array | Twitter (X) search URLs.
searchTerms | array | Search terms or queries you want to search from Twitter (X). Check [this](https://github.com/igorbrigadir/twitter-advanced-search) page for further reference.
sort | String | Sort order for search tweets. It can be either "Latest" or "Top".
authTokens | array | Authentication tokens for Twitter (X) to improve reliability and access more data.
maxItems | number | Maximum number of items that you want to receive as output
tweetLanguage | string | Restricts tweets to the given language, given by an ISO 639-1 code.
onlyVerifiedUsers | boolean | Returns only tweets by users who are verified.
onlyTwitterBlue | boolean | Returns only tweets by users who are Twitter Blue subscribers.
onlyImage | boolean | Returns only tweets that contain images.
onlyVideo | boolean | Returns only tweets that contain videos.
onlyQuote | boolean | Returns only tweets that are quotes.
author | String | Returns tweets sent by the given user. It should be a Twitter (X) Handle.
inReplyTo | String | Returns tweets that are replies to the given user. It should be a Twitter (X) Handle.
mentioning | String | Returns tweets mentioning the given user. It should be a Twitter (X) Handle.
geotaggedNear | String | Returns tweets sent near the given location.
withinRadius | String | Returns tweets sent within the given radius of the given location.
geocode | String | Returns tweets sent by users located within a given radius of the given latitude/longitude.
placeObjectId | String | Returns tweets tagged with the given place.
minimumRetweets | number | Returns tweets with at least the given number of retweets.
minimumFavorites | number | Returns tweets with at least the given number of favorites.
minimumReplies | number | Returns tweets with at least the given number of replies.
start | String | Returns tweets sent after the given date.
end | String | Returns tweets sent before the given date.
includeSearchTerms | boolean | If selected, a field will be added to each tweet about the search term that was used to find it.
customMapFunction | string | Function that takes each of the objects as argument and returns data that will be mapped by the function itself.
proxy | object | Select proxies to be used by your crawler.
This solution requires the use of **Proxy servers**, either your own proxy servers or you can use [Apify Proxy](https://www.apify.com/docs/proxy).
#### Tip
**Important**: You must provide your own Twitter/X authentication tokens in the `authTokens` array for the scraper to work. These can be obtained from your Twitter accounts.
When you want to scrape over a specific search URL, just copy and paste the link as one of the **startUrl**.
If you would like to scrape only the first page of search results then put the link for the page and have the `endPage` as 1.
With the last approach that is explained above you can also fetch any interval of pages. If you provide the 5th page of a search and define the `endPage` parameter as 6 then you'll have the 5th and 6th pages only.
#### Compute Unit Consumption
The actor is optimized to run blazing fast and scrape many tweets as possible. Therefore, it forefronts all tweet detail requests. If the actor doesn't block very often it'll scrape 100 tweets in 20 seconds with ~0.02-0.025 compute units.
#### Twitter Search Scraper Input example
```json
{
"proxy": {
"useApifyProxy": true
},
"startUrls": [
"https://x.com/KMbappe",
"https://x.com/search?q=real%20madrid&src=typed_query",
"https://x.com/realmadrid/status/1877491794000412879"
],
"searchQueries": [
"barcelona",
"atletico madrid"
],
"searchMode": "Latest",
"maxItems": 100,
"endPage": 5,
"authTokens": [
"auth_token_example_1",
"auth_token_example_2"
],
}
```
### During the Run
During the run, the actor will output messages letting you know what is going on. Each message always contains a short label specifying which page from the provided list is currently specified.
When items are loaded from the page, you should see a message about this event with a loaded item count and total item count for each page.
If you provide incorrect input to the actor, it will immediately stop with a failure state and output an explanation of what is wrong.
### Tweet Export
During the run, the actor stores results into a dataset. Each tweet is a separate item in the dataset.
You can manage the results in any language (Python, PHP, Node JS/NPM). See the FAQ or our API reference to learn more about getting results from this Twitter Search actor.
### Scraped Twitter Properties
The structure of each tweet looks like this:
#### Tweet Detail
```json
{
"id": "1877102356627673443",
"url": "https://x.com/10Ronaldinho/status/1877102356627673443",
"text": "Hermosos recuerdos en Barcelona... Sempre Barça!!!!! https://t.co/1ljiuvNHf9",
"retweetCount": 4078,
"replyCount": 357,
"likeCount": 58982,
"quoteCount": 147,
"createdAt": "Wed Jan 08 21:17:31 +0000 2025",
"lang": "es",
"protected": false,
"bookmarkCount": 421,
"isReply": false,
"entities": {
"hashtags": [],
"media": [
{
"display_url": "pic.x.com/1ljiuvNHf9",
"expanded_url": "https://x.com/10Ronaldinho/status/1877102356627673443/photo/1",
"id_str": "1877102283202162688",
"indices": [
53,
76
],
"media_key": "3_1877102283202162688",
"media_url_https": "https://pbs.twimg.com/media/GgzOlM1W8AAXri7.jpg",
"type": "photo",
"url": "https://t.co/1ljiuvNHf9",
"ext_media_availability": {
"status": "Available"
},
"features": {
"all": {
"tags": [
{
"user_id": "96951800",
"name": "FC Barcelona",
"screen_name": "FCBarcelona",
"type": "user"
}
]
},
"large": {
"faces": []
},
"medium": {
"faces": []
},
"small": {
"faces": []
},
"orig": {
"faces": []
}
},
"sizes": {
"large": {
"h": 1280,
"w": 1280,
"resize": "fit"
},
"medium": {
"h": 1200,
"w": 1200,
"resize": "fit"
},
"small": {
"h": 680,
"w": 680,
"resize": "fit"
},
"thumb": {
"h": 150,
"w": 150,
"resize": "crop"
}
},
"original_info": {
"height": 1280,
"width": 1280,
"focus_rects": [
{
"x": 0,
"y": 0,
"w": 1280,
"h": 717
},
{
"x": 0,
"y": 0,
"w": 1280,
"h": 1280
},
{
"x": 111,
"y": 0,
"w": 1123,
"h": 1280
},
{
"x": 352,
"y": 0,
"w": 640,
"h": 1280
},
{
"x": 0,
"y": 0,
"w": 1280,
"h": 1280
}
]
},
"media_results": {
"result": {
"media_key": "3_1877102283202162688"
}
}
}
],
"symbols": [],
"timestamps": [],
"urls": [],
"user_mentions": []
},
"extentedEntities": {
"media": [
{
"display_url": "pic.x.com/1ljiuvNHf9",
"expanded_url": "https://x.com/10Ronaldinho/status/1877102356627673443/photo/1",
"id_str": "1877102283202162688",
"indices": [
53,
76
],
"media_key": "3_1877102283202162688",
"media_url_https": "https://pbs.twimg.com/media/GgzOlM1W8AAXri7.jpg",
"type": "photo",
"url": "https://t.co/1ljiuvNHf9",
"ext_media_availability": {
"status": "Available"
},
"features": {
"all": {
"tags": [
{
"user_id": "96951800",
"name": "FC Barcelona",
"screen_name": "FCBarcelona",
"type": "user"
}
]
},
"large": {
"faces": []
},
"medium": {
"faces": []
},
"small": {
"faces": []
},
"orig": {
"faces": []
}
},
"sizes": {
"large": {
"h": 1280,
"w": 1280,
"resize": "fit"
},
"medium": {
"h": 1200,
"w": 1200,
"resize": "fit"
},
"small": {
"h": 680,
"w": 680,
"resize": "fit"
},
"thumb": {
"h": 150,
"w": 150,
"resize": "crop"
}
},
"original_info": {
"height": 1280,
"width": 1280,
"focus_rects": [
{
"x": 0,
"y": 0,
"w": 1280,
"h": 717
},
{
"x": 0,
"y": 0,
"w": 1280,
"h": 1280
},
{
"x": 111,
"y": 0,
"w": 1123,
"h": 1280
},
{
"x": 352,
"y": 0,
"w": 640,
"h": 1280
},
{
"x": 0,
"y": 0,
"w": 1280,
"h": 1280
}
]
},
"media_results": {
"result": {
"media_key": "3_1877102283202162688"
}
}
}
]
},
"author": {
"username": "10Ronaldinho",
"url": "https://x.com/10Ronaldinho",
"id": "29479000",
"name": "Ronaldinho",
"isVerified": false,
"isBlueVerified": true,
"profilePicture": "https://pbs.twimg.com/profile_images/1537593562619060225/AmmxIwWi_normal.jpg",
"coverPicture": "https://pbs.twimg.com/profile_banners/29479000/1650911561",
"description": "Twitter oficial de Ronaldinho Gaúcho. Duas vezes eleito o melhor do mundo pela FIFA.\n\n@BetifyOfficial embaixador\nhttps://t.co/jctU4nkSvp",
"location": "",
"followers": 21437149,
"following": 152,
"canDm": false,
"canMediaTag": false,
"createdAt": "Tue Apr 07 16:11:50 +0000 2009",
"entities": {
"description": {
"urls": [
{
"display_url": "bit.ly/RonniexBetify",
"expanded_url": "https://bit.ly/RonniexBetify",
"url": "https://t.co/jctU4nkSvp",
"indices": [
113,
136
]
}
]
},
"url": {
"urls": [
{
"display_url": "nooke.ag",
"expanded_url": "http://www.nooke.ag/",
"url": "https://t.co/f6O6lqd6yM",
"indices": [
0,
23
]
}
]
}
},
"fastFollowersCount": 0,
"favouritesCount": 1552,
"listedCount": 26192,
"hasCustomTimelines": false,
"isTranslator": false,
"mediaCount": 3449,
"statusesCount": 7101,
"withheldInCountries": [],
"affiliatesHighlightedLabel": {}
}
}
```
### Contact
Please visit us through [epctex.com](https://epctex.com) to see all the products that are available for you. If you are looking for any custom integration or so, please reach out to us through the chat box in [epctex.com](https://epctex.com). In need of support? is at your service.
# Actor input Schema
## `startUrls` (type: `array`):
You can provide a list of starting URLs for the scraper, which may include search URLs only.
## `authTokens` (type: `array`):
Select authentication tokens to be used by your crawler.
## `searchTerms` (type: `array`):
Search terms you want to search from Twitter (X). You can refer to https://github.com/igorbrigadir/twitter-advanced-search.
## `maxItems` (type: `integer`):
Maximum number of items that you want as output.
## `sort` (type: `string`):
Sorts search results by the given option. Only works with search terms and search URLs.
## `tweetLanguage` (type: `string`):
Restricts tweets to the given language, given by an ISO 639-1 code.
## `onlyVerifiedUsers` (type: `boolean`):
If selected, only returns tweets by users who are verified.
## `onlyTwitterBlue` (type: `boolean`):
If selected, only returns tweets by users who are Twitter Blue subscribers.
## `onlyImage` (type: `boolean`):
If selected, only returns tweets that contain images.
## `onlyVideo` (type: `boolean`):
If selected, only returns tweets that contain videos.
## `onlyQuote` (type: `boolean`):
If selected, only returns tweets that are quotes.
## `author` (type: `string`):
Returns tweets sent by the given user. It should be a Twitter (X) Handle.
## `inReplyTo` (type: `string`):
Returns tweets that are replies to the given user. It should be a Twitter (X) Handle.
## `mentioning` (type: `string`):
Returns tweets mentioning the given user. It should be a Twitter (X) Handle.
## `geotaggedNear` (type: `string`):
Returns tweets sent near the given location.
## `withinRadius` (type: `string`):
Returns tweets sent within the given radius of the given location.
## `geocode` (type: `string`):
Returns tweets sent by users located within a given radius of the given latitude/longitude.
## `placeObjectId` (type: `string`):
Returns tweets tagged with the given place.
## `minimumRetweets` (type: `integer`):
Returns tweets with at least the given number of retweets.
## `minimumFavorites` (type: `integer`):
Returns tweets with at least the given number of favorites.
## `minimumReplies` (type: `integer`):
Returns tweets with at least the given number of replies.
## `start` (type: `string`):
Returns tweets sent after the given date.
## `end` (type: `string`):
Returns tweets sent before the given date.
## `includeSearchTerms` (type: `boolean`):
If selected, a field will be added to each tweets about the search term that was used to find it.
## `customMapFunction` (type: `string`):
Function that takes each of the objects as argument and returns data that will be mapped by the function itself. This function is not intended for filtering, please don't use it for filtering purposes or you will get banned automatically.
## `proxy` (type: `object`):
Select proxies to be used by your crawler.
## Actor input object example
```json
{
"startUrls": [
"https://x.com/search?q=real%20madrid&src=typed_query"
],
"searchTerms": [
"web scraping",
"scraping from:apify"
],
"maxItems": 1000,
"sort": "Latest",
"tweetLanguage": "en",
"author": "apify",
"inReplyTo": "webexpo",
"mentioning": "elonmusk",
"geotaggedNear": "Los Angeles",
"withinRadius": "15km",
"geocode": "37.7764685,-122.4172004,10km",
"placeObjectId": "96683cc9126741d1",
"minimumRetweets": 5,
"minimumFavorites": 5,
"minimumReplies": 5,
"start": "2021-07-01",
"end": "2021-07-02",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useApifyProxy": true
}
}
```
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"startUrls": [
"https://x.com/search?q=real%20madrid&src=typed_query"
],
"searchTerms": [
"web scraping",
"scraping from:apify"
],
"maxItems": 1000,
"sort": "Latest",
"tweetLanguage": "en",
"author": "apify",
"inReplyTo": "webexpo",
"mentioning": "elonmusk",
"geotaggedNear": "Los Angeles",
"withinRadius": "15km",
"geocode": "37.7764685,-122.4172004,10km",
"placeObjectId": "96683cc9126741d1",
"minimumRetweets": 5,
"minimumFavorites": 5,
"minimumReplies": 5,
"start": "2021-07-01",
"end": "2021-07-02",
"customMapFunction": (object) => { return {...object} }
};
// Run the Actor and wait for it to finish
const run = await client.actor("epctex/twitter-search-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"startUrls": ["https://x.com/search?q=real%20madrid&src=typed_query"],
"searchTerms": [
"web scraping",
"scraping from:apify",
],
"maxItems": 1000,
"sort": "Latest",
"tweetLanguage": "en",
"author": "apify",
"inReplyTo": "webexpo",
"mentioning": "elonmusk",
"geotaggedNear": "Los Angeles",
"withinRadius": "15km",
"geocode": "37.7764685,-122.4172004,10km",
"placeObjectId": "96683cc9126741d1",
"minimumRetweets": 5,
"minimumFavorites": 5,
"minimumReplies": 5,
"start": "2021-07-01",
"end": "2021-07-02",
"customMapFunction": "(object) => { return {...object} }",
}
# Run the Actor and wait for it to finish
run = client.actor("epctex/twitter-search-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"startUrls": [
"https://x.com/search?q=real%20madrid&src=typed_query"
],
"searchTerms": [
"web scraping",
"scraping from:apify"
],
"maxItems": 1000,
"sort": "Latest",
"tweetLanguage": "en",
"author": "apify",
"inReplyTo": "webexpo",
"mentioning": "elonmusk",
"geotaggedNear": "Los Angeles",
"withinRadius": "15km",
"geocode": "37.7764685,-122.4172004,10km",
"placeObjectId": "96683cc9126741d1",
"minimumRetweets": 5,
"minimumFavorites": 5,
"minimumReplies": 5,
"start": "2021-07-01",
"end": "2021-07-02",
"customMapFunction": "(object) => { return {...object} }"
}' |
apify call epctex/twitter-search-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"type": "http",
"url": "https://mcp.apify.com/?tools=fetch-actor-details,epctex/twitter-search-scraper"
}
}
}
```
The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).
## OpenAPI specification
Download the OpenAPI definition: https://api.apify.com/v2/actors/ZhtmZ8nXsl5iI9zLH/builds/O9x3l1SSTO5HJfURi/openapi.json