# X (Twitter) Community Search Post Scraper 𝕏 (`api-ninja/x-twitter-community-search-post-scraper`) Actor Extract tweets from X (Twitter) communities. Process single or multiple communities, get comprehensive tweet data with community context, and handle large-scale data collection efficiently. - **URL**: https://apify.com/api-ninja/x-twitter-community-search-post-scraper.md - **Developed by:** [API ninja](https://apify.com/api-ninja) (community) - **Categories:** Social media, Integrations, Automation - **Stats:** 381 total users, 82 monthly users, 99.7% runs succeeded, 39 bookmarks - **User rating**: 4.94 out of 5 stars ## Pricing from $0.35 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## X (Twitter) Community Tweets Scraper **Extract tweets, media, and engagement data from any X (Twitter) Community. Support for multiple communities, pagination, and sorting options.** --- ### What does X (Twitter) Community Tweets Scraper do? **X (Twitter) Community Tweets Scraper** is a powerful Apify Actor designed to **scrape X (Twitter) Communities** without needing the official Twitter API. It allows you to extract public posts, media, and engagement metrics from niche communities on X, delivering the data in structured formats like **JSON, CSV, Excel, or HTML**. This is a **no-code** tool suitable for everyoneβ€”from developers to marketers. It handles complex tasks like pagination, rate limiting, and proxy management automatically, so you can focus on the data. It is the perfect tool to **download X Community data** for analysis. #### Who is this for? * **Researchers**: Analyzing social groups, public discourse, and niche topics. * **Marketers**: Tracking community sentiment, trends, and influencer activity. * **Data Analysts**: Building datasets for NLP, sentiment analysis, or social science research. * **Businesses**: Monitoring brand mentions and feedback within specific communities. ### Use Cases for X (Twitter) Community Data * **Sentiment Analysis**: Gauge how specific communities feel about topics, products, or events. * **Trend Monitoring**: Identify emerging trends and viral content within niche groups before they hit the mainstream. * **Lead Generation**: Find active users and potential leads discussing relevant topics in specialized communities. * **Content Strategy**: Analyze top-performing posts to understand what resonates with specific audiences. * **Academic Research**: Collect large-scale datasets for sociological or political studies. ### Features of X (Twitter) Community Tweets Scraper This tool is built for flexibility and scale. Here is what makes it powerful: * **πŸ‘₯ Scrape Multiple Communities**: Enter a list of Community IDs to scrape them all in a single run. * **↕️ Flexible Sorting**: Sort tweets by **Latest** (real-time monitoring) or **Top** (most popular/viral content). * **πŸ”„ Deep Pagination**: Automatically scrolls through the timeline to collect hundreds or thousands of tweets. * **♾️ Scrape All Mode**: Enable "Scrape All" to retrieve every available tweet from the community timeline. * **πŸ“Š Rich Metadata**: Extracts tweet text, author details, like/retweet counts, media URLs, and timestamps. * **πŸ†” Context Preserved**: Every scraped tweet is tagged with its source `communityId`, making it easy to organize data from multiple sources. #### The Apify Platform Advantage Using this Actor on the Apify platform gives you enterprise-grade benefits: * **Full API Access**: Integrate the scraper programmatically into your existing pipelines. * **Integrations**: Push data directly to **Google Sheets**, **Slack**, **Zapier**, **Make**, or via webhooks. * **Proxy Rotation**: Automatic handling of IP rotation and residential proxies to prevent blocking. * **Scheduling**: Set up cron jobs to monitor communities daily, hourly, or weekly. * **Cloud Storage**: Long-term retention of your datasets in the secure Apify cloud. ### What data can I extract? The scraper provides comprehensive data for every tweet. Here are the main data points: | Field | Description | |-------|-------------| | `full_text` | The actual text content of the tweet. | | `created_at` | The exact date and time the tweet was posted. | | `favorite_count` | Number of likes/hearts. | | `retweet_count` | Number of retweets. | | `reply_count` | Number of replies. | | `user` | Object containing author details (screen name, name, profile image). | | `media` | Links to images or videos attached to the tweet. | | `communityId` | The ID of the community this tweet belongs to. | | `views` | View count (impression metrics). | ### How to scrape X (Twitter) Communities Getting started is easy and requires no programming skills. 1. **Find the Community ID**: Visit a community on X (e.g., [Twitter Communities](https://twitter.com/communities)). The **Community ID** is the long number at the end of the URL (e.g., in `https://twitter.com/i/communities/1472105760389668865`, the ID is `1472105760389668865`). 2. **Input the ID**: Copy that number and paste it into the **Community IDs** field in the Actor input. You can add multiple IDs to scrape them simultaneously. 3. **Choose Sorting**: Select **Latest** to see what's happening now, or **Top** to see the best content. 4. **Set Limits**: Choose how many tweets you want (e.g., 100) or enable **Scrape All** to get everything available. 5. **Run**: Click the **Start** button. ### Pricing This Actor uses a consumption-based model. You only pay for the **Compute Units (CUs)** used to scrape the data. * **Efficient**: Optimized to minimize resource usage. * **Scalable**: The cost scales linearly with the amount of data you extract. * **Transparent**: No hidden fees or separate Twitter API subscriptions required. *Note: The cost per 1,000 tweets can vary depending on the complexity of the community and the depth of the scroll.* ### Input and Output Examples #### Input For a full list of input options, see the **Input tab** in Apify Console. ```json { "communityIds": [ "1472105760389668865", "1567890123456789012" ], "sortBy": "Latest", "numberOfTweets": 100, "scrapeAll": false } ```` #### Output The results are delivered in a structured JSON format. **You can download the dataset in JSON, CSV, Excel, XML, HTML, or RSS, or access it via API.** ```json [ { "created_at": "Fri Feb 01 12:00:00 +0000 2026", "id_str": "1885829471029384756", "full_text": "Just discovered a great new feature in Python! #coding #community", "user": { "id_str": "12345678", "name": "Python Fan", "screen_name": "python_fan_123" }, "favorite_count": 42, "retweet_count": 5, "communityId": "1472105760389668865" } ] ``` ### Related Actors If you need more data from X (Twitter), check out our other specialized tools: - **[X (Twitter) Advanced Search](https://apify.com/api-ninja/x-twitter-advanced-search)**: Perform complex search queries to find historical tweets, monitor hashtags, or track keywords. - **[X (Twitter) Followers Scraper](https://apify.com/api-ninja/x-twitter-followers-scraper)**: Export followers and following lists from any public profile. ### FAQ and Legal #### Is it legal to scrape X (Twitter)? Our scrapers are designed to collect only **publicly available data**. We do not extract private user data (such as email addresses, gender, or location) unless it is publicly shared by the user. We believe that our scrapers, when used for ethical purposes by Apify users, are safe. However, you should not scrape personal data unless you have a legitimate reason to do so. If you're unsure, consult your legal advisors. #### Do I need a Twitter API Key? No! This Actor handles all the authentication and scraping logic for you. You just need an Apify account. #### Why are the results sorted differently than I see on the web? The "Top" and "Latest" algorithms on X can vary slightly based on personalization. Our scraper fetches data directly from the community endpoints to give you the most raw and unfiltered view possible based on your selected sort order. #### Support If you encounter any bugs, have feature requests, or need a custom scraping solution, please create an issue in the **Issues** tab or reach out to us directly. # Actor input Schema ## `communityIds` (type: `array`): Enter the numeric Community IDs of X (Twitter) communities you want to scrape tweets from. Each Community ID is a 19-digit number (e.g., '1472105760389668865'). You can find Community IDs in the URL when visiting a community page on X/Twitter or by using X's API. Add multiple IDs to scrape several communities in a single run. ## `sortBy` (type: `string`): Sort order for tweets. ## `numberOfTweets` (type: `integer`): Specify the maximum number of tweets to collect from EACH community. Scraper will collect up to this many tweets, paginating if necessary. ## `scrapeAll` (type: `boolean`): When enabled, scrapes all tweets found for each community, ignoring the per-community limit. ## Actor input object example ```json { "communityIds": [ "1472105760389668865" ], "sortBy": "Latest", "numberOfTweets": 100, "scrapeAll": false } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "communityIds": [ "1472105760389668865" ] }; // Run the Actor and wait for it to finish const run = await client.actor("api-ninja/x-twitter-community-search-post-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "communityIds": ["1472105760389668865"] } # Run the Actor and wait for it to finish run = client.actor("api-ninja/x-twitter-community-search-post-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "communityIds": [ "1472105760389668865" ] }' | apify call api-ninja/x-twitter-community-search-post-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=api-ninja/x-twitter-community-search-post-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "X (Twitter) Community Search Post Scraper 𝕏", "description": "Extract tweets from X (Twitter) communities. Process single or multiple communities, get comprehensive tweet data with community context, and handle large-scale data collection efficiently.", "version": "0.0", "x-build-id": "2Sud9Gz9qV6RRuyTP" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/api-ninja~x-twitter-community-search-post-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-api-ninja-x-twitter-community-search-post-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/api-ninja~x-twitter-community-search-post-scraper/runs": { "post": { "operationId": "runs-sync-api-ninja-x-twitter-community-search-post-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/api-ninja~x-twitter-community-search-post-scraper/run-sync": { "post": { "operationId": "run-sync-api-ninja-x-twitter-community-search-post-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "communityIds": { "title": "πŸ‘₯ Community IDs", "minItems": 1, "maxItems": 10, "type": "array", "description": "Enter the numeric Community IDs of X (Twitter) communities you want to scrape tweets from. Each Community ID is a 19-digit number (e.g., '1472105760389668865'). You can find Community IDs in the URL when visiting a community page on X/Twitter or by using X's API. Add multiple IDs to scrape several communities in a single run.", "items": { "type": "string", "pattern": "^[0-9]{19}$" } }, "sortBy": { "title": "↕️ Sort by", "enum": [ "Latest", "Top" ], "type": "string", "description": "Sort order for tweets.", "default": "Latest" }, "numberOfTweets": { "title": "Number of Tweets Per Community", "minimum": 20, "type": "integer", "description": "Specify the maximum number of tweets to collect from EACH community. Scraper will collect up to this many tweets, paginating if necessary.", "default": 100 }, "scrapeAll": { "title": "Scrape All Available Results", "type": "boolean", "description": "When enabled, scrapes all tweets found for each community, ignoring the per-community limit.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```