# TripAdvisor & Booking.com Review Aggregator (`second_coming/travel-review-aggregator`) Actor

Scrape hotel, restaurant, and attraction reviews, ratings, pricing, and ranking data from TripAdvisor and Booking.com. Hospitality reputation management tool.

- **URL**: https://apify.com/second\_coming/travel-review-aggregator.md
- **Developed by:** [Richard P](https://apify.com/second_coming) (community)
- **Categories:** Travel, Business
- **Stats:** 1 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.02 / scan run

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Travel Review Aggregator — Apify Actor

Scrape hotel, restaurant, and attraction **reviews**, **ratings**, **pricing**, and **ranking** data from **TripAdvisor** and **Booking.com**.

Built for hospitality reputation management, competitive analysis, and travel market research.

***

### Features

- ✅ **TripAdvisor** — parse `Hotel_Review`, `Restaurant_Review`, and `Attraction_Review` pages
- ✅ **Booking.com** — parse property pages (`booking.com/hotel/…`)
- ✅ **JSON-LD structured data** — primary extraction path (rich, machine-readable)
- ✅ **Meta tag fallback** — OG/Twitter/standard meta tags when JSON-LD is absent
- ✅ **HTML review parsing** — extract whatever reviews are server-side rendered
- ✅ **User-agent rotation** — 10 modern desktop + mobile agents
- ✅ **Polite crawling** — 2–4s random delays between requests
- ✅ **Retry with exponential backoff** — max 3 retries on 403/429/network errors
- ✅ **Graceful abort** — clean shutdown on Apify platform abort signal
- ✅ **Concurrency control** — semaphore-limited (max 3 concurrent requests)
- ✅ **Apify dataset output** — results pushed to the default dataset

***

### Input

| Property | Type | Default | Description |
|---|---|---|---|
| `tripadvisorUrls` | `string[]` | `[]` | List of TripAdvisor listing page URLs |
| `bookingUrls` | `string[]` | `[]` | List of Booking.com property page URLs |
| `maxReviews` | `integer` | `50` | Max reviews to extract per listing (server-side only) |
| `includeReviewerInfo` | `boolean` | `true` | Whether to include reviewer names in output |

#### Example Input (JSON)

```json
{
  "tripadvisorUrls": [
    "https://www.tripadvisor.com/Hotel_Review-g60763-d122439-Reviews-New_York_Marriott_Marquis-New_York_City_New_York.html",
    "https://www.tripadvisor.com/Restaurant_Review-g60763-d4256615-Reviews-ABC_Kitchen-New_York_City_New_York.html"
  ],
  "bookingUrls": [
    "https://www.booking.com/hotel/us/new-york-marriott-marquis.html"
  ],
  "maxReviews": 50,
  "includeReviewerInfo": true
}
```

***

### Output

Each listing produces a dataset item with the following fields:

| Field | Type | Description |
|---|---|---|
| `source` | `string` | `"tripadvisor"` or `"booking"` |
| `name` | `string` | Listing name |
| `url` | `string` | Listing URL |
| `pageType` | `string` | `"hotel"`, `"restaurant"`, `"attraction"`, or `"unknown"` |
| `address` | `string` | Full address |
| `latitude` | `number` | Geographic latitude |
| `longitude` | `number` | Geographic longitude |
| `rating` | `number` | Overall aggregate rating |
| `totalReviews` | `number` | Total review count |
| `ranking` | `string` | TripAdvisor ranking text (e.g. `"#3 of 1,200 things to do in Paris"`) |
| `priceRange` | `string` | Price level or range |
| `starRating` | `number` | Hotel star rating (Booking.com) |
| `telephone` | `string` | Phone number |
| `description` | `string` | Listing description |
| `servesCuisine` | `string[]` | Cuisine types (restaurants) |
| `amenities` | `string[]` | Amenities / features |
| `reviewScore` | `object` | Booking.com score breakdown |
| `reviews` | `array` | Array of review objects |
| `breadcrumbs` | `string[]` | Breadcrumb navigation |
| `scrapedAt` | `string` | ISO-8601 scrape timestamp |

#### Review Object

| Field | Type | Description |
|---|---|---|
| `author` | `string` | Reviewer name (null when `includeReviewerInfo` is false) |
| `rating` | `number` | Individual review rating |
| `date` | `string` | Review date |
| `title` | `string` | Review title |
| `body` | `string` | Review body text |
| `language` | `string` | Review language code |
| `stayDate` | `string` | Stay date (Booking.com only) |
| `roomType` | `string` | Room type (Booking.com only) |
| `travelerType` | `string` | Traveler type (Booking.com only) |

***

### How It Works

#### Scraping Strategy

1. **Primary — JSON-LD**: Parse `<script type="application/ld+json">` blocks for structured data (business info, aggregate ratings, individual reviews).
2. **Secondary — Meta tags**: Extract OG (`og:title`, `og:description`), Twitter, and standard meta tags as fallback.
3. **Tertiary — HTML parsing**: Parse visible HTML for review cards, amenity tags, pricing, and ranking information not available in structured data.

#### Anti-Scraping Measures

Both TripAdvisor and Booking.com actively protect their content. This actor implements:

- **User-agent rotation** — 10 modern browser user agents, randomly selected per request
- **Realistic delays** — 2–4 seconds between requests (configurable via `MIN_DELAY` / `MAX_DELAY`)
- **Retry with backoff** — up to 3 retries with exponential backoff on 403 (Forbidden), 429 (Too Many Requests), and network errors
- **Polite concurrency** — maximum 3 concurrent requests via `asyncio.Semaphore`
- **Realistic headers** — `Accept-Language`, `Sec-Fetch-*`, and other browser-like headers

***

### Limitations

#### Server-Side Rendering

Both TripAdvisor and Booking.com load the **majority of reviews dynamically via JavaScript**. This actor uses plain HTTP requests (not a headless browser), so only reviews present in the initial server-side HTML or JSON-LD blocks will be captured — typically **3–10 reviews** per page, not the full list.

**If you need all reviews**, consider:

- Using Apify's **PuppeteerCrawler** or **PlaywrightCrawler** with this code as a base
- Paginating through review pages (TripAdvisor offers `-oa30-` offset URLs, Booking.com uses API calls)

#### Anti-Scraping Blocks

- TripAdvisor may return 403 or a CAPTCHA page under aggressive crawling
- Booking.com may return 403 or serve a cookie-consent wall
- The built-in delays and retries mitigate this but cannot guarantee access

#### Data Freshness

Ratings, prices, and review counts are **as-of-the-request** snapshots and may change.

***

### Local Development

```bash
## Install dependencies
pip install -r requirements.txt

## Run with Apify CLI
apify run --purge
```

#### Requirements

- Python 3.12+
- `apify` ≥ 3.0
- `beautifulsoup4` with `lxml`
- `httpx` ≥ 0.28

***

### Docker

The included `Dockerfile` builds a production image using `apify/actor-python:3.14`.

```bash
docker build -t travel-review-aggregator .
docker run travel-review-aggregator
```

***

### License

MIT

# Actor input Schema

## `tripadvisorUrls` (type: `array`):

List of TripAdvisor listing page URLs (Hotel\_Review, Restaurant\_Review, Attraction\_Review).

## `bookingUrls` (type: `array`):

List of Booking.com property page URLs.

## `maxReviews` (type: `integer`):

Maximum number of reviews to extract per listing page.

## `includeReviewerInfo` (type: `boolean`):

If enabled, reviewer names and details are included in the output. Disable for privacy-focused scraping.

## Actor input object example

```json
{
  "tripadvisorUrls": [
    "https://www.tripadvisor.com/Attraction_Review-g60763-d103437-Reviews-Statue_of_Liberty-New_York_City_New_York.html"
  ],
  "bookingUrls": [],
  "maxReviews": 5,
  "includeReviewerInfo": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "tripadvisorUrls": [
        "https://www.tripadvisor.com/Attraction_Review-g60763-d103437-Reviews-Statue_of_Liberty-New_York_City_New_York.html"
    ],
    "bookingUrls": [],
    "maxReviews": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("second_coming/travel-review-aggregator").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "tripadvisorUrls": ["https://www.tripadvisor.com/Attraction_Review-g60763-d103437-Reviews-Statue_of_Liberty-New_York_City_New_York.html"],
    "bookingUrls": [],
    "maxReviews": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("second_coming/travel-review-aggregator").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "tripadvisorUrls": [
    "https://www.tripadvisor.com/Attraction_Review-g60763-d103437-Reviews-Statue_of_Liberty-New_York_City_New_York.html"
  ],
  "bookingUrls": [],
  "maxReviews": 5
}' |
apify call second_coming/travel-review-aggregator --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,second_coming/travel-review-aggregator"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/KdnSTWXZvaAqSkYMg/builds/m42bPwUeNHppbEqH8/openapi.json