How to Scrape Youtube Data: Step-by-Step Guide

9 min readNov 10, 2025

YouTube is one of the world’s largest content platforms, with over 360 hours of video uploaded every minute and a library of more than 5 billion videos in 2025. This incredible scale makes it a goldmine for anyone looking to analyze trends, monitor competitors, or gather audience insights. In this guide, we’ll show how to use a YouTube scraper and access the wealth of all this data efficiently.

YouTube web scraping: what data can you get

Before diving into the tutorial, it’s helpful to know what types of data you can extract. YouTube web scraping allows you to access a wide range of publicly available information, including:

Video metadata: titles, descriptions, upload dates, duration, and view counts
Comments: content, author details, likes, and timestamps
Channel information: subscriber counts, total videos, and channel descriptions
Search results: rankings, video previews, and suggested content
Video statistics: likes, dislikes, and engagement metrics

Using YouTube scraper: the tutorial

Now, let’s get into the actual YouTube web scraping part of the article. For this tutorial, we’re going to use Python, the yt-dlp library, proxies, and Web Scraper API.

You might look into other tools on the market like WebShare, ScrapingBee, or Decodo. However, we chose the Oxylabs’ YouTube scraper tool as it’s known to be very powerful and resilient to anti-bot measures, making it ideal for complex targets like YouTube.

Channel information: subscriber counts, total videos, and channel descriptions
Search results: rankings, video previews, and suggested content
Video statistics: likes, dislikes, and engagement metrics

1.Setting up the environment

If you haven’t already, install the latest version of Python from the official website.

1.1. Installing the dependencies

Next, run the following command in your terminal to install the necessary modules:

pip install yt-dlp requests

1.2 Get Web Scraper API credentials

To follow along with this tutorial, you’ll need an active Web Scraper API subscription — a free trial will suffice.

After signing up, you’ll receive a set of user credentials. Keep them handy — you’ll use these credentials later to pull various types of public YouTube data: video transcripts, subtitles, channel details, subscriber numbers, and search result information.

2. Downloading public video data

Please note that the information in this guide is for educational purposes only. It does not grant any rights to the data, videos, or images referenced, which may be protected by copyright, intellectual property, or other legal restrictions. Before performing any scraping activities, consult your legal counsel and review the website’s terms of service or obtain the appropriate scraping license.

Now, let’s walk through how to scrape a video from a website using yt-dlp, a widely used library for extracting video content from major sharing platforms. For demonstration purposes, you can use this sample video URL.

To get started, import the library and run the download() method, as shown below:

from yt_dlp import YoutubeDL


video_url = "https://www.youtube.com/watch?v=w3XcMfyUGxY"
opts = dict()

with YoutubeDL(opts) as yt:
    yt.download([video_url])

Warning! Make sure ffmpeg is installed before running the script. For macOS users, you can install it via Homebrew: brew install ffmpeg.

After running the code, the video will be downloaded and saved to your project’s current directory.

If you need to download multiple videos at once, check out the YouTube scraper documentation for the instructions.

3. Scraping video transcripts and subtitles

Scraping subtitles and transcripts can unlock valuable natural language data for tasks like LLM training and text analysis. Let’s use a YouTube scraper to gather video transcripts — be sure to replace the placeholder credentials with your own:

import requests
import json


payload = {
    "source": "youtube_transcript",
    "query": "mDveiNIpqyw",
    "context": [{"key": "language_code", "value": "en"}]
}

response = requests.post(
    "https://realtime.oxylabs.io/v1/queries",
    auth=("USERNAME", "PASSWORD"),
    json=payload
)

print(response.status_code)
print(response.json())

with open("yt_transcript.json", "w") as file:
    json.dump(response.json(), file, indent=4)

To scrape video subtitles, you can find a request sample here.

4. Collecting video data

You can also use the yt-dlp library to pull publicly available video metadata. This includes details such as:

TItle
Video resolution
Dimensions
Language
… and more.

Now let’s pull the video details from the file we downloaded earlier. To do this, use the extract_info() method with download=False, which prevents the video from being fetched again. The method will return a dictionary containing all available metadata for that video:

from yt_dlp import YoutubeDL


video_url = "https://www.youtube.com/watch?v=mDveiNIpqyw"
opts = dict()

with YoutubeDL(opts) as yt:
    info = yt.extract_info(video_url, download=False)
    video_title = info.get("title", "")
    width = info.get("width", "")
    height = info.get("height", "")
    language = info.get("language", "")
    print(video_url, video_title, width, height, language)

You can also use this YouTube scraper to download metadata — click here for the instructions.

5. Extracting comments

Disclaimer: The information provided here is for educational purposes only and does not grant any rights to the data, which may be protected by privacy, copyright, or other legal restrictions. Before scraping, consult legal counsel and review the website’s terms of service or obtain the appropriate scraping license.

To retrieve all comments from a video, enable the getcomments option when initializing the yt-dlp library.

After setting getcomments=True, the extract_info() method will retrieve all comment threads along with the video metadata. You can then access just the comments from the returned info dictionary, like this:

from pprint import pprint
from yt_dlp import YoutubeDL


video_url = "https://www.youtube.com/watch?v=w3XcMfyUGxY"
opts = {"getcomments": True}

with YoutubeDL(opts) as yt:
    info = yt.extract_info(video_url, download=False)
    comments = info["comments"]
    thread_count = info["comment_count"]
    print("Number of threads: {}".format(thread_count))
    pprint(comments)

6. Scraping channel details

For this example, we’ll extract the channel name and description from the Rockstar Games channel’s “About” section.

6.1. Inspecting elements

The first step is identifying the XPath selectors needed to extract the channel name and description. If you prefer CSS selectors, click here for further information.

To locate these selectors, open the channel’s “About” page in your browser and use Developer Tools to inspect the elements. On Windows, press CTRL + SHIFT + I; on macOS, press Option + Command + I.

By examining the elements, you can build relative XPath selectors based on their associated IDs. The resulting XPath selectors are:

Get Lambert Watts’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Channel name XPath

//h2[@id="title"]/yt-formatted-string/text()

Description XPath

//yt-attributed-string[contains(@id, "description")]/span/text()

6.2 Setting parsing instructions

Next, we’re using the XPath selectors to set up the parsing instructions for our YouTube scraper. This involves creating a dictionary that defines how to extract each piece of data from the HTML content. Start by importing the requests module and defining a variable, instructions, to hold your parsing logic:

import requests


url = "https://www.youtube.com/@rockstargames/about"

instructions = {
    "Channel Name": {
        "_fns": [{
            "_fn": "xpath_one",
            "_args": ['//h2[@id="title"]/yt-formatted-string/text()']
            }]
    },
    "Description": {
            "_fns": [{
                "_fn": "xpath_one",
                "_args": ['//yt-attributed-string[contains(@id, "description")]/span/text()']
            }]
    }
}

Notice the xpath_one function, which instructs the API to select only the first matching element. You can also use similar functions to extract additional channel data, such as social links or other details.

6.3. Preparing payload

Next, create a variable called payload to hold the scraping parameters along with the parsing instructions that will be sent to the API:

payload = {
    "source": "universal",
    "render": "html",
    "parse": True,
    "parsing_instructions": instructions,
    "url": url
}

The render parameter is set to “html”, instructing the API to execute JavaScript and render dynamic content. The parse parameter is set to true, indicating that the payload contains parsing instructions.

6.4 Sending a POST request to the API

Replace USERNAME and PASSWORD with your credentials, then run the code. If successful, the API will return a status_code of 200.

response = requests.post(
    "https://realtime.oxylabs.io/v1/queries",
    auth=("USERNAME", "PASSWORD"),
    json=payload
)
print(response.status_code)

6.5 Extracting channel name and description

Our YouTube scraper returns a JSON response, from which you can extract the parsed channel name and description, as shown below:

channel_name = response.json()["results"][0]["content"]["Channel Name"]
description = response.json()["results"][0]["content"]["Description"]

print(channel_name)
print(description)

That’s it! Here’s the complete code for your convenience:

import requests


url = "https://www.youtube.com/@rockstargames/about"

instructions = {
    "Channel Name": {
        "_fns": [{
            "_fn": "xpath_one",
            "_args": ['//h2[@id="title"]/yt-formatted-string/text()']
            }]
    },
    "Description": {
            "_fns": [{
                "_fn": "xpath_one",
                "_args": ['//yt-attributed-string[contains(@id, "description")]/span/text()']
            }]
    }
}

payload = {
    "source": "universal",
    "render": "html",
    "parse": True,
    "parsing_instructions": instructions,
    "url": url,
}

response = requests.post(
    "https://realtime.oxylabs.io/v1/queries",
    auth=("USERNAME", "PASSWORD"),
    json=payload
)
print(response.status_code)

channel_name = response.json()["results"][0]["content"]["Channel Name"]
description = response.json()["results"][0]["content"]["Description"]

print(channel_name)
print(description)

7. Scraping subscribers count

You can retrieve a channel’s subscriber count using the same method. For this example, we’ll again use the Rockstar Games channel’s “About” page.

Using Developer Tools, you can construct the XPath for the subscriber count:

//td[contains(@class, "ytd-about-channel-renderer") and contains(text(), "subscribers")]/text()

With this XPath, you can define the parsing instructions as shown below:

instructions = {
    "subscribers": {
        "_fns": [{
            "_fn": "xpath_one",
            "_args": [
                '//td[contains(@class, "ytd-about-channel-renderer") '
                'and contains(text(), "subscribers")]/text()'
            ]
        }]
    }
}

As before, the xpath_one function selects only the first matching element. The rest of the code remains largely unchanged — simply replace the parsing instructions in the previously shown API code and extract the subscriber count from the JSON response:

subscribers = response.json()["results"][0]["content"]["subscribers"]
print(subscribers)

Just like before, xpath_one selects the first matching element. Using the same API code, replace the parsing instructions and extract the subscriber count from the JSON response to print it as output.

8. Collecting search results with proxies

You can scrape YouTube search results using either the yt-dlp module or our YouTube scraper tool. Let’s explore both approaches in action.

8.1 Importing modules, defining parameters

We first import the json and yt_dlp modules:

import json
from yt_dlp import YoutubeDL

Then, we define the search query and the number of results we want to scrape:

MAX_RESULTS = 20
QUERY = "oxylabs"

To avoid IP blocks and access localized results, we’ll need some robust, high-quality proxies. For this tutorial, we’re going to use Residential Proxies, but you can look into other options by providers like Decodo or WebShare.

The code below sets up requests through proxies based in the United States:

USER = "USERNAME"
PASSW = "PASSWORD"

opts = {
    "proxy": f"@us-pr.oxylabs.io:10000">https://customer-{USER}:{PASSW}@us-pr.oxylabs.io:10000",
    "default_search": "ytsearch",
    "noplaylist": True,
    "extract_flat": True,
    "quiet": True
}

8.2. Sending the request

Next, pass the opts dictionary into YouTubeDL and run the request using the parameters you defined earlier:

with YoutubeDL(opts) as yt:
    search_query = f"ytsearch{MAX_RESULTS}:{QUERY}"
    info = yt.extract_info(search_query, download=False)
    entries = info.get("entries", [])

8.3. Extracting search results

Inside the yt context manager, we add our extraction logic. We loop through the returned entries and specify which data points we want to collect:

    data = []
    for video in entries:
        data.append({
            "title": video.get("title"),
            "channel": video.get("channel"),
            "view_count": video.get("view_count"),
            "duration": video.get("duration"),
            "url": video.get("url"),
            "channel_url": video.get("channel_url")
        })

Here, you can also extract more metadata, including:

Video IDs
Channel IDs
Thumbnail URLs
… other fields.

If you want to see everything yt-dlp returns, simply add print(video) inside the for loop and run the script to inspect all available data points.

8.4. Saving results to JSON

Finally, use the json module to save all extracted results to a JSON file for later use:

with open("yt_search.json", "w") as file:
    json.dump(data, file, indent=4)

Putting all the steps together, your final Python script will look like this, combining proxy setup, search extraction, and JSON export into one workflow:

import json
from yt_dlp import YoutubeDL


MAX_RESULTS = 20
QUERY = "oxylabs"

USER = "USERNAME"
PASSW = "PASSWORD"

opts = {
    "proxy": f"@us-pr.oxylabs.io:10000">https://customer-{USER}:{PASSW}@us-pr.oxylabs.io:10000",
    "default_search": "ytsearch",
    "noplaylist": True,
    "extract_flat": True,
    "quiet": True
}

with YoutubeDL(opts) as yt:
    search_query = f"ytsearch{MAX_RESULTS}:{QUERY}"
    info = yt.extract_info(search_query, download=False)
    entries = info.get("entries", [])

    data = []
    for video in entries:
        data.append({
            "title": video.get("title"),
            "channel": video.get("channel"),
            "view_count": video.get("view_count"),
            "duration": video.get("duration"),
            "url": video.get("url"),
            "channel_url": video.get("channel_url")
        })

with open("yt_search.json", "w") as file:
    json.dump(data, file, indent=4)

YouTube web scraping with API

In the example above, we had to configure our proxies to avoid blocks. In the long run, we’d have to do proxy maintenance and make sure our scraper doesn’t break. Meanwhile, YouTube scraper automatically extracts public data and uses a built-in proxy pool to bypass blocks. Visit their documentation for more information about filtering search results.

import json
import requests


payload = {
    "source": "youtube_search",
    "query": "oxylabs"
}

response = requests.post(
    "https://realtime.oxylabs.io/v1/queries",
    auth=("USERNAME", "PASSWORD"),
    json=payload
)

print(response.status_code)
print(response.json())

with open("yt_search_api.json", "w") as file:
    json.dump(response.json(), file, indent=4)

YouTube web scraping methods: side-by-side comparison

With all the YouTube scraping methods laid out, let’s compare them side by side and understand when each approach is the most effective:

In summation

This guide covered YouTube scraping methods, from basic tools and proxies to the YouTube Scraper API. We showed how to extract video metadata, comments, channel info, and search results. While DIY methods work for small tasks, the Oxylabs API offers the most reliable, automated, and scalable solution for large-scale YouTube data extraction.

Lambert Watts

How to Scrape Youtube Data: Step-by-Step Guide

YouTube web scraping: what data can you get

Using YouTube scraper: the tutorial

1.Setting up the environment

1.2 Get Web Scraper API credentials

4. Collecting video data

6.1. Inspecting elements

Get Lambert Watts’s stories in your inbox

6.2 Setting parsing instructions

6.3. Preparing payload

6.4 Sending a POST request to the API

6.5 Extracting channel name and description

7. Scraping subscribers count

8. Collecting search results with proxies

8.1 Importing modules, defining parameters

8.2. Sending the request

8.3. Extracting search results

YouTube web scraping with API

YouTube web scraping methods: side-by-side comparison

In summation

Written by Lambert Watts

No responses yet

More from Lambert Watts

What I Learned About Janitor AI, And Why It’s Actually Great If You Use Proxies

I read Ben Fairchild’s piece on why JanitorAI sucks. And honestly? He’s not wrong about the default experience. There’s very little custom…

How to Use Google Sheets to Get Data From a Website in 2026

Knowing how to pull data from a website to Google Sheets is more useful than you might think, and it doesn’t require any coding or add-ons…

cURL POST Requests: Send Data & Headers via cURL

Learn how to send HTTP POST requests with cURL, including headers, JSON, form data, and file uploads, with a practical cURL POST example…

HTTPX vs Requests in 2026: Which Python HTTP Client Should You Use?

If you’re building Python applications that need to make HTTP requests, picking the right library can save you a lot of headaches down the…

Recommended from Medium

10 AI Tools Everyone Will Be Using in Six Months, But Almost Nobody’s Found Yet

You’re going to see these in someone’s toolkit next year and wonder why you didn’t hear about them first. Here’s your head start.

How to Become an AI Engineer in 2026: The Complete Learning Roadmap

Whether you’re starting from scratch or transitioning from software engineering, here’s a structured roadmap to learn the skills companies…

I Became a Millionaire in My 30s by Doing These Boring Money Habits

Why boring financial choices can lead to real wealth

I Gave AI 30 Days to Help Me Build an Online Income Stream. Here’s What Actually Happened

The complete breakdown and blueprint.

8 New AI Terms You Should Know to Be Ahead of 90% of People

A few more AI terms to know beyond tokens, prompts, and hallucinations.

What Wealthy People Do Before 9 AM That Broke People Do Never

A five-year study tracked 233 millionaires and 128 people struggling financially. The differences weren’t income. They weren’t lucky. There…