API in 10 Points

What is an API? What is it used for? Where is it used? How is it used? Are all APIs used the same way? Does every web page have its own API? What’s the difference between it and web scraping?

Türkçe için

This article aims to provide a simple overview of APIs and explain them in a way that everyone can understand.

To further clarify and illustrate the topic, an API application example for the EIA page is cited. In this application example, data is extracted from the EIA (the US Department of Energy’s data service) website using Python, visualized, analyzed, and a predictive model is created. For the full code:

GitHub - nrdnay/eia-Petroleum-Products-Consumption-Production: This project examined Global…

🔍 1. What is an API? What is it used for?

API stands for Application Programming Interface. It can be defined as the connection interface used by an application to access data, server software, or other programs. In other words, API is a bridge that allows one software to communicate with another software, and this communication is established in a standard way within the framework of certain definitions and protocols.

📌 What is it used for?

🖊 Retrieving data without visiting websites

🖊 Accessing data faster and more systematically

🖊 Automating analysis

🖊 Creating applications or visualization dashboards

An API essentially allows users to automate manual tasks.

The most common API uses are:

• In social media applications, when login permission is granted
• When retrieving personal data from Facebook servers
• When contacting banks for e-commerce payments
• In weather applications
• To track real-time stock market data
• To distribute software updates in applications
• To provide location data

🔹 EIA API Application Example: Instead of going to the EIA website each time to communicate with EIA and downloading data as CSV, requests are sent directly with Python, accessing EIA servers through the API, and the data is retrieved as a DataFrame.

🔍 2. What Can Be Done Differently with the API?

Here’s a summary of what we can do differently with the API than without it:

🔹 EIA API Application Example:

Using the EIA website’s API, data for the requested Series ID was requested, the received data was cleaned, and visualized.

🔍 3. Does Every Website Have Its Own API?

Not every website has an API. However, many data services, applications, or government agencies do.

📌 Common institutions/domains with APIs:

🖊 Government institutions (EIA, NASA, TUIK)

🖊 Social media (Twitter, Reddit, YouTube)

🖊 Weather services

🖊 Finance (Stock Exchange APIs, CoinMarketCap)

💡 To see if an API is available: Simply type “[site name] API documentation” (e.g., “EIA API documentation”) into Google.

🔍 4. What is an API Key? How to Obtain It? How to Use It?

An API Key is a password that identifies a user. It is provided to developers to protect the API from misuse.

🔐 Where is it used? It is typically used in headers or params when making data requests.
🔍 How to Obtain It? How to obtain an API key varies for each API.

🔹 According to the EIA API Application Example:

You visit https://www.eia.gov/opendata/ and register,
You request an API key for access,
And the system issues an API key.
📌 Coding Usage:

API_KEY = "Vd3wFJayLszsK4K8dH9tvYfpWPKSbk30UnxWWCna"
headers = {"X-API-KEY": API_KEY}
response = requests.get(url, headers=headers)

🔍 5. Are There APIs Accessible Without an API Key?

Yes, some APIs are password-free and open-access. These APIs do not require authentication. Typically:

🖊 They offer training, sample applications, or publicly available data.

🖊 They have a low data rate limit.

🔓 Examples of Password-Free APIs:

Open-Meteo: Weather API (no API key required)

Rest Countries: Country information (population, flag, language, etc.)

CoinGecko: Cryptocurrency prices (no API key required)

Public-APIs: List of open APIs

💡 However, most major APIs (Google, Twitter, EIA, NASA) require API keys to monitor and control access.

🔍 6. Does API Usage Differ for Each Site? How to Learn How to Use the API?

Each API has different rules. Most REST APIs follow this structure:

BASE_URL + /endpoint + ?param1=…&param2=…

📘 The API documentation is always the primary source. Just like a device’s user manual, every API has its own official documentation, a user guide.

📌 The API documentation includes:

🖊 What endpoints are there?

🖊 How are parameters written?

🖊 Where is the API key placed?

🖊 What is the response structure?

🔹 EIA API Implementation Example:
The EIA API documentation describes several ways to query data:

• Retrieving data using series_id
• Retrieving data by filtering with the data endpoint
• Querying groups using category_id

Querying using series_id was preferred because it is the most direct and simple method. The EIA document (https://www.eia.gov/opendata/documentation.php) explicitly states:

“You can request a specific series using https://api.eia.gov/v2/seriesid/{series_id}”

In other words, the series_id endpoint is used to retrieve a specific time series data from the EIA. The documentation demonstrates how this endpoint works, along with a sample URL.

🔑 In short, the document specifies what is required (such as series_id) and provides examples of the URL structure. The correct series_id values ​​were identified using https://www.eia.gov/opendata/browser/.

🔹 To summarize the flow in the EIA API Implementation Example:

1. API key defined

headers = {"X-API-KEY": API_KEY}

2. After learning the Series ID from the EIA page

series = {
"Tüketim": "INTL.54–2-RUS-TBPD.A",
"Üretim": "INTL.54–1-RUS-TBPD.A"
}

3. The URL is prepared correctly

url = f"https://api.eia.gov/v2/seriesid/{series_id}"

4. Data was requested with requests and the JSON response was read

response = requests.get(url, headers=headers)

5. Data was converted to DataFrame and analyzed with pandas

Get Nurdan Aydogdu’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

df = pd.DataFrame(data[“response”][“data”])

🔍 7. What is an Endpoint?

An endpoint is a special URL extension that provides access to one of the data services offered by an API. It is the path segment following the base address and determines which dataset the API will access.

📌 Base URL (usually fixed):
https://api.eia.gov/v2/ ← This is the base address of the EIA API.

🔹 Explained with an EIA API Application Example:

🔸 1. Series ID endpoint

url = f"https://api.eia.gov/v2/seriesid/{series_id}"

• This provides direct access to a predefined data series.
• Example series_id: “INTL.54–2-IRN-TBPD.A” → Iran’s oil consumption.
• When this endpoint is used, a single time series is extracted as is.

🔸 2. Data endpoint (For example: oil price dataset)

url = "https://api.eia.gov/v2/petroleum/pri/spt/data/"

🖊 This endpoint is a large dataset containing all EIA oil price data.

🖊 This type of endpoint requires specifying details with params:

• Which series?
• Which date range?
• Which frequency? (monthly, yearly, etc.)
• Which columns? (data[0]=value)

📌 Comparative Summary

🔑 To explain it with an analogy: API = Menu, Endpoint = Meal on the Menu

• To order “Chicken,” you need to know the correct meal (endpoint)
• To order “Spicy Chicken,” you need to add a parameter.

🔍 8. Is the request.get() Module Always Used to Retrieve Data from an API?

In Python, the requests module is a common standard for communicating with APIs. Its most basic usage is as follows:

response = requests.get(
url="https://api.example.com/endpoint",
headers={"X-API-KEY": "senin_keyin"},
params={"param1": "deger1", "param2": "deger2"} # opsiyonel
)

📌 requests.get() function:

🖊 Sends a GET request to the URL.

🖊 headers: for authentication, content type, etc.

🖊 params: for parameters to be added to the URL.

📌 There are also alternative methods:

requests.post() → to send data
requests.put() → to update
requests.delete() → to delete

🔍 9. What is params? What does it do?

params → allows query parameters to be added to the URL. It is used to send specific filtering or selection criteria to an API endpoint.

📌 General Syntax:

response = requests.get(
url="https://api.example.com/data",
params={"year": "2023", "country": "USA", "sort": "desc"}
)

The URL this code calls is:

https://api.example.com/data?year=2023&country=USA&sort=desc

So, params, after the ?, forms the dynamic part of the URL.

🎯 What is it used for?

🔹 Using params in the EIA API Application Example

In the EIA API, filtering can be done using params at some endpoints. In the code example below, we see that;

• A specific time range (start, end)
• Specific series (facets)
• A specific order (sort) is requested.

url = "https://api.eia.gov/v2/petroleum/pri/spt/data/"
params = {
"frequency": "monthly",
"data[0]": "value",
"start": "2020–01",
"end": "2023–01",
"facets[series]": ["PET.RWTC.D"],
"sort[0][column]": "period",
"sort[0][direction]": "desc"
}
response = requests.get(url, headers=headers, params=params)

✅ Summary

🔍 10. What are the Differences Between API Applications and Web Scraping? What are their Purposes?

API data extraction and Web Scraping are different methods. Let’s compare them:

📊 1. DEFINITIONS

✅ API (Application Programming Interface)

🖊 A way for websites to present data in a formal and structured manner.

🖊 Data is typically returned via a URL in formats such as JSON and XML.

🖊 Authorization (API key) may be required.

Example: Retrieving annual oil consumption data with the EIA API.

✅ Web Scraping

🖊 This is the process of scraping the HTML content of web pages and extracting data from them.

🖊 The page is visually appealing (e.g., data is extracted from HTML elements such as <table>, <div>, <span>).

If the page design changes, the code must also change.

Example: Seeing the table on the EIA page with your eyes and then trying to extract it with Python.

2. Differences — Comparative Table

📊 3. When and Which?

When to Use an API?

🖊 If the website offers an API: API is always preferred first.

🖊 If the data is clean, structured, and up-to-date.

Example: Weather data, exchange rates, energy statistics.

When to Use Web Scraping?

🖊 If the website doesn’t have an API.

🖊 If you need to access visual data (product names, prices).

Example: Car sales websites, news headlines, forum comments.

🔹 Practical Example:

🧪 With an API:

import requests
url = "https://api.eia.gov/v2/petroleum/sum/snd/data/"
params = {"frequency": "annual", "facets[series]": ["MLUUPUS1"]}
headers = {"X-API-KEY": "YOUR_API_KEY"}
r = requests.get(url, headers=headers, params=params)
data = r.json()

🧪 With Web Scraping:

import requests
from bs4 import BeautifulSoup

url = "https://example.com/lubricants-data"
r = requests.get(url)
soup = BeautifulSoup(r.text, "html.parser")
rows = soup.find_all("tr")

✅ CONCLUSION:

🚩If available: Use the API.
🚩If not available: Scraping may be an option.
🚩 However, web scraping can be sensitive, fragile, and carry legal risks.