GET Parameters: A Guide to Understanding Browser Requests and Authentication Headers

Whenever we visit a URL, our browsers use the GET request as a default to obtain remote resources hosted at a URL.

Once the browser receives the requested home page, it can send other requests using various HTTP methods.

We can verify this through the Network tab in DevTools.

Basic Authentication

We will perform a challenge at the end of this section. In it, we are presented with a login box requesting the username and password.

Unlike usual login forms that use HTTP parameters to validate credentials (POST request) — this type of authentication presented here uses a basic HTTP authentication method, which directly deals with the webserver to protect a specific page or directory, without interacting directly with the application.

To access the page, we need to enter valid credentials, which in our case are admin: admin.

Once we enter the credentials, we will have access to the following page.

Let’s try to access the page through our terminal using cURL and we will use the flag -i to view the response headers.

curl -i http://<SERVER_IP>:<PORT>/
HTTP/1.1 401 Authorization Required
Date: Mon, 21 Feb 2022 13:11:46 GMT
Server: Apache/2.4.41 (Ubuntu)
Cache-Control: no-cache, must-revalidate, max-age=0
WWW-Authenticate: Basic realm="Access denied"
Content-Length: 13
Content-Type: text/html; charset=UTF-8
Access denied

We received a denied access in the response body and also a Basic realm="Access denied" in the WWW-Authenticate header, which confirms that the page used the basic HTTP auth or basic HTTP authentication.

To pass the credentials through cURL, we use the -u flag.

curl -u admin:admin http://<SERVER_IP>:<PORT>/
<!DOCTYPE html>
<html lang="en">
<head>
...SNIP...

This time we were able to obtain the page. There is another method we can use to pass credentials with basic HTTP auth, which is directly through the URL username:password@URL.

If we try the same thing using cURL or the browser, we can access the page.

curl http://admin:admin@<SERVER_IP>:<PORT>/
<!DOCTYPE html>
<html lang="en">
<head>
...SNIP...

We can try to visit the same URL in the browser and we would get access the same way.

📌 This is a common way we use during CTFs.

HTTP authorization header

If we add the -v flag to our previous cURL command, we will have

curl -v http://admin:admin@<SERVER_IP>:<PORT>/
*   Trying <SERVER_IP>:<PORT>...
* Connected to <SERVER_IP> (<SERVER_IP>) port PORT (#0)
* Server auth using Basic with user 'admin'
> GET / HTTP/1.1
> Host: <SERVER_IP>
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.77.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Mon, 21 Feb 2022 13:19:57 GMT
< Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: no-store, no-cache, must-revalidate
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Pragma: no-cache
< Vary: Accept-Encoding
< Content-Length: 1453
< Content-Type: text/html; charset=UTF-8
< 
<!DOCTYPE html>
<html lang="en">
<head>
...SNIP...

As we are using the basic HTTP auth method, we see that our HTTP request configures the Authorization header to BasicYWRtaW46YWRtaW4=, which is the base 64 encoded value of admin: admin.

If we were using a modern authentication method (such as JWT), the Authorization would be of type Bearer and would contain a long encrypted token.

Let’s try to manually configure the Authorization without removing the credentials, to see if we can access the page.

Get CAROLINE’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

We can set headers with the -H flag and use the same value as the previous request. We can add the -H flag multiple times to specify different headers.

curl -H 'Authorization: Basic YWRtaW46YWRtaW4=' http://<SERVER_IP>:<PORT>/
<!DOCTYPE html
<html lang="en">
<head>
...SNIP...

As we can see, this also gave us access to the page.

These are some methods we can use to authenticate ourselves.

Most modern applications use login forms built with a backend language (such as PHP) that uses POST requests to authenticate users and return a cookie to maintain the login.

GET Parameter

Once we log in, we gain access to the City Search the function where we can search and get a list of corresponding cities.

While the page returns the results, it contacts a remote server to get the information and then returns it to us on the page.

To verify this happening, we can open the dev tools and go to the network tab again (CTRL + SHIFT + E).

Before we type the city and see the results, we need to click on the trash can icon in the left corner, to ensure that we clear any past requests and thus can only see the one we will make now.

Now we can search and press enter, and thus we can immediately verify a new request being sent to the backend.

When we click on the request, we see that it is being sent to search.php with the GET parameter search=le used in the URL.

This helps us understand that the search function requests another page to get the results.

Now we can send the same request directly to the search.php page and get the complete results - which will probably return the results to us in a different format (JSON) without the HTML layout.

To send a GET request with cURL, we can use the same URL.

However, the dev tools provide us with a more convenient way to get the complete cURL command.

We can right-click on the request and select Copy>Copy as cURL.

This way, we copy and paste the command into the terminal and execute it. This way, we get the response:

curl 'http://<SERVER_IP>:<PORT>/search.php?search=le' -H 'Authorization: 
Basic YWRtaW46YWRtaW4='

Leeds (UK)

Leicester (UK)

Note

The copied command contains all the headers used in the HTTP request. But we can remove most of them and keep only the authentication headers, such as Authorization.

We can repeat the same request in the browser by selecting Copy>Copy as Fetch.

→ This will copy the same request using the Fetch library in JavaScript.

Thus, we can go to the JavaScript console tab using CRTL + SHIFT + K, paste our Fetch command, and press enter to send the request.

As we can see, the browser sent our request and we can see the response after that. We can view the details by clicking on it.

Thank you for reading my content.

Check out my other works: bento.me