Log analysis with essential Linux Tools

This article focuses on understanding each tool individually before moving on to combinations.

I will use a common Apache access log example like this:

192.168.1.10 - - [21/Apr/2026:14:32:10 +0200] "GET /index.html HTTP/1.1" 200 1543 "https://example.com" "Mozilla/5.0"

And usually this file located in:

/var/log/apache2/access.log

For shorter examples, I may also write: access.log

First understand the structure of an Apache log line

Before using the tools, it helps to understand what they are reading:

192.168.1.10 - - [21/Apr/2026:14:32:10 +0200] "GET /index.html HTTP/1.1" 200 1543

Useful fields in a classic Apache log format:

• $1 = client IP
• $4 = beginning of date
• $5 = end of date / timezone
• $6 = HTTP method ("GET)
• $7 = requested URL (/index.html)
• $8 = protocol (HTTP/1.1")
• $9 = HTTP status code (200)
• $10 = response size

This matters especially for awk and sometimes cut.

1. tail

What it does

tail shows the end of a file.

It is useful when you want to:

• see the most recent log entries
• monitor a log in real time
• avoid opening the whole file

For Apache logs, this is often the first command you use.

Think:

tail = show me the latest activity

Basic syntax

tail access.log

By default, it shows the last 10 lines.

Common examples

Show the last 10 lines

tail access.log

Show the last 20 lines

tail -n 20 access.log

Show the last 5 lines

tail -n 5 access.log

Apache examples

Show the latest requests

tail -n 20 /var/log/apache2/access.log

Example output

192.168.1.10 - - [21/Apr/2026:14:31:58 +0200] "GET / HTTP/1.1" 200 1543
203.0.113.5 - - [21/Apr/2026:14:32:01 +0200] "GET /login HTTP/1.1" 404 421
198.51.100.7 - - [21/Apr/2026:14:32:03 +0200] "POST /api/auth HTTP/1.1" 500 812

Important option: -f

-f means follow.

It keeps the command open and shows new lines as they are written.

Monitor the log in real time

tail -f /var/log/apache2/access.log

This is useful when you want to:

• watch traffic live
• test a page in your browser and see the request appear
• watch errors happen in real time
• observe suspicious activity

Useful combination with grep

Watch only 404 errors live

tail -f access.log | grep ' 404 '

Watch requests to /login

tail -f access.log | grep '/login'

Example output

203.0.113.5 - - [21/Apr/2026:14:40:11 +0200] "GET /login HTTP/1.1" 404 421
203.0.113.5 - - [21/Apr/2026:14:40:12 +0200] "GET /login.php HTTP/1.1" 404 421

Common mistakes

• Using tail when you need to inspect the whole file
• Forgetting -f when you want live monitoring
• Thinking tail searches the log; it only shows the end

2. less

What it does

less lets you open and navigate through a large file.

It is much better than cat for logs because:

• logs can be huge
• you can search inside them
• you can move forward and backward
• you can jump to the beginning or end

Think:

less = read and explore

Basic syntax

less access.log

Useful navigation keys

Inside less:

• Shift + G → go to the end
• g → go to the beginning
• /404 → search for 404
• n → next match
• N → previous match
• q → quit

Apache examples

Open the log

less /var/log/apache2/access.log

Search for 500 errors

Inside less, type:

/ 500

Search for /wp-login.php

/wp-login.php

Open directly at the end

less +G access.log

This is useful for logs because the newest entries are at the bottom.

Why less is better than cat

cat access.log prints the whole file at once.
That becomes unreadable on a large log.

less lets you:

• read page by page
• search
• move back
• inspect context

Common mistakes

• Using cat instead of less on huge logs
• Forgetting that /pattern works inside less
• Using less for counting or extracting data instead of analysis tools

3. grep

What it does

grep filters lines that match a pattern.

It is one of the most useful tools for log analysis:

• find 404
• find 500
• find an IP
• find a URL
• find POST
• exclude lines you do not want

Think:

grep = keep only matching lines

Basic syntax

grep 'pattern' access.log

Common examples

Find all 404 errors

grep ' 404 ' access.log

Find /login

grep '/login' access.log

Find POST requests

grep '"POST ' access.log

Find a specific IP

grep '^192.168.1.10 ' access.log

The ^ means “start of line”.

Apache examples

All 500 errors

grep ' 500 ' /var/log/apache2/access.log

Requests to /admin

grep '/admin' /var/log/apache2/access.log

Requests from one IP

grep '^203.0.113.10 ' /var/log/apache2/access.log

Example output

203.0.113.10 - - [21/Apr/2026:15:01:10 +0200] "GET /admin HTTP/1.1" 403 512
203.0.113.10 - - [21/Apr/2026:15:01:12 +0200] "GET /admin/login HTTP/1.1" 404 421

Useful option: -v

-v inverts the match.

Exclude successful requests

grep -v ' 200 ' access.log

Exclude favicon requests

grep -v '/favicon.ico' access.log

Useful option: -E

-E enables extended regular expressions.

Match several patterns

grep -E ' 404 | 500 ' access.log

Search common suspicious paths

grep -E '/admin|/login|/phpmyadmin' access.log

Useful option: -i

Ignore case:

grep -i 'mozilla' access.log

Useful option: -c

Count matching lines directly.

Count 404 errors

grep -c ' 404 ' access.log

Example output

128

Difference between grep -c and grep | wc -l

These both count matches:

grep ' 404 ' access.log | wc -l


grep -c ' 404 ' access.log

Both work. grep -c is shorter.

Common mistakes

• Searching 404 without spaces and matching things you did not intend
• Forgetting quotes around the pattern
• Trying to extract columns with grep instead of awk

When to use it

Use grep when you want:

• fast filtering
• a quick search
• to reduce the log before more analysis

4. awk

What it does

awk is one of the most powerful tools for log analysis.

It can:

• extract columns
• filter with conditions
• reformat output
• count
• do quick analysis

It works very well on Apache logs because many useful values are in fixed positions.

Think:

awk = work with fields

awk reads each line, splits it into fields, and lets you operate on them.

Basic syntax

awk '{print $1}' access.log

This prints the first field.

Understanding fields

For a line like:

192.168.1.10 - - [21/Apr/2026:14:32:10 +0200] "GET /index.html HTTP/1.1" 200 1543

• $1 = IP
• $6 = method with quote
• $7 = URL
• $9 = status code
• $10 = response size

Common examples

Show IP addresses

awk '{print $1}' access.log

Show requested URLs

awk '{print $7}' access.log

Show HTTP status codes

awk '{print $9}' access.log

Show IP + URL + status

awk '{print $1, $7, $9}' access.log

Example output

192.168.1.10 /index.html 200
203.0.113.5 /login 404
198.51.100.7 /api/auth 500

Filtering with conditions

Only 404 errors

awk '$9 == 404 {print}' access.log

All HTTP errors

awk '$9 >= 400 {print}' access.log

Only server errors

awk '$9 >= 500 && $9 < 600 {print}' access.log

Only GET requests

awk '$6 ~ /GET/ {print}' access.log

Cleaning values with gsub

Apache often stores the method as "GET instead of GET.

Remove quotes from the method

awk '{gsub(/"/,"",$6); print $6, $7}' access.log

Example output

GET /index.html
POST /api/auth
GET /login

Simple counting with awk

Count POST requests

awk '$6 ~ /POST/ {count++} END {print count}' access.log

Count all errors

awk '$9 >= 400 {count++} END {print count}' access.log

Example output

87

Apache examples

Show IP + method + URL + code

awk '{gsub(/"/,"",$6); print $1, $6, $7, $9}' access.log

Example output

192.168.1.10 GET /index.html 200
203.0.113.5 GET /login 404
198.51.100.7 POST /api/auth 500

Show only URLs that returned 404

awk '$9 == 404 {print $7}' access.log

Show 500 errors with the source IP

awk '$9 == 500 {print $1, $7, $9}' access.log

Why awk is often better than cut

awk is better for logs because it can:

• filter
• count
• format output
• use conditions

cut is simpler, but much more limited.

Common mistakes

• Using the wrong field number
• Forgetting that the method field may contain quotes
• Thinking awk is only for printing fields

When to use it

Use awk when you want:

• a specific field
• field-based filtering
• custom formatted output
• quick structured analysis

5. cut

What it does

cut extracts pieces of text using a delimiter.

It is simple and fast, but less flexible than awk.

Get Khalil Z.’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Think:

cut = split by a fixed separator

Basic syntax

cut -d' ' -f1 access.log

• -d = delimiter
• -f = field

Common examples

Extract a single field

cut -d':' -f1 file.txt

Extract multiple fields

cut -d',' -f1,3,5 file.txt

Extract a field range

cut -d'|' -f2-4 file.txt

Extract by character position

cut -c1-10 file.txt

Apache examples

Extract IP addresses

cut -d' ' -f1 access.log

Example output

192.168.1.10
203.0.113.5
198.51.100.7

Extract status codes

cut -d' ' -f9 access.log

Example output

200
404
500

Why cut is weaker for Apache logs

Apache logs are not always easy to split cleanly with one delimiter:

• multiple spaces may appear
• quoted fields exist
• dates use brackets
• referers and user-agents are more complex

So cut works for simple cases, but becomes fragile quickly.

cut vs awk

cut

• simpler
• fast for easy cases
• limited

awk

• more powerful
• better for real log analysis
• supports conditions and formatting

Common mistakes

• Using cut on a format that is not perfectly regular
• Expecting cut to understand Apache fields logically
• Using cut where awk would be safer

When to use it

Use cut when:

• the format is simple
• you need a quick extraction
• you do not need conditions

Otherwise, prefer awk.

6. sort

What it does

sort sorts lines.

In log analysis, it is often used to:

• prepare data before uniq
• sort IPs
• sort URLs
• sort counts

Think:

sort = put lines in order

Basic syntax

sort file.txt

Or in a pipeline:

awk '{print $1}' access.log | sort

Common examples

Sort IP addresses

awk '{print $1}' access.log | sort

Sort status codes

awk '{print $9}' access.log | sort

Useful option: -n

Numeric sort:

sort -n file.txt

Useful option: -r

Reverse order:

sort -r file.txt

Very common: -nr

Numeric reverse sort.

This is heavily used after uniq -c.

sort -nr

Apache examples

Prepare IPs for counting

awk '{print $1}' access.log | sort

Count and rank IPs

awk '{print $1}' access.log | sort | uniq -c | sort -nr

Example output

120 192.168.1.10
85 203.0.113.5
42 198.51.100.7

Why sort is usually needed before uniq

uniq only merges adjacent identical lines.
That means sorting usually comes first.

Correct:

awk '{print $1}' access.log | sort | uniq -c

Common mistakes

• Forgetting sort before uniq
• Using plain sort on numbers when sort -n is needed

When to use it

Use sort when you want:

• ordered results
• grouped data
• ranking after counting

7. uniq

What it does

uniq removes adjacent duplicate lines, or counts them with -c.

In log analysis, it is useful for:

• top IPs
• top URLs
• top status codes
• top user-agents

Think:

uniq = group identical lines

Basic syntax

uniq file.txt

In practice, it is usually used after sort.

Most useful option: -c

Count duplicates

sort file.txt | uniq -c

Apache examples

Count IP addresses

awk '{print $1}' access.log | sort | uniq -c

Example output

15 192.168.1.10
 7 10.0.0.5
 3 203.0.113.22

Count requested URLs

awk '{print $7}' access.log | sort | uniq -c

Count HTTP status codes

awk '{print $9}' access.log | sort | uniq -c

Example output

1540 200
120 404
15 500

Build a real top list

Add descending numeric sort:

awk '{print $1}' access.log | sort | uniq -c | sort -nr

With head:

awk '{print $1}' access.log | sort | uniq -c | sort -nr | head

Example output

A Top 10 list of IPs hitting your server, ranked by request count.

120 192.168.1.10
85 203.0.113.5
42 198.51.100.7

Common mistakes

• Forgetting that uniq only works on adjacent duplicates
• Using uniq -c without sorting first
• Forgetting sort -nr to rank results

When to use it

Use uniq when you want:

• grouped repeated values
• counts of repeated lines
• top lists after sorting

8. wc

What it does

wc counts things.

In log analysis, the most useful form is counting lines.

That usually means:

• total requests
• number of errors after filtering
• volume of matching entries

Think:

wc = count

Basic syntax

wc -l access.log

-l means lines.

Common examples

Count total requests

wc -l access.log

Example output

5421 access.log

Count 404 errors

grep ' 404 ' access.log | wc -l

Count POST requests

grep '"POST ' access.log | wc -l

Example output

128

Other options

• -w = words
• -c = characters
• -l = lignes

For logs, -l is the one you use most.

Apache examples

Count requests from one IP

grep '^192.168.1.10 ' access.log | wc -l

Count requests to /login

grep '/login' access.log | wc -l

Count server errors

grep ' 500 ' access.log | wc -l

Common mistakes

• Forgetting -l
• Confusing line count with unique value count
• Using wc when you actually need uniq -c

When to use it

Use wc when you want:

• total volume
• a quick count
• counting after filtering

9. sed

What it does

sed transforms text.

In log analysis, it is often used to:

• clean output
• replace text
• remove unwanted characters
• reformat a result

It is not usually the main analysis tool, but it is very useful to make output easier to read.

Think:

sed = edit text in the stream

Basic syntax

sed 's/old/new/' access.log

• s = substitute
• replaces the first match on each line

Useful option: global replacement

Use g to replace all matches on the line.

sed 's/"//g' access.log

Common examples

Remove all quotes

sed 's/"//g' access.log

Remove square brackets

sed 's/\[//g; s/\]//g' access.log

Replace 404 with NOT_FOUND

sed 's/ 404 / NOT_FOUND /'

Apache examples

Make 404 output easier to read

grep ' 404 ' access.log | sed 's/ 404 / NOT_FOUND /'

Example output

203.0.113.5 - - [21/Apr/2026:15:10:21 +0200] "GET /login HTTP/1.1" NOT_FOUND 421

Remove quotes from the whole line

sed 's/"//g' access.log

Example output

192.168.1.10 - - [21/Apr/2026:14:32:10 +0200] GET /index.html HTTP/1.1 200 1543

Clean the date

sed 's/\[//g; s/\]//g' access.log

Example output

192.168.1.10 - - 21/Apr/2026:14:32:10 +0200 "GET /index.html HTTP/1.1" 200 1543

When sed is useful with other tools

With grep

Filter first, then clean the result:

grep ' 500 ' access.log | sed 's/ 500 / SERVER_ERROR /'

With awk

awk can already do some cleaning, but sed can help with final formatting.

Common mistakes

• Using sed for work better done by awk
• Forgetting g for global replacement
• Thinking it changes the original file by default

When to use it

Use sed when you want:

• cleaner output
• quick substitutions
• basic text transformation in a pipeline

10. Which tool for which job?

See current activity

• tail → watch the end

tail -f access.log

Explore a big log

• less → read

less access.log

Filter matching lines

• grep → filter

grep ' 404 ' access.log

Extract key fields

• awk → extract / analyze

awk '{print $1, $7, $9}' access.log

Quick fixed-column extraction

• cut → split

cut -d' ' -f1 access.log

Sort results

• sort → sort

sort

Count repeated values

• uniq → group / count duplicates

uniq -c

Count lines

• wc → count

wc -l

Clean output

• sed → clean / replace

sed 's/"//g'