# Advanced Command-1

# 1️⃣ `grep` — Search Text in Files

## What `grep` does

`grep` searches for **specific words or patterns inside files**.

Syntax:

```plaintext
grep "pattern" file
```

Example:

```plaintext
grep ERROR app.log
```

Output:

```plaintext
[ERROR] Database connection failed
[ERROR] Timeout while connecting
```

* * *

## Useful Options

| Option | Purpose |
| --- | --- |
| `-i` | case insensitive search |
| `-n` | show line numbers |
| `-r` | search recursively |
| `-v` | exclude pattern |
| `-c` | count matches |

* * *

## Example 1 — Find Errors in Logs

```plaintext
grep ERROR /var/log/syslog
```

Used to quickly find **error events**.

* * *

## Example 2 — Search Multiple Files

```plaintext
grep -r "database" /var/log
```

Searches for **database-related logs** in all log files.

* * *

## Example 3 — Real-time Log Filtering

Combine with `tail`:

```plaintext
tail -f app.log | grep ERROR
```

This shows **only error logs in real time**.

Very useful in **production debugging**.

* * *

# 2️⃣ `awk` — Pattern Processing & Data Extraction

`awk` is used to **analyze structured text data and extract fields**.

Logs often have structured format:

```plaintext
192.168.1.10 - - [16/Mar/2026] "GET /index.html" 200
```

Each column is a **field**.

* * *

## Basic Syntax

```plaintext
awk '{print $field_number}' file
```

Example:

```plaintext
awk '{print $1}' access.log
```

Output:

```plaintext
192.168.1.10
192.168.1.11
192.168.1.15
```

This extracts **client IP addresses**.

* * *

## Example 1 — Find Unique Visitors

```plaintext
awk '{print $1}' access.log | sort | uniq
```

Shows unique IPs visiting the website.

* * *

## Example 2 — Count Requests per IP

```plaintext
awk '{print $1}' access.log | sort | uniq -c
```

Output:

```plaintext
50 192.168.1.10
20 192.168.1.11
10 192.168.1.15
```

Useful for detecting **DDoS attacks**.

* * *

## Example 3 — Extract Response Codes

```plaintext
awk '{print $9}' access.log
```

Output:

```plaintext
200
404
500
```

You can analyze **HTTP status distribution**.

* * *

# 3️⃣ `sed` — Stream Editor (Text Modification)

`sed` is used to **modify or transform text in files or streams**.

Syntax:

```plaintext
sed 's/old/new/' file
```

`s` = substitute.

* * *

## Example 1 — Replace Text

```plaintext
sed 's/error/ERROR/' logs.txt
```

Replaces **error → ERROR**.

* * *

## Example 2 — Modify Configuration Files

Example:

```plaintext
port=8080
```

Change to:

```plaintext
port=9090
```

Command:

```plaintext
sed 's/8080/9090/' config.txt
```

* * *

## Example 3 — Remove Lines

Remove blank lines:

```plaintext
sed '/^$/d' file.txt
```

* * *

## DevOps Use Case

Automating config changes in **CI/CD pipelines**.

Example:

```plaintext
sed -i 's/ENV=dev/ENV=prod/' .env
```

This modifies environment settings during deployment.

* * *

# 4️⃣ `journalctl` — View Systemd Logs

Modern Linux systems use **systemd** for service management.

`journalctl` is used to **view logs stored by systemd**.

* * *

## Basic Command

```plaintext
journalctl
```

Shows **system logs**.

* * *

## Example 1 — View Logs for Specific Service

```plaintext
journalctl -u nginx
```

Shows logs for **Nginx service**.

* * *

## Example 2 — Follow Logs in Real Time

```plaintext
journalctl -u nginx -f
```

Similar to:

```plaintext
tail -f
```

* * *

## Example 3 — Logs Since Boot

```plaintext
journalctl -b
```

Shows logs from **current boot session**.

* * *

## Example 4 — Logs in Time Range

```plaintext
journalctl --since "1 hour ago"
```

Useful during **incident investigation**.

* * *

## DevOps Use Case

Example scenario:

```plaintext
Nginx service fails to start
```

Check logs:

```plaintext
journalctl -u nginx
```

You might see:

```plaintext
nginx: configuration file test failed
```

Then fix configuration.

* * *

# 5️⃣ `multitail` — Monitor Multiple Logs

`multitail` is like **tail but more powerful**.

It allows you to **monitor multiple logs simultaneously**.

Install:

```plaintext
sudo apt install multitail
```

* * *

## Example

```plaintext
multitail /var/log/nginx/access.log /var/log/nginx/error.log
```

This opens **two log streams in one terminal**.

* * *

## Features

*   color highlighting
    
*   multiple log windows
    
*   pattern filtering
    

* * *

## DevOps Use Case

During deployment you might monitor:

```plaintext
Nginx logs
Application logs
Database logs
```

Command:

```plaintext
multitail nginx.log app.log mysql.log
```

You can observe **all logs simultaneously**.

* * *

# 🚀 Real Production Debugging Workflow

Example problem:

```plaintext
Users report website errors
```

Step 1 — Watch logs

```plaintext
tail -f /var/log/nginx/error.log
```

Step 2 — Filter errors

```plaintext
tail -f error.log | grep timeout
```

Step 3 — Analyze request patterns

```plaintext
awk '{print $1}' access.log | sort | uniq -c
```

Step 4 — Check service logs

```plaintext
journalctl -u nginx
```

Step 5 — Monitor multiple logs

```plaintext
multitail nginx.log app.log
```

* * *

# 📊 Summary

| Command | Purpose |
| --- | --- |
| `grep` | search patterns in files |
| `awk` | extract and analyze structured data |
| `sed` | modify text streams |
| `journalctl` | view systemd logs |
| `multitail` | monitor multiple logs |

* * *

✅ **Why these tools are critical for DevOps**

They allow engineers to:

*   diagnose production issues quickly
    
*   analyze traffic patterns
    
*   automate configuration updates
    
*   monitor services
    
*   investigate incidents
    

These commands are **used daily in real production environments**.

In real production environments, **SREs and DevOps engineers rely heavily on command-line tools to analyze logs quickly**. When systems fail, there is usually **no time to open heavy tools** — engineers use Linux pipelines to **filter, analyze, and extract insights from logs instantly**.

Below is a **DevOps Log Analysis Toolkit — 10 powerful commands used daily in production debugging**.

* * *

# 🔧 DevOps Log Analysis Toolkit (10 Essential Commands)

These commands help you **search, filter, monitor, summarize, and analyze logs**.

| Command | Purpose |
| --- | --- |
| `grep` | Search patterns in logs |
| `awk` | Extract fields and analyze structured logs |
| `sed` | Modify or transform log data |
| `tail` | View latest logs |
| `head` | View beginning of logs |
| `less` | Read large logs efficiently |
| `sort` | Organize log output |
| `uniq` | Count repeated entries |
| `cut` | Extract specific columns |
| `wc` | Count lines, words, characters |

These commands are usually **combined in pipelines** for powerful debugging.

* * *

# 1️⃣ `grep` — Find Errors Quickly

Search specific patterns inside logs.

Example:

```plaintext
grep ERROR /var/log/syslog
```

Find login failures:

```plaintext
grep "Failed password" /var/log/auth.log
```

### Production use case

Detect suspicious login attempts:

```plaintext
grep "Failed password" /var/log/auth.log
```

Example output:

```plaintext
Failed password for root from 192.168.1.50
Failed password for admin from 192.168.1.60
```

Helps detect **brute force attacks**.

* * *

# 2️⃣ `tail` — View Latest Logs

Logs are **constantly appended**, so the most useful data is often at the end.

Example:

```plaintext
tail /var/log/syslog
```

View last 50 lines:

```plaintext
tail -n 50 /var/log/nginx/error.log
```

* * *

# 3️⃣ `tail -f` — Real-Time Log Monitoring

This is one of the **most used commands in production debugging**.

```plaintext
tail -f /var/log/nginx/access.log
```

You can watch logs **as requests happen**.

Example output:

```plaintext
192.168.1.10 GET /login 200
192.168.1.11 GET /dashboard 200
192.168.1.15 POST /login 401
```

* * *

# 4️⃣ `less` — Efficient Log Reading

Large logs can be **gigabytes in size**.

Use:

```plaintext
less /var/log/syslog
```

Useful features:

| Key | Function |
| --- | --- |
| `/error` | search |
| `n` | next match |
| `Shift+G` | go to end |
| `q` | quit |

* * *

# 5️⃣ `awk` — Extract Structured Data

Many logs have **structured columns**.

Example Nginx log:

```plaintext
192.168.1.10 - - [16/Mar/2026] "GET /index.html" 200
```

Extract IP addresses:

```plaintext
awk '{print $1}' access.log
```

Extract HTTP status codes:

```plaintext
awk '{print $9}' access.log
```

* * *

# 6️⃣ `sort` — Organize Log Output

Sort results alphabetically or numerically.

Example:

```plaintext
sort access.log
```

Often used with other commands.

* * *

# 7️⃣ `uniq` — Count Duplicate Entries

Works best after sorting.

Example:

```plaintext
sort access.log | uniq
```

Count repeated lines:

```plaintext
sort access.log | uniq -c
```

Output example:

```plaintext
50 192.168.1.10
20 192.168.1.11
```

Shows **top IPs accessing the server**.

* * *

# 8️⃣ `cut` — Extract Specific Columns

Used to extract **specific fields separated by delimiters**.

Example log:

```plaintext
2026-03-16 INFO UserLogin
```

Extract timestamp:

```plaintext
cut -d " " -f1 logs.txt
```

`-d` → delimiter  
  
`-f` → field

* * *

# 9️⃣ `wc` — Count Lines

Useful for quick metrics.

Example:

```plaintext
wc -l access.log
```

Output:

```plaintext
12000 access.log
```

Meaning **12,000 requests logged**.

* * *

# 🔟 `sed` — Modify Log Streams

Example replace text:

```plaintext
sed 's/error/ERROR/' logs.txt
```

Remove blank lines:

```plaintext
sed '/^$/d' logs.txt
```

Often used in **automation scripts**.

* * *

# 🚀 Powerful DevOps Pipelines (Real Production Debugging)

Pipelines combine commands with `|`.

* * *

# Example 1 — Find Top IPs Hitting Server

```plaintext
awk '{print $1}' access.log | sort | uniq -c | sort -nr | head
```

Explanation:

| Step | Action |
| --- | --- |
| `awk` | extract IP |
| `sort` | organize |
| `uniq -c` | count occurrences |
| `sort -nr` | highest first |
| `head` | show top results |

Output:

```plaintext
5000 192.168.1.10
2000 192.168.1.20
500 192.168.1.30
```

Detects **traffic spikes or attacks**.

* * *

# Example 2 — Find HTTP 500 Errors

```plaintext
grep " 500 " access.log
```

Or count them:

```plaintext
grep " 500 " access.log | wc -l
```

Shows number of **server errors**.

* * *

# Example 3 — Monitor Errors in Real Time

```plaintext
tail -f app.log | grep ERROR
```

Shows only error logs live.

* * *

# Example 4 — Detect Brute Force Login Attempts

```plaintext
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c
```

Output:

```plaintext
20 192.168.1.50
15 192.168.1.60
```

These IPs may be **attackers**.

* * *

# Example 5 — Find Slow API Requests

If logs contain response time:

```plaintext
awk '$NF > 1 {print $0}' access.log
```

Shows requests taking **more than 1 second**.

* * *

# Example 6 — Count Requests Per Endpoint

```plaintext
awk '{print $7}' access.log | sort | uniq -c | sort -nr
```

Output:

```plaintext
500 /login
300 /dashboard
100 /api/users
```

Shows **most accessed endpoints**.

* * *

# 🔥 Real DevOps Debugging Scenario

Production issue:

```plaintext
Users report website slow
```

Engineer workflow:

Step 1

```plaintext
tail -f /var/log/nginx/access.log
```

Step 2

```plaintext
grep 500 access.log
```

Step 3

```plaintext
awk '{print $1}' access.log | sort | uniq -c | sort -nr
```

Step 4

```plaintext
journalctl -u nginx
```

This quickly identifies **traffic spikes, errors, or crashes**.

* * *

# 📊 Why SREs Use CLI Tools Instead of GUI Tools

Command-line tools:

*   work on **remote servers via SSH**
    
*   handle **huge log files**
    
*   allow **automation**
    
*   support **powerful pipelines**
    

* * *

# ⭐ Key Takeaway

In DevOps and SRE roles, these commands allow engineers to:

*   debug production issues quickly
    
*   detect attacks
    
*   analyze traffic patterns
    
*   monitor services
    
*   extract metrics from logs
    

Mastering these tools means you can **diagnose system problems within minutes**.