Bash: Streams and Redirection

Streams and redirection are fundamental to Unix/Linux philosophy: small programs working together via input/output. Understanding stdin, stdout, stderr, pipes, and redirection will make you exponentially more powerful on the command line. This is one of the most important lessons for mastering Bash.

What You'll Learn

Understanding the three streams (stdin, stdout, stderr)
Output redirection (>, >>)
Input redirection (<)
Pipes (|)
Combining commands
File descriptors
Here documents and here strings
Practical pentesting workflows

The Three Streams

Standard Input (stdin) - File Descriptor 0

stdin = Standard Input
- Data going INTO a program
- Default: keyboard
- File descriptor: 0

Standard Output (stdout) - File Descriptor 1

stdout = Standard Output
- Normal output FROM a program
- Default: terminal screen
- File descriptor: 1

Standard Error (stderr) - File Descriptor 2

stderr = Standard Error
- Error messages FROM a program
- Default: terminal screen (same as stdout)
- File descriptor: 2

Visualization:

         ┌──────────────┐
stdin  ──┤   Command    ├── stdout
(0)      │              ├── stderr
         └──────────────┘   (2)
            (1)

Output Redirection

Redirect stdout to File

Overwrite file (>):

command > file.txt

Append to file (>>):

command >> file.txt

Examples:

# Save output to file
ls -la > directory_listing.txt

# Append date to log
echo "Scan started: $(date)" >> scan.log

# Save nmap output
nmap 192.168.1.1 > scan_results.txt

# Overwrite previous results
echo "New scan" > results.txt

Redirect stderr to File

Redirect errors (2>):

command 2> errors.txt

Examples:

# Save errors only
find / -name "*.conf" 2> errors.txt

# Discard errors (black hole)
find / -name "*.conf" 2> /dev/null

# Scan without error clutter
nmap 192.168.1.0/24 2> /dev/null

Redirect Both stdout and stderr

Method 1: Redirect separately:

command > output.txt 2> errors.txt

Method 2: Redirect both to same file:

command > combined.txt 2>&1

# Modern bash syntax:
command &> combined.txt

Append both:

command >> combined.txt 2>&1

Examples:

# Save everything
nmap 192.168.1.1 > scan.log 2>&1

# Save output, discard errors
nmap 192.168.1.1 > scan.log 2> /dev/null

# Discard everything
command > /dev/null 2>&1

Input Redirection

Redirect stdin from File

Feed file as input (<):

command < input.txt

Examples:

# Sort file contents
sort < unsorted.txt

# Count lines from file
wc -l < file.txt

# Read from file
while read line; do
    echo "Processing: $line"
done < targets.txt

# MySQL from file
mysql -u root -p database < query.sql

Pipes (`|`)

Connect Commands

Pipe stdout of command1 to stdin of command2:

command1 | command2

Chain multiple commands:

command1 | command2 | command3 | command4

Common Pipe Patterns

1. Filter output:

# Show only .txt files
ls -la | grep ".txt"

# Find processes
ps aux | grep nginx

# Show open ports
netstat -tuln | grep LISTEN

2. Sort and process:

# Sort files by size
ls -lh | sort -k5

# Count unique IPs
cat access.log | cut -d' ' -f1 | sort | uniq | wc -l

# Top 10 IPs
cat access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn | head -10

3. Search and extract:

# Find open ports in nmap output
nmap 192.168.1.1 | grep "open"

# Extract usernames
cat /etc/passwd | cut -d':' -f1

# Find SUID binaries
find / -perm -4000 2>/dev/null | grep -v "proc"

Combining Streams and Pipes

Complex Pipelines

Example 1: Log analysis:

# Find errors, count by type
cat /var/log/syslog | grep -i error | cut -d' ' -f5- | sort | uniq -c | sort -rn

Example 2: Find large files:

# Find files > 100MB
find / -type f -size +100M 2>/dev/null | xargs ls -lh | sort -k5 -hr

Example 3: Network recon:

# Live hosts from ping sweep
for i in {1..254}; do
    ping -c 1 -W 1 192.168.1.$i 2>/dev/null | grep "64 bytes" | cut -d' ' -f4 | sed 's/:$//' &
done | tee live_hosts.txt

File Descriptors

What Are They?

File Descriptor = Number representing an open file
0 = stdin
1 = stdout
2 = stderr
3+ = Custom file descriptors

Redirect Specific Descriptors

Redirect stdout to file:

command 1> file.txt
# Same as: command > file.txt

Redirect stderr to stdout:

command 2>&1
# stderr goes where stdout goes

Redirect stdout to stderr:

command 1>&2
# stdout goes where stderr goes

Examples:

# Separate output and errors
command 1> output.txt 2> errors.txt

# Send stderr to stdout, pipe both
command 2>&1 | grep "pattern"

# Suppress errors, keep output
command 2>/dev/null

# Suppress output, keep errors
command 1>/dev/null

# Suppress everything
command &>/dev/null
# Or: command > /dev/null 2>&1

Here Documents (`<<`)

Multi-line Input

Syntax:

command << DELIMITER
line 1
line 2
line 3
DELIMITER

Examples:

1. Create file with heredoc:

cat << EOF > script.sh
#!/bin/bash
echo "Hello, World!"
echo "This is line 2"
EOF

2. SQL commands:

mysql -u root -p database << EOF
CREATE TABLE users (id INT, name VARCHAR(50));
INSERT INTO users VALUES (1, 'Alice');
INSERT INTO users VALUES (2, 'Bob');
EOF

3. SSH commands:

ssh user@remote << EOF
cd /var/www
ls -la
exit
EOF

4. Multi-line echo:

cat << 'EOF'
This is line 1
Variable $HOME is not expanded
EOF

Here Strings (`<<<`)

Single-line Input

Syntax:

command <<< "string"

Examples:

# Count words in string
wc -w <<< "hello world test"

# Reverse string
rev <<< "hello"

# Base64 encode
base64 <<< "secret"

# MD5 hash
md5sum <<< "password"

# Read into variable
read var1 var2 var3 <<< "one two three"
echo $var2  # Outputs: two

The Black Hole: /dev/null

What Is It?

/dev/null = Null device
- Discards all data written to it
- Returns EOF when read from
- Use to discard unwanted output

Common Uses

Discard errors:

find / -name "*.conf" 2> /dev/null

Discard output:

command > /dev/null

Discard everything:

command > /dev/null 2>&1

Silent execution:

# Run command silently
./script.sh &> /dev/null

Practical Pentesting Workflows

Workflow 1: Nmap + Analysis

# Scan and save
nmap -sV -p- 192.168.1.1 > scan_full.txt 2>&1

# Extract open ports
cat scan_full.txt | grep "open" | cut -d'/' -f1 > open_ports.txt

# Count services
cat scan_full.txt | grep "open" | wc -l

# Find specific service
cat scan_full.txt | grep -i "ssh"

Workflow 2: Subdomain Enumeration

# Run multiple tools, combine results
subfinder -d example.com -silent > subs1.txt
amass enum -passive -d example.com 2>/dev/null >> subs1.txt

# Remove duplicates
cat subs1.txt | sort -u > unique_subs.txt

# Check for live hosts
cat unique_subs.txt | while read sub; do
    if ping -c 1 -W 1 "$sub" &>/dev/null; then
        echo "$sub" >> live_subs.txt
    fi
done

Workflow 3: Log Analysis

# Analyze web logs
cat access.log | \
    cut -d' ' -f1 | \
    sort | \
    uniq -c | \
    sort -rn | \
    head -20 > top_ips.txt

# Find failed logins
grep "Failed password" /var/log/auth.log | \
    grep -oP '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}' | \
    sort | \
    uniq -c | \
    sort -rn > failed_login_ips.txt

Workflow 4: Password Cracking

# Crack hashes, save cracked
john hashes.txt --wordlist=rockyou.txt 2>/dev/null | tee crack_progress.txt

# Extract cracked passwords
john --show hashes.txt > cracked.txt

# Statistics
john --show hashes.txt | grep "password hash cracked" | cut -d' ' -f1 > stats.txt

Practical Exercises

Exercise 1: Basic Redirection

# Create test file
ls -la > listing.txt

# Append date
echo "Generated on: $(date)" >> listing.txt

# View result
cat listing.txt

Exercise 2: Error Handling

# Generate errors
find / -name "*.conf" > found.txt 2> errors.txt

# Discard errors
find / -name "*.conf" 2>/dev/null > found_clean.txt

# Save both
find / -name "*.conf" > all_output.txt 2>&1

Exercise 3: Pipes

# Count files
ls -la | wc -l

# Find largest files
ls -lh | sort -k5 -hr | head -5

# Find process
ps aux | grep bash

Exercise 4: Complex Pipeline

# Analyze /etc/passwd
cat /etc/passwd | \
    cut -d':' -f1,3 | \
    sort -t':' -k2 -n | \
    tail -10

# Top 10 largest directories
du -sh /* 2>/dev/null | sort -hr | head -10

Quick Reference

Output Redirection:
command > file               # Overwrite file
command >> file              # Append to file
command 2> file              # Redirect errors
command 2>&1                 # Stderr to stdout
command &> file              # Both to file
command > /dev/null 2>&1     # Discard all

Input Redirection:
command < file               # Read from file
command << EOF               # Here document
command <<< "string"         # Here string

Pipes:
cmd1 | cmd2                  # Connect commands
cmd1 | cmd2 | cmd3           # Chain multiple

File Descriptors:
0 = stdin
1 = stdout  
2 = stderr

Common Patterns:
cmd 2>/dev/null              # Silence errors
cmd | grep pattern           # Filter output
cmd | sort | uniq            # Unique sorted
cmd | wc -l                  # Count lines
cmd | tee file.txt           # Display & save

Resources

Man Pages:

man bash    # Search for "REDIRECTION"

Online:

Bash Guide: https://mywiki.wooledge.org/BashGuide
Advanced Bash Scripting: https://tldp.org/LDP/abs/html/

Next Steps

Practice pipes daily - chain commands together
Master redirection for clean output
Use /dev/null to suppress noise
Build complex pipelines for data processing
Learn tee for simultaneous display and save

Streams and redirection are the heart of command-line power. Master them and you'll work 10x faster!

"| : Because sometimes commands need to talk to each other."

Bash: Streams and Redirection

What You'll Learn

The Three Streams

Standard Input (stdin) - File Descriptor 0

Standard Output (stdout) - File Descriptor 1

Standard Error (stderr) - File Descriptor 2

Visualization:

Output Redirection

Redirect stdout to File

Redirect stderr to File

Redirect Both stdout and stderr

Input Redirection

Redirect stdin from File

Pipes (|)

Connect Commands

Common Pipe Patterns

Combining Streams and Pipes

Complex Pipelines

File Descriptors

What Are They?

Redirect Specific Descriptors

Examples:

Here Documents (<<)

Multi-line Input

Here Strings (<<<)

Single-line Input

The Black Hole: /dev/null

What Is It?

Common Uses

Practical Pentesting Workflows

Workflow 1: Nmap + Analysis

Workflow 2: Subdomain Enumeration

Workflow 3: Log Analysis

Workflow 4: Password Cracking

Practical Exercises

Exercise 1: Basic Redirection

Exercise 2: Error Handling

Exercise 3: Pipes

Exercise 4: Complex Pipeline

Quick Reference

Resources

Man Pages:

Online:

Next Steps

Pipes (`|`)

Here Documents (`<<`)

Here Strings (`<<<`)