Bash: xargs - Build and Execute Commands

xargs is a powerful command that builds and executes commands from standard input. It's essential for processing large lists of files, running commands in parallel, and handling bulk operations efficiently. This lesson will transform how you work with data at scale.

What You'll Learn

Understanding xargs basics
Building command lines from input
Processing files in bulk
Parallel execution
Combining with find, grep, and pipes
Practical pentesting workflows
Safety and best practices

What Is xargs?

xargs = eXtended ARGuments
- Converts stdin to command arguments
- Runs commands on each input item
- Handles spaces and special characters
- Enables parallel execution
- Essential for bulk operations

Problem: Many commands don't accept stdin directly:

# This doesn't work:
find . -name "*.txt" | rm
# rm doesn't read from stdin!

# Solution with xargs:
find . -name "*.txt" | xargs rm
# xargs converts stdin to arguments for rm

Basic xargs Usage

Simple Example

# List files, delete with xargs
ls *.txt | xargs rm

# Echo converts stdin to arguments
echo "file1.txt file2.txt file3.txt" | xargs ls -l

How It Works

Input:  file1.txt\nfile2.txt\nfile3.txt
        ↓
xargs:  Converts newlines/spaces to arguments
        ↓
Output: command file1.txt file2.txt file3.txt

Common xargs Options

-n: Items Per Command

# 1 item per command
echo "a b c" | xargs -n1 echo "Processing:"
# Output:
# Processing: a
# Processing: b
# Processing: c

# 2 items per command
echo "a b c d" | xargs -n2 echo "Pair:"
# Output:
# Pair: a b
# Pair: c d

-I: Replacement String

# Replace {} with each input
echo "file1.txt file2.txt" | xargs -I {} cp {} backup/{}

# More readable with custom placeholder
echo "target1 target2" | xargs -I % nmap -sV %

# Create backups
ls *.txt | xargs -I {} cp {} {}.bak

-P: Parallel Execution

# Run 4 commands in parallel
cat targets.txt | xargs -P 4 -I {} nmap -sV {}

# Max parallel processes
echo {1..10} | xargs -P 5 -n1 sleep

-t: Print Command Before Executing

# Verbose mode (see what's executed)
echo "a b c" | xargs -t rm
# Output: rm a b c

# Useful for debugging
find . -name "*.log" | xargs -t rm

-p: Prompt Before Executing

# Interactive mode
echo "important.txt" | xargs -p rm
# rm important.txt?... (y/n)

# Confirm each action
find . -name "*.tmp" | xargs -p rm

-0: Null Separator

# Handle filenames with spaces/newlines
find . -name "*.txt" -print0 | xargs -0 rm

# Safer than default
locate -0 "*.conf" | xargs -0 grep "password"

-r: No Run if Empty

# Don't run if no input
find . -name "*.xyz" | xargs -r rm
# If no *.xyz files, rm won't run

# Prevents errors with empty input

xargs with find

Basic Combination

# Delete found files
find . -name "*.tmp" | xargs rm

# Better: use -print0 and -0
find . -name "*.tmp" -print0 | xargs -0 rm

# Alternative: find -exec
find . -name "*.tmp" -exec rm {} +

Examples:

# Change permissions
find . -type f -name "*.sh" -print0 | xargs -0 chmod +x

# Move files
find . -name "*.log" -print0 | xargs -0 -I {} mv {} /var/log/

# Search in files
find . -name "*.conf" -print0 | xargs -0 grep "password"

# Count lines in all files
find . -type f -print0 | xargs -0 wc -l

# Copy files to directory
find . -name "*.jpg" -print0 | xargs -0 -I {} cp {} /backup/

xargs with grep

Search Multiple Files

# Find pattern in list of files
cat filelist.txt | xargs grep "pattern"

# With line numbers
cat filelist.txt | xargs grep -n "error"

# Case-insensitive
cat filelist.txt | xargs grep -i "password"

Examples:

# Search for IPs in logs
ls *.log | xargs grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'

# Find credentials in configs
find /etc -name "*.conf" -print0 | xargs -0 grep -i "password"

# Search recursively
find . -type f -print0 | xargs -0 grep -l "TODO"

Parallel Execution

Run Commands in Parallel

# Scan multiple targets in parallel
cat targets.txt | xargs -P 10 -I {} nmap -sV {}

# Process files in parallel
ls *.jpg | xargs -P 4 -I {} convert {} -resize 50% small_{}

# Download in parallel
cat urls.txt | xargs -P 5 -n1 wget

Examples:

# Ping sweep (parallel)
seq 1 254 | xargs -P 50 -I {} ping -c 1 -W 1 192.168.1.{} | grep "64 bytes"

# Hash files in parallel
find . -type f -print0 | xargs -0 -P 4 md5sum

# Compress files in parallel
ls *.txt | xargs -P 4 -I {} gzip {}

Handling Spaces and Special Characters

The Problem

# File with spaces: "my file.txt"
ls "my file.txt" | xargs rm
# Tries to delete "my" and "file.txt" separately!

The Solution: -print0 and -0

# Safe method
find . -name "*.txt" -print0 | xargs -0 rm

# With locate
locate -0 "my file" | xargs -0 ls -l

# Manual null separator
printf "file 1.txt\0file 2.txt\0" | xargs -0 echo

Practical Pentesting Scenarios

Scenario 1: Bulk Port Scanning

# Scan list of targets
cat targets.txt | xargs -P 10 -I {} nmap -sV -p 80,443 {} -oN {}_scan.txt

# Quick host discovery
cat ips.txt | xargs -P 20 -I {} ping -c 1 -W 1 {} | grep "64 bytes" | cut -d' ' -f4 | sed 's/:$//' > live_hosts.txt

# Service enumeration
cat live_hosts.txt | xargs -P 5 -I {} nmap -sV -p- {} -oN {}_full.txt

Scenario 2: Subdomain Enumeration

# Resolve subdomains
cat subdomains.txt | xargs -P 20 -I {} host {} | grep "has address"

# Check HTTP status
cat subdomains.txt | xargs -P 10 -I {} curl -s -o /dev/null -w "%{http_code} {}\n" https://{}

# Screenshot subdomains
cat live_subs.txt | xargs -P 5 -I {} gowitness single https://{}

Scenario 3: Find Credentials

# Search for passwords in files
find /var/www -type f -print0 | xargs -0 grep -i "password"

# Find database strings
find . -name "*.php" -print0 | xargs -0 grep -i "mysql://"

# Search for API keys
find . -type f -print0 | xargs -0 grep -oE '[0-9a-fA-F]{32}'

Scenario 4: Brute Force Attack

# Test usernames
cat users.txt | xargs -P 5 -I {} hydra -l {} -p password123 ssh://target

# Try passwords
cat passwords.txt | xargs -P 10 -I {} sshpass -p {} ssh user@target "echo success"

# Dictionary attack
cat wordlist.txt | xargs -I {} john --wordlist={} hashes.txt

Scenario 5: Log Analysis

# Extract IPs from logs
find /var/log -name "*.log" -print0 | xargs -0 grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | sort -u

# Find errors in logs
find /var/log -type f -print0 | xargs -0 grep -i "error" | wc -l

# Parse failed logins
grep "Failed password" /var/log/auth.log | grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | xargs -I {} echo "Failed login from: {}"

Scenario 6: File Operations

# Backup important files
find /etc -name "*.conf" -print0 | xargs -0 -I {} cp {} /backup/

# Change ownership
find /var/www -type f -print0 | xargs -0 chown www-data:www-data

# Calculate hashes
find . -name "*.exe" -print0 | xargs -0 md5sum > hashes.txt

# Compress logs
find /var/log -name "*.log" -mtime +7 -print0 | xargs -0 -P 4 gzip

xargs vs find -exec

Comparison:

xargs:

find . -name "*.txt" | xargs rm
# Runs: rm file1.txt file2.txt file3.txt
# Fewer processes

find -exec:

find . -name "*.txt" -exec rm {} \;
# Runs: rm file1.txt; rm file2.txt; rm file3.txt
# More processes

find . -name "*.txt" -exec rm {} +
# Runs: rm file1.txt file2.txt file3.txt
# Similar to xargs

When to Use Each:

Use xargs:

Need parallel execution (-P)
Complex command building
Want to see commands (-t)
Interactive confirmation (-p)

Use find -exec:

Simple one-liner
Don't need parallel
Simpler syntax

Safety Practices

1. Always Test First

# Preview without executing
find . -name "*.tmp" | xargs -t echo rm
# Shows what would be deleted

# Or use -p for confirmation
find . -name "*.tmp" | xargs -p rm

2. Use -print0 and -0

# Safe with spaces/newlines
find . -name "*.txt" -print0 | xargs -0 command

# Not safe:
find . -name "*.txt" | xargs command

3. Limit Parallel Jobs

# Don't overwhelm system
cat targets.txt | xargs -P 10 -I {} nmap {}
# Not: xargs -P 1000 (too many!)

4. Use -r to Avoid Empty Runs

# Prevents errors if no input
find . -name "*.xyz" | xargs -r rm

Practical Exercises

Exercise 1: Basic xargs

# Create test files
touch file{1..5}.txt

# Delete with xargs
ls *.txt | xargs rm

# Verify
ls *.txt

Exercise 2: With find

# Create files
mkdir test
touch test/file{1..10}.log

# Find and delete
find test -name "*.log" -print0 | xargs -0 rm

# Verify
ls test/

Exercise 3: Parallel Execution

# Time sequential
time echo {1..5} | xargs -n1 sleep

# Time parallel
time echo {1..5} | xargs -P 5 -n1 sleep

# Compare speeds!

Exercise 4: Build Commands

# Create backups
ls *.txt 2>/dev/null | xargs -I {} cp {} {}.bak

# Rename with xargs
ls *.txt 2>/dev/null | xargs -I {} mv {} old_{}

# Verify
ls

Quick Reference

Basic Usage:
command | xargs command2         # Convert stdin to args

Common Options:
-n N     Items per command
-I {}    Replacement string
-P N     Parallel execution (N jobs)
-t       Print command before executing
-p       Prompt before executing
-0       Null separator (with -print0)
-r       Don't run if empty input

With find:
find ... -print0 | xargs -0 cmd  # Safe method
find ... | xargs cmd             # Simple method
find ... -exec cmd {} +          # Alternative

Parallel:
cat list | xargs -P 10 -I {} command {}

Preview:
command | xargs -t echo          # Show without running
command | xargs -p               # Ask before running

Safe Deletion:
find . -name "*.tmp" -print0 | xargs -0 -p rm

Resources

Man Pages:

man xargs
man find
man parallel  # GNU parallel (xargs on steroids)

Online:

GNU Parallel: https://www.gnu.org/software/parallel/
xargs Examples: https://www.tecmint.com/xargs-command-examples/

Next Steps

Replace loops with xargs for speed
Always use -print0/-0 for safety
Master parallel execution (-P)
Learn GNU parallel for advanced use
Combine with find for powerful workflows

xargs is a game-changer for bulk operations. Master it and you'll handle thousands of files effortlessly!

"xargs: Because typing the same command 1000 times is not an option."