By Davy Rogers. Use these techniques only in environments you own or have explicit permission to test.

Network Protocols

Network protocols are the rules and standards that govern how data is transmitted across networks. Think of them as the "languages" that computers use to communicate. Just as humans need a common language to understand each other, devices need protocols to exchange information reliably and securely. For cybersecurity professionals, understanding protocols is critical—every attack exploits or manipulates these communication rules.

What You'll Learn

  • What network protocols are and why they matter
  • Common protocols at each OSI layer
  • How protocols work (handshakes, packets, sessions)
  • Protocol vulnerabilities and exploits
  • How to analyze protocols with Wireshark
  • Secure vs insecure protocols
  • Protocol selection for pentesting

What Is a Network Protocol?

A network protocol is a set of rules defining:

  • How data is formatted
  • How data is transmitted
  • How devices identify each other
  • How errors are handled
  • How connections are established/terminated

Protocol Components:

┌─────────────────────────────────────┐
│ Syntax    │ How data is structured  │
│ Semantics │ What commands mean      │
│ Timing    │ When/how fast to send   │
│ Error     │ What to do when failed  │
└─────────────────────────────────────┘

Example: HTTP Protocol

GET /index.html HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0
Accept: text/html

[Response]
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1234

<html>...</html>

Syntax: Commands like GET, POST Semantics: GET means "retrieve this resource" Timing: Client waits for server response Error handling: Status codes like 404 Not Found


Protocol Suites

Protocols are organized in protocol suites (stacks):

TCP/IP Protocol Suite

┌─────────────────────────────────────┐
│ Application  │ HTTP, FTP, SSH, DNS  │
├─────────────────────────────────────┤
│ Transport    │ TCP, UDP             │
├─────────────────────────────────────┤
│ Internet     │ IP, ICMP, ARP        │
├─────────────────────────────────────┤
│ Link         │ Ethernet, Wi-Fi      │
└─────────────────────────────────────┘

Protocols work together in layers, each handling specific tasks.


Major Network Protocols

Application Layer (Layer 7)

HTTP (HyperText Transfer Protocol)

  • Port: 80
  • Purpose: Web browsing
  • Security: ⚠️ Unencrypted
# HTTP request
curl http://example.com

# View headers
curl -v http://example.com

HTTPS (HTTP Secure)

  • Port: 443
  • Purpose: Secure web browsing
  • Security: ✅ Encrypted with TLS/SSL
# HTTPS request
curl https://example.com

# View certificate
openssl s_client -connect example.com:443

FTP (File Transfer Protocol)

  • Port: 20 (data), 21 (control)
  • Purpose: File transfer
  • Security: ⚠️ Unencrypted (passwords in plaintext!)
# Connect to FTP
ftp example.com

# Anonymous login often allowed
Username: anonymous
Password: [any email]

Pentesting FTP:

# Scan for FTP
nmap -p 21 192.168.1.0/24

# Brute force FTP
hydra -L users.txt -P passwords.txt ftp://192.168.1.5

# Check for anonymous access
ftp 192.168.1.5
# Try: anonymous / anonymous

FTPS (FTP Secure)

  • Port: 990 (implicit), 21 (explicit)
  • Purpose: Encrypted file transfer
  • Security: ✅ TLS/SSL encryption

SFTP (SSH File Transfer Protocol)

  • Port: 22
  • Purpose: Secure file transfer over SSH
  • Security: ✅ Encrypted
# Connect via SFTP
sftp [email protected]

# Upload file
put localfile.txt

# Download file
get remotefile.txt

SSH (Secure Shell)

  • Port: 22
  • Purpose: Remote command-line access
  • Security: ✅ Encrypted
# SSH login
ssh [email protected]

# SSH with key
ssh -i private_key.pem [email protected]

# Port forwarding
ssh -L 8080:localhost:80 [email protected]

SSH Security:

  • Use key-based authentication (not passwords)
  • Disable root login
  • Change default port (security through obscurity)
  • Use fail2ban to prevent brute force

Telnet

  • Port: 23
  • Purpose: Remote command-line access
  • Security: ❌ NEVER USE - everything in plaintext
# Telnet (only for testing services)
telnet example.com 80
GET / HTTP/1.1
Host: example.com

DNS (Domain Name System)

  • Port: 53 (UDP typically, TCP for large transfers)
  • Purpose: Resolve domain names to IP addresses
# DNS lookup
nslookup google.com

# Detailed DNS query
dig google.com

# Reverse DNS
nslookup 8.8.8.8

DNS Enumeration:

# Zone transfer (often fails, but worth trying)
dig @ns1.example.com example.com AXFR

# Enumerate subdomains
dnsrecon -d example.com

# DNS brute force
dnsenum example.com

SMTP (Simple Mail Transfer Protocol)

  • Port: 25 (unencrypted), 465 (SMTPS), 587 (submission)
  • Purpose: Send email
# Connect to SMTP
telnet mail.example.com 25

# SMTP commands
HELO attacker.com
MAIL FROM:<[email protected]m>
RCPT TO:<[email protected]m>
DATA
Subject: Test
This is a test email.
.
QUIT

POP3 (Post Office Protocol)

  • Port: 110 (unencrypted), 995 (encrypted)
  • Purpose: Receive email (download and delete from server)

IMAP (Internet Message Access Protocol)

  • Port: 143 (unencrypted), 993 (encrypted)
  • Purpose: Receive email (sync with server)

DHCP (Dynamic Host Configuration Protocol)

  • Port: 67 (server), 68 (client) - UDP
  • Purpose: Automatically assign IP addresses

DHCP Process (DORA):

1. Discover: Client broadcasts "I need an IP!"
2. Offer: Server responds "Here's an available IP"
3. Request: Client says "I accept that IP"
4. Acknowledge: Server confirms assignment

DHCP Attacks:

  • DHCP Starvation: Request all available IPs (DoS)
  • Rogue DHCP Server: Provide malicious gateway/DNS

NTP (Network Time Protocol)

  • Port: 123 (UDP)
  • Purpose: Synchronize clocks
# Query NTP server
ntpdate -q pool.ntp.org

Why time matters:

  • Log correlation across systems
  • Kerberos authentication (requires time sync)
  • Certificate validity

SNMP (Simple Network Management Protocol)

  • Port: 161 (queries), 162 (traps)
  • Purpose: Network device management
# SNMP enumeration
snmpwalk -v 2c -c public 192.168.1.1

# Brute force community strings
onesixtyone -c community.txt 192.168.1.0/24

Default community strings:

  • public (read-only)
  • private (read-write)

⚠️ SNMPv1 and v2 are insecure - use SNMPv3

RDP (Remote Desktop Protocol)

  • Port: 3389
  • Purpose: Remote desktop access (Windows)
# RDP from Linux
rdesktop 192.168.1.5
xfreerdp /u:Administrator /v:192.168.1.5

# RDP brute force
hydra -l administrator -P passwords.txt rdp://192.168.1.5

BlueKeep (CVE-2019-0708): Critical RDP vulnerability

SMB (Server Message Block)

  • Port: 445 (direct), 137-139 (NetBIOS)
  • Purpose: File/printer sharing (Windows)
# List SMB shares
smbclient -L //192.168.1.5

# Connect to share
smbclient //192.168.1.5/ShareName

# Enumerate with enum4linux
enum4linux -a 192.168.1.5

# Enumerate with nmap
nmap --script smb-enum-shares.nse -p445 192.168.1.5

EternalBlue (MS17-010): SMBv1 exploit (WannaCry ransomware)

LDAP (Lightweight Directory Access Protocol)

  • Port: 389 (unencrypted), 636 (LDAPS)
  • Purpose: Access directory services (Active Directory)
# LDAP search
ldapsearch -h 192.168.1.5 -x -b "dc=example,dc=com"

Transport Layer (Layer 4)

TCP (Transmission Control Protocol)

  • Purpose: Reliable, connection-oriented communication
  • Features:
    • Three-way handshake
    • Guaranteed delivery
    • Flow control
    • Error checking

TCP Header:

Source Port | Dest Port | Seq | Ack | Flags | Window | Checksum

TCP Flags:

  • SYN: Synchronize (start connection)
  • ACK: Acknowledge
  • FIN: Finish (close connection)
  • RST: Reset (abort connection)
  • PSH: Push (send data immediately)
  • URG: Urgent

Three-Way Handshake:

Client → Server: SYN (seq=100)
Server → Client: SYN-ACK (seq=200, ack=101)
Client → Server: ACK (seq=101, ack=201)

TCP Attacks:

  • SYN Flood: Send SYNs, never complete handshake
  • Connection Hijacking: Predict sequence numbers

UDP (User Datagram Protocol)

  • Purpose: Fast, connectionless communication
  • Features:
    • No handshake
    • No reliability
    • Low overhead
    • Good for real-time apps

UDP Header (much simpler):

Source Port | Dest Port | Length | Checksum

Use Cases:

  • DNS
  • DHCP
  • Video streaming
  • VoIP
  • Gaming

UDP Attacks:

  • UDP Flood: Overwhelm with UDP packets

Network Layer (Layer 3)

IP (Internet Protocol)

  • Purpose: Logical addressing and routing

IPv4 Address: 192.168.1.1 (32-bit) IPv6 Address: 2001:0db8::1 (128-bit)

ICMP (Internet Control Message Protocol)

  • Purpose: Error messages and diagnostics

ICMP Types:

  • Type 0: Echo Reply (ping response)
  • Type 3: Destination Unreachable
  • Type 8: Echo Request (ping)
  • Type 11: Time Exceeded (traceroute)
# Ping (ICMP echo)
ping google.com

# Traceroute (ICMP TTL)
traceroute google.com  # Linux
tracert google.com     # Windows

# Advanced ping
ping -c 4 -s 1000 192.168.1.1  # 4 packets, 1000 bytes

ICMP Attacks:

  • Ping Flood: Overwhelm with pings
  • Ping of Death: Oversized ping packet
  • Smurf Attack: Amplified ping flood
  • ICMP Tunneling: Covert channel for data exfiltration

ARP (Address Resolution Protocol)

  • Purpose: Map IP addresses to MAC addresses
# View ARP cache
arp -a

# ARP scan
arp-scan -l

ARP Poisoning:

# MITM attack with arpspoof
arpspoof -i eth0 -t 192.168.1.5 192.168.1.1
arpspoof -i eth0 -t 192.168.1.1 192.168.1.5

# Now intercept traffic with:
tcpdump -i eth0 -w capture.pcap

Ethernet (IEEE 802.3)

  • Purpose: Frame data for transmission on LAN

Frame Structure:

Preamble | Dest MAC | Src MAC | Type | Data | FCS

Wi-Fi (IEEE 802.11)

  • Purpose: Wireless networking

Security Protocols:

  • WEP: ❌ Broken (crack in seconds)
  • WPA: ⚠️ Vulnerable (TKIP deprecated)
  • WPA2: ✅ Secure (AES-CCMP)
  • WPA3: ✅ Most secure (SAE handshake)
# Monitor Wi-Fi
iwconfig
iw dev wlan0 scan

# Crack WPA2 with aircrack-ng
airmon-ng start wlan0
airodump-ng wlan0mon
aireplay-ng --deauth 10 -a [AP_MAC] wlan0mon
aircrack-ng -w wordlist.txt capture.cap

Secure vs Insecure Protocols

Insecure ❌Secure ✅Port
HTTPHTTPS80 → 443
FTPFTPS/SFTP21 → 990/22
TelnetSSH23 → 22
SMTPSMTPS25 → 465
POP3POP3S110 → 995
IMAPIMAPS143 → 993
LDAPLDAPS389 → 636
SNMPv1/v2SNMPv3161

Rule: If unencrypted version exists, never use it in production.


Protocol Analysis with Wireshark

Wireshark is essential for understanding protocols.

# Install Wireshark
sudo apt install wireshark  # Linux
# Download from wireshark.org for Windows

# Capture traffic
sudo wireshark
# Select interface → Start capture

Analyzing HTTP:

  1. Capture HTTP traffic:

    • Open Wireshark
    • Filter: http
    • Visit http://example.com in browser
  2. Observe:

    • TCP three-way handshake (SYN, SYN-ACK, ACK)
    • HTTP GET request
    • HTTP 200 OK response
    • Data transfer
    • TCP connection termination (FIN, ACK)

Wireshark Filters:

# Protocol filters
tcp
udp
icmp
arp
dns
http

# IP-based filters
ip.addr == 192.168.1.5
ip.src == 192.168.1.5
ip.dst == 8.8.8.8

# Port-based filters
tcp.port == 80
tcp.dstport == 443

# Combined filters
http and ip.addr == 192.168.1.5
tcp.port == 22 and tcp.flags.syn == 1

# Follow TCP stream
Right-click packet → Follow → TCP Stream

Exercise: Capture Your Traffic

# Generate different traffic types

# HTTP
curl http://example.com

# HTTPS
curl https://google.com

# DNS
nslookup google.com

# Ping
ping -c 4 google.com

# SSH
ssh [email protected]

Then analyze each in Wireshark. Can you see the difference between encrypted (SSH, HTTPS) and unencrypted (HTTP, Telnet) traffic?


Common Protocol Vulnerabilities

1. Cleartext Protocols

  • Issue: Credentials/data sent unencrypted
  • Examples: HTTP, FTP, Telnet, SMTP
  • Attack: Packet sniffing
  • Fix: Use encrypted alternatives

2. Authentication Weaknesses

  • Issue: Weak or default credentials
  • Examples: SNMP (public/private), FTP (anonymous)
  • Attack: Brute force
  • Fix: Strong passwords, disable defaults

3. Replay Attacks

  • Issue: Capture and resend valid packets
  • Examples: Kerberos tickets, cookies
  • Attack: Replay captured authentication
  • Fix: Nonces, timestamps, session tokens

4. Man-in-the-Middle (MITM)

  • Issue: Intercept communication
  • Examples: ARP poisoning, rogue DHCP, DNS spoofing
  • Attack: Modify or eavesdrop on traffic
  • Fix: Encryption, certificate pinning, ARP inspection

5. Denial of Service (DoS)

  • Issue: Overwhelm target
  • Examples: SYN flood, UDP flood, ping flood
  • Attack: Exhaust resources
  • Fix: Rate limiting, SYN cookies, firewalls

6. Protocol Exploits

  • Issue: Bugs in protocol implementation
  • Examples: EternalBlue (SMB), Heartbleed (OpenSSL)
  • Attack: Remote code execution
  • Fix: Patch systems, disable vulnerable protocols

Protocol Selection in Pentesting

Reconnaissance Phase:

# Discover what protocols are running
nmap -sV 192.168.1.5

# Example output:
# 21/tcp  open  ftp      vsftpd 3.0.3
# 22/tcp  open  ssh      OpenSSH 7.9
# 80/tcp  open  http     Apache 2.4.41
# 443/tcp open  https    nginx 1.18.0

Target Prioritization:

High Priority (easier to exploit):

  1. Unencrypted protocols (FTP, Telnet, HTTP)
  2. Outdated versions (old SSH, SMB)
  3. Default credentials (SNMP, admin panels)

Medium Priority: 4. Services with known CVEs 5. Misconfigured services

Low Priority: 6. Up-to-date, encrypted, hardened services

Protocol-Specific Attacks:

ProtocolCommon Attacks
HTTPSQL injection, XSS, directory traversal
FTPAnonymous access, bounce attack
SSHBrute force, weak keys
SMBEternalBlue, relay attacks
DNSZone transfer, cache poisoning
SMTPOpen relay, user enumeration

Try It Yourself

Exercise 1: Protocol Identification

# Scan a target
nmap -sV 192.168.1.5

# Identify:
# - What protocols are running?
# - What versions?
# - Any known vulnerabilities? (search ExploitDB)

Exercise 2: Wireshark Analysis

  1. Start Wireshark capture
  2. Visit http://testphp.vulnweb.com
  3. Stop capture
  4. Find the HTTP GET request
  5. Can you see the full URL?
  6. Now try https://google.com - can you see the URL? (No! Encrypted)

Exercise 3: Protocol Comparison

# FTP (insecure)
echo "open ftp.example.com
user testuser
pass testpass
quit" | ftp

# SFTP (secure)
sftp [email protected]

Capture both in Wireshark. Notice how FTP credentials are in plaintext, but SFTP is encrypted.

Exercise 4: Build a Protocol Map

For your network/lab:

  1. List all devices
  2. Identify protocols each uses
  3. Mark secure vs insecure
  4. Recommend replacements for insecure protocols

Protocol Cheat Sheet

# Quick reference for common protocols

# Web
curl http://example.com          # HTTP
curl https://example.com         # HTTPS

# File Transfer
ftp example.com                  # FTP
sftp [email protected]            # SFTP

# Remote Access
ssh [email protected]             # SSH
telnet example.com               # Telnet (don't use!)

# Email
telnet mail.example.com 25       # SMTP
telnet mail.example.com 110      # POP3
telnet mail.example.com 143      # IMAP

# DNS
nslookup google.com              # Query DNS
dig google.com                   # Detailed DNS

# Network
ping google.com                  # ICMP
traceroute google.com            # Routing path
arp -a                           # ARP cache

# Services
snmpwalk -v2c -c public 192.168.1.1   # SNMP
ldapsearch -h 192.168.1.5 -x          # LDAP

Resources

Tools:

  • Wireshark: Protocol analyzer - wireshark.org
  • tcpdump: Command-line packet capture
  • nmap: Service detection
  • Burp Suite: HTTP/HTTPS analysis

Reading:

Practice:

  • Capture your own traffic with Wireshark
  • Compare secure vs insecure protocol versions
  • Try exploiting common protocol vulnerabilities in a lab

Next Steps

  1. Master Wireshark: Analyze different protocols
  2. Security audit: Identify insecure protocols on your network
  3. Learn the three-way handshake: Deep dive into TCP
  4. Explore DNS: Understand zone transfers and enumeration
  5. Practice exploitation: Use vulnerable labs (DVWA, Metasploitable)

Understanding protocols is fundamental to cybersecurity. Every attack you'll perform or defend against involves manipulating, exploiting, or protecting these communication channels!


"In God we trust, all others must send packets" - Understanding protocols is understanding how the internet works.