Passive Reconnaissance: crt.sh (Certificate Transparency)

Certificate Transparency (CT) is a system that logs all SSL/TLS certificates issued by Certificate Authorities. The website crt.sh provides a searchable interface to these logs, making it an invaluable tool for passive reconnaissance. By searching CT logs, penetration testers can discover subdomains, infrastructure, and services without ever touching the target's systems.

What You'll Learn

What Certificate Transparency is and why it exists
How to use crt.sh for subdomain enumeration
Understanding SSL/TLS certificate information
Automating CT log searches
Finding hidden infrastructure and services
Combining CT logs with other reconnaissance
Using CT logs for asset discovery
Privacy implications of CT logs

What Is Certificate Transparency?

Certificate Transparency is an open framework for monitoring and auditing SSL/TLS certificates in real-time.

The Problem CT Solves:

Before CT:
1. CA issues certificate
2. Only CA and domain owner know
3. Malicious/fraudulent certs can exist undetected
4. No public record

After CT (2013+):
1. CA issues certificate
2. Certificate logged to public CT logs
3. Anyone can search and monitor
4. Detect fraudulent certificates
5. Accountability for CAs

Why This Matters for Recon:

Every subdomain that gets an SSL certificate:
→ Gets logged to CT
→ Becomes publicly searchable
→ Reveals infrastructure

Even if subdomain is:
- Not linked anywhere
- Not in DNS
- "Hidden"
→ CT logs expose it!

What Is crt.sh?

crt.sh is a web interface to search Certificate Transparency logs.

Access:

Website: https://crt.sh
Free, no registration required
Searches multiple CT logs

What You Can Find:

✓ Subdomains (most common use)
✓ Certificate details
✓ Certificate issuers
✓ Validity periods
✓ SANs (Subject Alternative Names)
✓ Wildcard certificates
✓ Historical certificates
✓ Related domains

Using crt.sh for Subdomain Enumeration

Basic Search:

1. Go to https://crt.sh
2. Enter domain: example.com
3. Click "Search"
4. View all certificates for that domain

Search Operators:

Exact domain:

example.com

Include subdomains:

%.example.com

% = wildcard (like * in other tools)

Specific subdomain:

mail.example.com

Understanding the Results

Example Output:

Logged At              | crt.sh ID | Issuer         | Not Before | Not After  | Subject
──────────────────────────────────────────────────────────────────────────────────────
2024-01-15 14:23:45   | 12345678  | Let's Encrypt  | 2024-01-15 | 2024-04-15 | CN=example.com
2024-01-15 14:23:45   | 12345678  |                |            |            | DNS:example.com
                       |           |                |            |            | DNS:www.example.com
                       |           |                |            |            | DNS:mail.example.com
                       |           |                |            |            | DNS:api.example.com

Key Fields:

Logged At:

When certificate was added to CT log
Recent = currently in use
Old = historical (might be deprecated)

Issuer:

Certificate Authority that issued cert
Common: Let's Encrypt, DigiCert, Cloudflare

Subject & SANs:

Domain names covered by certificate
SANs (Subject Alternative Names):
- Multiple domains on one cert
- Reveals related subdomains

Validity Period:

Not Before: Certificate valid from
Not After: Certificate expires

Expired certs:
- Might indicate old infrastructure
- Services may be decommissioned

Command-Line CT Log Queries

Using curl:

# JSON output
curl -s "https://crt.sh/?q=%.example.com&output=json" | jq .

# Extract just domain names
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u

# Save to file
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u > subdomains.txt

Parsing and Cleaning:

# Remove wildcards and duplicates
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sed 's/\*\.//g' | \
  sort -u | \
  grep -v "@"  # Remove email addresses if any

One-Liner for Quick Recon:

curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u | \
  tee subdomains.txt

Automated CT Log Enumeration

Python Script:

#!/usr/bin/env python3
import requests
import json

def get_subdomains(domain):
    """Fetch subdomains from crt.sh"""
    url = f"https://crt.sh/?q=%.{domain}&output=json"
    
    try:
        response = requests.get(url, timeout=30)
        data = json.loads(response.text)
        
        subdomains = set()
        for cert in data:
            # Extract common_name
            name = cert.get('name_value', '')
            # Split by newlines (SANs on separate lines)
            for subdomain in name.split('\n'):
                # Remove wildcards
                subdomain = subdomain.replace('*.', '')
                # Remove leading/trailing whitespace
                subdomain = subdomain.strip()
                if subdomain and '@' not in subdomain:
                    subdomains.add(subdomain)
        
        return sorted(subdomains)
    
    except Exception as e:
        print(f"Error: {e}")
        return []

if __name__ == "__main__":
    domain = "example.com"
    print(f"[*] Searching CT logs for: {domain}")
    
    subdomains = get_subdomains(domain)
    
    print(f"\n[+] Found {len(subdomains)} subdomains:\n")
    for subdomain in subdomains:
        print(subdomain)

Usage:

python3 crt_search.py
chmod +x crt_search.py
./crt_search.py

What Can You Discover?

1. Subdomains

Common findings:
- mail.example.com (email server)
- vpn.example.com (VPN portal)
- dev.example.com (development)
- staging.example.com (staging environment)
- api.example.com (API endpoints)
- admin.example.com (admin panel)
- internal.example.com (internal tools)

2. Infrastructure

Hosting providers:
- aws-ec2-1.example.com (AWS)
- azure-vm-prod.example.com (Azure)
- gcp-instance.example.com (Google Cloud)

Load balancers:
- lb1.example.com
- lb2.example.com

CDN:
- cdn.example.com
- static.example.com

3. Development/Staging

Non-production environments:
- dev.example.com
- test.example.com
- staging.example.com
- qa.example.com
- uat.example.com

Often less secure!
Good targets for testing

4. Regional/Geographic

- us.example.com
- eu.example.com
- uk.example.com
- asia.example.com

Reveals global infrastructure

5. Services

- webmail.example.com
- ftp.example.com
- ssh.example.com
- remote.example.com
- citrix.example.com

Advanced CT Log Techniques

1. Wildcard Certificate Analysis

If you find: *.example.com
This means:
- Organization uses wildcard cert
- Potentially many subdomains
- Other recon methods needed to find them

But:
- CT logs may show specific certs too
- Keep searching for non-wildcard certs

2. Historical Analysis

crt.sh shows ALL certificates:
- Current
- Expired
- Revoked

Old certificates reveal:
- Former subdomains
- Deprecated services
- Infrastructure changes
- Mergers/acquisitions

3. Certificate Issuer Patterns

Let's Encrypt:
- Free, automated
- Short validity (90 days)
- Frequent renewals
- Modern infrastructure

Paid CAs (DigiCert, etc.):
- Longer validity
- Organizational validation
- Higher assurance

Self-signed:
- Internal services
- Development/testing
- Security concern if in production

4. Subject Alternative Names (SANs)

One certificate can cover multiple domains:

Certificate for: example.com
SANs:
- www.example.com
- mail.example.com
- api.example.com
- shop.example.com

All revealed in one CT log entry!

Combining with Other Tools

Workflow:

1. crt.sh: Find subdomains
   ↓
2. Verify they resolve:
   ↓
   host subdomain.example.com
   ↓
3. Port scan live hosts:
   ↓
   nmap -sV subdomain.example.com
   ↓
4. Further enumeration:
   ↓
   nikto, gobuster, etc.

Example:

# Step 1: Get subdomains from crt.sh
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u > all_subdomains.txt

# Step 2: Check which resolve
while read subdomain; do
  if host "$subdomain" > /dev/null 2>&1; then
    echo "$subdomain" >> live_subdomains.txt
  fi
done < all_subdomains.txt

# Step 3: Scan live subdomains
nmap -iL live_subdomains.txt -oA scan_results

Other CT Log Tools

subfinder:

# Install
go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest

# Use (includes CT logs)
subfinder -d example.com

# Output to file
subfinder -d example.com -o subdomains.txt

Amass:

# Install
go install -v github.com/owasp-amass/amass/v4/...@master

# Use (includes CT logs and much more)
amass enum -d example.com

# Passive only (including CT)
amass enum -passive -d example.com

ct-exposer:

# Python tool specifically for CT logs
git clone https://github.com/chris408/ct-exposer.git
cd ct-exposer
pip3 install -r requirements.txt

python3 ct-exposer.py -d example.com

Try It Yourself

Exercise 1: Basic crt.sh Search

1. Go to https://crt.sh
2. Search: %.tesla.com
3. Browse results
4. Identify:
   - How many subdomains?
   - Any interesting ones? (dev, staging, internal?)
   - Certificate issuers?
   - Wildcard certs?

Exercise 2: Command-Line Search

# Search a domain (use a company you're authorized to test)
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u

# How many unique subdomains?
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u | \
  wc -l

Exercise 3: Verify Resolution

# Get subdomains
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u > subs.txt

# Check which resolve
while read sub; do
  ip=$(host "$sub" 2>/dev/null | grep "has address" | awk '{print $4}')
  if [ -n "$ip" ]; then
    echo "$sub -> $ip"
  fi
done < subs.txt

Exercise 4: Compare with Other Methods

# CT logs
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u > ct_subs.txt

# DNS brute force (smaller wordlist for testing)
dnsrecon -d example.com -t brt -D small_wordlist.txt -o dns_subs.txt

# Compare unique findings
comm -3 <(sort ct_subs.txt) <(sort dns_subs.txt)

Limitations of CT Logs

What CT Logs Don't Show:

✗ Subdomains without SSL certificates
✗ Internal-only domains (no public cert)
✗ Subdomains using wildcard certs (only)
✗ Services on non-443 ports (without HTTPS)
✗ Recently created subdomains (lag time)

Solutions:

Combine with:
1. DNS brute forcing
2. Google dorking (site:example.com)
3. Shodan/Censys searches
4. Zone transfers (if allowed)
5. Web scraping/spidering
6. API enumeration

Privacy and Security Implications

For Organizations:

CT logs expose:
✓ All public SSL certificates
✓ Subdomain structure
✓ Infrastructure details
✓ Development environments
✓ Third-party services

Mitigation:
- Assume all cert info is public
- Don't use descriptive subdomain names
- Secure all hosts (even "hidden" ones)
- Monitor CT logs for your own domain

For Pentesters:

✓ Completely passive
✓ Legal (public data)
✓ No interaction with target
✓ Fast reconnaissance
✓ Historical data available

Perfect for initial recon phase!

Quick Reference

# Web interface
https://crt.sh/?q=%.example.com

# JSON API
curl -s "https://crt.sh/?q=%.example.com&output=json"

# Extract subdomains
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  jq -r '.[].name_value' | \
  sort -u

# Remove wildcards
sed 's/\*\.//g'

# Filter valid domains
grep -E '^[a-zA-Z0-9.-]+$'

Resources

Tools:

crt.sh: Web interface to CT logs
subfinder: Automated subdomain discovery (includes CT)
Amass: Comprehensive recon (includes CT)
ct-exposer: CT log specific tool

Other CT Log Interfaces:

Censys.io: Search engine including CT logs
Facebook CT Monitor: Monitor your domains
Google CT Search: https://transparencyreport.google.com/https/certificates

Reading:

Practice:

Search major companies on crt.sh
Automate CT log queries
Combine with other recon methods
Build a subdomain monitoring tool

Next Steps

Master crt.sh queries for subdomain enumeration
Automate CT log searches with scripts
Verify discovered subdomains are live
Combine with DNS enumeration for comprehensive results
Monitor CT logs for defensive purposes

Certificate Transparency logs are a goldmine for passive reconnaissance. Use them to map attack surfaces without touching target systems!

"crt.sh: Where every SSL certificate tells a story about your infrastructure."