A high-performance Python tool for large-scale subdomain enumeration using the SecurityTrails API. Optimized for processing thousands of domains with intelligent rate limiting, auto-scaling concurrency, and resume capabilities.

• Large Scale Support: Process 2000+ domains efficiently
• Smart API Usage: Only uses scroll API when limit_reached=True (following SecurityTrails best practices)
• Auto-Scaling Concurrency: Automatically adjusts concurrent requests (2-8) based on performance metrics
• Professional Plan Optimized: Optimized for SecurityTrails Professional plans with higher rate limits
• Resume Capability: Interrupt and resume large jobs without losing progress
• Memory Efficient: Streaming output to prevent memory exhaustion on large datasets
• Real-time Progress: Live progress tracking with ETA and statistics
• Rate Limit Compliant: Respects SecurityTrails 5 requests/second limit
• Robust Error Handling: Comprehensive retry logic and error recovery
• Real Firefox User Agent: Uses authentic browser user agent for better compatibility

• Python 3.7+
• SecurityTrails API key (Free or Professional plan)
• Required packages: aiohttp, asyncio (built-in)

1. Clone the repository:

git clone https://github.com/Nishantbhagat57/STdomains.git
cd STdomains

2. Install dependencies:

pip3 install aiohttp

3. Make executable:

chmod +x st_domains.py

Get your SecurityTrails API key from SecurityTrails Dashboard:

• Free Plan: 50 requests/month, 2000 subdomains per request limit
• Professional Plan: 20,000 requests/month, full scroll API access

# Single domain
python3 st_domains.py "example.com" -k "your_api_key_here"

# Single domain with output file
python3 st_domains.py "example.com" -k "your_api_key" -o subdomains.txt

# Multiple domains from file
python3 st_domains.py domains.txt -k "your_api_key" -o results.txt

# Large scale with debug output
python3 st_domains.py domains_2000.txt -k "your_api_key" -o results.txt --debug

# Resume interrupted job
python3 st_domains.py domains.txt -k "your_api_key" -o results.txt --resume st_progress.pkl

# Custom resume file location
python3 st_domains.py domains.txt -k "your_api_key" --resume /path/to/custom_progress.pkl

Single Domain:

python3 st_domains.py "starbucks.com" -k "your_api_key"
python3 st_domains.py "xyz.starbucks.com.au" -k "your_api_key"

Domain File (domains.txt):

starbucks.com
amazon.com
accounts.starbucks.com.au
api.example.com
more-domains.com

[*] Checking API usage...
[✓] API Usage: 1547/20000 requests (7.7%)
[*] Processing 3 domains (total: 3)
[*] Output: results.txt
[*] Processing batch 1/1 (concurrency: 5)
[*] Processing: example.com
[✓] example.com: 1,247 subdomains
[*] Processing: large-corp.com
[*] large-corp.com: Limit reached, using scroll API...
[✓] large-corp.com: 156 + 2,891 = 3,047 subdomains (scroll)
[*] Increased concurrency to 6
[✓] Enumeration completed!
[✓] Domains processed: 3
[✓] Total subdomains found: 6,523
[✓] Results saved to: results.txt

• Small Domains (< 2K subdomains): ~2-5 seconds per domain
• Large Domains (scroll API needed): ~30-120 seconds per domain
• Rate Limiting: 4.8 requests/second (within SecurityTrails limits)
• Concurrency: Auto-scales from 2-8 concurrent domains
• Memory Usage: < 50MB for 2000+ domain lists

For processing 1000+ domains:

1. Use Professional Plan: Free plan limited to 50 requests/month
2. Enable Resume: Use --resume flag for long-running jobs
3. Monitor Progress: Use --debug for detailed progress tracking
4. Output to File: Always use -o flag for large datasets

1. Initial Request: Uses /v1/domain/{domain}/subdomains endpoint
2. Limit Detection: Checks meta.limit_reached in response
3. Scroll API: Only when needed (limit_reached=True)
4. Pagination: Handles scroll API pagination automatically

• Conservative Approach: 4.8 requests/second (under 5/sec limit)
• Simple Implementation: 200ms delay between requests
• Retry Logic: Exponential backoff for 429 errors
• Auto-adjustment: Reduces rate on persistent errors

• Auto-scaling: Starts at 5 concurrent domains
• Performance Monitoring: Tracks success rate and response times
• Dynamic Adjustment: Increases/decreases based on performance
• Range: 2-8 concurrent domains maximum

The script automatically saves progress and can resume interrupted jobs:

# If interrupted, simply run the same command again
python3 st_domains.py domains.txt -k "your_api_key" -o results.txt

# Or specify custom resume file
python3 st_domains.py domains.txt -k "your_api_key" --resume my_progress.pkl

Resume File Contains:

• Completed domains list
• Failed domains with error reasons
• Total subdomains found
• Processing statistics

Script Freezes:

# Run with debug to see detailed progress
python3 st_domains.py "domain.com" -k "your_key" --debug

Rate Limit Errors:

• Automatic retry with exponential backoff
• Reduces concurrency if persistent errors
• Check API usage with account/usage endpoint

Memory Issues:

• Uses streaming output (no memory buildup)
• Processes domains in batches
• Resume functionality prevents data loss

API Key Issues:

# Verify API key with single domain test
python3 st_domains.py "google.com" -k "your_key" --debug

The script automatically checks your API usage:

[✓] API Usage: 1547/20000 requests (7.7%)

Usage Tips:

• Monitor usage with --debug flag
• Professional plans have 20,000+ monthly requests
• Scroll API requests count as additional requests
• Large domains can use 50-200+ requests each

Contributions are welcome! Please feel free to submit a Pull Request.

This project is licensed under the MIT License - see the LICENSE file for details.

This tool is for legitimate security research and bug bounty activities only. Users are responsible for complying with:

• SecurityTrails Terms of Service
• Target domain policies
• Applicable laws and regulations
• Rate limiting and API usage guidelines

• SecurityTrails API Documentation
• Subfinder
• Amass

• Issues: GitHub Issues
• SecurityTrails API: SecurityTrails Support
• Documentation: SecurityTrails Docs

⭐ If this tool helped you, please consider starring the repository!