Managing billions of records requires a specific approach to data architecture. To ensure that Hevar.co.uk assets are compatible with enterprise-grade tools and high-performance scripts, we follow a strict “Raw-First” delivery model.
1. The Format: Single-Column Plaintext
All our domain and subdomain lists are provided as single-column files (one record per line).
- Zero Overhead: By avoiding complex CSV headers or JSON structures, we reduce file sizes and ensure compatibility with legacy command-line tools like
grep,awk, andsed. - Universal Compatibility: Our
.txtand.csvfiles can be imported directly into any major database (PostgreSQL, BigQuery, Snowflake) without the need for custom ingestion scripts. - Encoding: Delivered in UTF-8 for maximum reliability across Windows, Linux, and macOS environments.
2. Optimized for High-Speed Recons
Our lists are designed specifically for:
- Security Researchers: Direct input for tools like Amass, Subfinder, and MassDNS.
- Data Scientists: Clean “seed” data for NLP, pattern recognition, and machine learning.
- SEO Professionals: Lightweight lists for large-scale backlink and TLD analysis.
3. Large-Scale Delivery Strategy
Because a list of 2.9 billion subdomains can exceed 150GB, we do not deliver via standard browser downloads which often time-out.
- Secure Cloud Storage: After purchase, you receive a private, high-speed link to an encrypted cloud bucket (AWS S3/Google Cloud).
- Segmented Volumes: For our largest datasets, files are provided in compressed, multi-part volumes (e.g., Part 1, Part 2) to ensure stable downloads even on standard internet connections.
Technical Advice: We do not recommend opening these files in standard text editors (Notepad, TextEdit). For the best experience, use high-performance database loaders or command-line utilities.