DataSploit · upgoingstar · Nov 20, 2025 · Sep 22, 2025
diff --git a/.gitignore b/.gitignore
@@ -3,8 +3,8 @@ __pycache__/
 *.py[cod]
 *$py.class
 
-config.py
 /config.py
+/config.ini
 facebook_user_details.py
 generate_passwords.py
 git_searcher.py
@@ -84,3 +84,6 @@ target/
 
 profile_pic/
 reports/json/
+
+.DS_Store
+remove/
diff --git a/.vscode/launch.json b/.vscode/launch.json
@@ -0,0 +1,16 @@
+{
+  // Use IntelliSense to learn about possible attributes.
+  // Hover to view descriptions of existing attributes.
+  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+  "version": "0.2.0",
+  "configurations": [
+    {
+      "name": "Python Debugger: Current File with Arguments",
+      "type": "debugpy",
+      "request": "launch",
+      "program": "${file}",
+      "console": "integratedTerminal",
+      "args": "${command:pickArgs}"
+    }
+  ]
+}
diff --git a/AGENTS.md b/AGENTS.md
@@ -0,0 +1,19 @@
+# Repository Guidelines
+
+## Project Structure & Module Organization
+`datasploit.py` is the single entry point. It classifies the target (domain/email/ip/username) and delegates to the shared runner in `core/`, which auto-discovers collectors under `domain/`, `emails/`, `ip/`, and `username/`. Each collector lives in the matching package as `<category>_<source>.py` and defines `ENABLED`, `MODULE_NAME`, and `REQUIRES`. Keep shared settings in `config.ini` or `datasploit_config.py`, contributor docs in `docs/`, and generated output under `reports/`. Cached avatars and similar assets belong in `profile_pic/`.
+
+## Build, Test, and Development Commands
+Target Python 3.12 as declared in `Pipfile`. Preferred workflow: `pipenv install` to resolve dependencies, then `pipenv shell` to run tools. Virtualenv is also fine: `python3 -m venv .venv && source .venv/bin/activate` followed by `pip install -r requirements.txt`. Exercise scripts before opening a pull request, e.g. `python datasploit.py -i example.com` for the full domain sweep or `python domain/domain_subdomains.py example.com` for a single collector. Append `-o text` when validating report writers.
+
+## Coding Style & Naming Conventions
+Follow PEP 8 with four-space indentation, lowercase module names, and descriptive function names. New collectors must live in the matching module folder and keep the `<module>_<source>.py` prefix so discovery stays consistent. Implement the `banner`, `main`, and `output` trio from the templates; let `main` return data and gate unfinished work with the `ENABLED` flag. Secrets stay outside the codebase—use config files, `.env`, or runtime variables.
+
+## Testing Guidelines
+There is no automated suite yet, so rely on targeted manual verification. Run the collector directly to confirm input handling, then trigger the orchestrator (`python datasploit.py -i test.tld`) to confirm integration. Inspect files created in `reports/` and keep console output readable. Document the targets and commands you used in the pull request so reviewers can repeat them.
+
+## Commit & Pull Request Guidelines
+Write concise, imperative commit summaries that mirror the existing history (`domain: scan domain with urlscan.io`). Squash noisy work-in-progress commits before review. Pull requests should explain the motivation, list configuration or dependency changes, and note verification steps or artifacts. Link to issues when relevant, and include screenshots only when they meaningfully illustrate UI or report changes.
+
+## Configuration & Security Notes
+Mirror changes between `config.template.ini` and `config.ini`, and flag new options in the docs. Never commit API tokens or target data; load them via environment variables or local config files. Review third-party integrations for rate limits and legal considerations, and highlight modules that may require elevated or commercial access.
diff --git a/CHANGES.md b/CHANGES.md
@@ -0,0 +1,49 @@
+# Changelog
+
+## Modernisation (2025-03-19)
+
+**Core architecture**
+- Introduced a `core` package with shared orchestration helpers:
+  - `core.collector.CollectorModule` wraps existing collectors with metadata and text-report support.
+  - `core.registry.CollectorRegistry` performs auto-discovery per target category and ensures the project root is on `sys.path` before imports.
+  - `core.runner.CollectorRunner` handles prerequisite checks, banner execution, and optional text report writing.
+  - `core.style` centralises ANSI styling; collectors import `style` instead of defining local classes.
+  - `core.input_classifier` offers unified target classification (`domain`, `email`, `ip`, `username`) and private-IP detection.
+  - `core.config` loads values from `config.ini` (falling back to legacy `config.py`) once and exposes `get_config_value` for collectors.
+- `datasploit.py` now uses the classifier + runner and skips private IPs automatically.
+- Legacy entry points (`domainOsint.py`, `emailOsint.py`, `ipOsint.py`, `usernameOsint.py`) have been removed in favour of a single `datasploit.py` entrypoint. Older workflows should switch to `python datasploit.py -i <target>`.
+
+**Collector requirements**
+- Every collector module must define:
+  - `ENABLED`: boolean flag to toggle execution.
+  - `MODULE_NAME`: user-facing name (derived from filename when omitted).
+  - `REQUIRES`: tuple of config keys required before execution (empty tuple if none).
+- `banner()` implementations now return a plain string; the runner handles consistent formatting via the shared `style` helper.
+- Modules missing `MODULE_NAME` or `REQUIRES` are skipped with an explanatory warning.
+- Many collectors now declare prerequisites (GitHub tokens, Shodan API key, etc.) so the runner can skip them gracefully when credentials are absent.
+
+**Configuration access**
+- `vault.py` is deprecated; calling `vault.get_key` raises an error instructing authors to use `core.config.get_config_value`.
+- All collectors use `get_config_value` instead of rolling their own config parsing.
+- The user-facing configuration moved to `config.ini` to align with a familiar INI-style template and avoid name clashes with `core/config.py`.
+
+**Package cleanup**
+- Module packages now auto-import their collectors so library users can access them via attributes like `datasploit.username.username_gitscrape`.
+- Removed the `base.py` shim from all module packages (`domain`, `emails`, `ip`, `username`). Discovery now relies on the registry’s path injection.
+ - Package `__init__` files perform lightweight auto-imports instead of remaining empty stubs, preserving backwards-compatible attribute access.
+- Documentation (`docs/Writing_Modules.md`) updated to reflect the new structure—no base helper or auto-import glue; collectors live directly in the package with the required metadata.
+
+**Dependency updates**
+- `requirements.txt` and `Pipfile` list the exact dependencies used at runtime; obsolete Python 2 packages dropped, and version floors set for Python 3 compatibility.
+- Optional integrations (e.g., `emails/email_hacked_emails.py`) now handle missing optional dependencies gracefully (`cfscrape`).
+
+**Contributor guidance**
+- When adding a new collector:
+  1. Place it under the appropriate package (`domain/`, `emails/`, `ip/`, `username/`).
+  2. Name the file `category_source.py` (e.g., `domain_newservice.py`).
+  3. Define `ENABLED`, `MODULE_NAME`, `REQUIRES`, and optionally `WRITE_TEXT_FILE` and `DESCRIPTION`.
+  4. Implement `banner()`, `main(target)`, `output(data, target)`; `output_text(data)` if you set `WRITE_TEXT_FILE = True`.
+  5. The shared runner auto-discovers it—no manual registry changes needed.
+- `config.ini` should contain any new keys referenced in `REQUIRES`; users populate them with their credentials.
+
+Refer to `core/collector.py` and `docs/Writing_Modules.md` for detailed examples.
diff --git a/Pipfile b/Pipfile
@@ -0,0 +1,25 @@
+[[source]]
+url = "https://pypi.org/simple"
+verify_ssl = true
+name = "pypi"
+
+[packages]
+beautifulsoup4 = ">=4.12.2"
+cfscrape = ">=2.1.1"
+configobj = ">=5.0.8"
+dnspython = ">=2.4.2"
+email-validator = ">=2.1.0.post1"
+ipwhois = ">=1.2.0"
+lxml = ">=4.9.3"
+python-Wappalyzer = ">=0.3.1"
+python-whois = ">=0.8.0"
+requests = ">=2.31.0"
+termcolor = ">=2.3.0"
+tldextract = ">=3.4.0"
+tweepy = ">=4.14.0"
+validators = ">=0.22.0"
+
+[dev-packages]
+
+[requires]
+python_version = "3.12"