Intelligent IOC Extractor 🛡️

A powerful, AI-driven application designed to extract, analyze, and contextualize Indicators of Compromise (IOCs) from various intelligence sources. Built with a futuristic Cyberpunk interface, this tool streamlines the workflow for threat intelligence analysts.

🚀 Key Features

• Multi-Source Ingestion:

  • URL Scan: Automatically scrapes and analyzes content from live websites using Selenium.
  • Plain Text: Process raw text pasted from emails, chats, or scratchpads.
  • PDF Analysis: Extract and analyze IOCs directly from uploaded PDF threat reports.

• AI-Powered Contextualization:

  • Leverages Google Gemini (1.5/2.5 Flash) to analyze extracted artifacts.
  • Determines if an artifact is a true IOC.
  • Provides a confidence score, severity classification, and detailed reasoning for each verdict.
  • Identifies the logical "Advisory Date" for the intelligence.

• Advanced Extraction Engine:

  • Regex-based pattern matching for a wide range of entities:
    • Network: IPv4, IPv6, Domains, URLs, Ports.
    • File: MD5, SHA1, SHA256 hashes.
    • Vulnerability: CVE IDs.
    • Other: Email addresses, Registry/File paths.
  • Auto-Deobfuscation: Handles "defanged" IOCs automatically (e.g., converting hxxp[:]// to http://).

• Reporting & Export:

  • Interactive data table with filtering (Verdict, Type).
  • One-click CSV and JSON export.

🛠️ Technology Stack

• Frontend: Streamlit (with custom CSS themes).
• Backend: Python 3.12+.
• AI/LLM: Google Generative AI (Gemini).
• Web Scraping: Selenium (Headless Chrome) & BeautifulSoup4.
• PDF Processing: pypdf.

📋 Prerequisites

• Python 3.10+ installed.
• Google AI Studio API Key: Get one here.
• Chrome Browser: Required for Selenium web scraping (Generic/Local driver setup included).

⚙️ Installation

1. Clone the repository:

   git clone <repository-url>
   cd ContextAwareIOC

2. Create and activate a virtual environment:

   python -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install dependencies:

   pip install -r requirements.txt

4. Configuration: Create a .env file in the root directory and add your Google Gemini API key:

   GEMMA_API_KEY=your_actual_api_key_here
   SUMMARIZE_MODEL=gemini-1.5-flash  # Optional, defaults to gemini-2.5-flash

🚀 Usage

Run the application using Streamlit:

streamlit run streamlit_app.py

The app will open in your default browser at http://localhost:8501.

Input Methods:

1. URL Scan: Enter a blog post, news article, or threat report URL. The system will scrape the text and analyze it.
2. Plain Text: Paste any unstructured text containing potential IOCs.
3. PDF/Files: Upload a PDF report (e.g., from CISA, FBI, or security vendors) to extract and analyze its contents.

📂 Project Structure

ContextAwareIOC/
├── app/                 # (Optional/Legacy) FastAPI structure
├── backend/             # Core Logic
│   ├── config.py        # Environment configuration
│   ├── ioc_extractor.py # Regex patterns and extraction logic
│   └── rlm.py           # Main service: Validation, Scraping, LLM Processing
├── drivers/             # Local Selenium drivers (if applicable)
├── requirements.txt     # Python dependencies
├── streamlit_app.py     # Main Streamlit Application (Frontend)
└── .env                 # Environment variables (GitIgnored)

⚠️ Disclaimer

This tool is intended for educational and defensive security purposes only. Always extract and analyze IOCs in a sandboxed or safe environment. The AI analysis is probabilistic and should be verified by human analysts before taking blocking actions.

🛡️ License

This project is licensed under the Apache License 2.0.
See the LICENSE file for details. © 2025 Mohan Kumar Manivannan