RAG-powered search assistant Accepts user questions via API Searches internal docs (Vector DB + Elastic) Optionally pulls live web content Feeds everything into a local LLM (e.g., Mistral) Returns high-quality answers

This project is a minimal Retrieval-Augmented Generation (RAG) app that:

• Accepts user queries via FastAPI
• Searches documents semantically (Vector DB) and by keyword (Elasticsearch)
• Uses a local or remote LLM to generate responses from the retrieved context

• Upload Docs: Users can upload PDF/TXT files through a simple UI or API.
  • 📥 Documents are parsed, embedded, and stored for retrieval.
• Query: Users submit a question.
  • 🧠 This query goes through vector search, keyword search, and optionally web search to gather relevant context.

• Stores document embeddings as dense vectors.
• Allows semantic search — returns results that are meaningfully similar, even if the words don't match.
• Powered by models like all-MiniLM-L6-v2.

• Indexes raw document text for exact keyword and metadata-based lookup.
• Very fast for filtering, name-based lookups, or date/tag filtering.
• Complements vector search for hybrid retrieval.

• Executes live online search (e.g., DuckDuckGo or SerpAPI).
• Pulls recent or missing information not found in local storage.
• Extracts clean page content via tools like trafilatura.

• Gathers results from vector DB, Elastic, and web search.
• Merges them into a single structured context window for the LLM.
• Ensures relevance, diversity, and context completeness.

• Processes the final prompt with embedded context.
• Can be:
  • Local using Ollama with models like mistral, phi, or tinyllama
  • Remote via OpenAI's gpt-4o or gpt-3.5-turbo
• Returns concise or multi-step answers based on query complexity.

1. User uploads docs (PDF/TXT) → stored in FAISS + Elastic.
2. User asks a question → hits API.
3. API runs:
   • Vector similarity search
   • Elastic keyword search
   • Optional web search for recent info
4. All results merged → sent as context to LLM.
5. 🧠 LLM generates final output → returned to user.

✅ Hybrid RAG uses both for best results.

System Assumptions:

• 32 GB RAM (use ~16 GB safely)
• 8 GB VRAM (unused for now)
• Goal: CPU-only text generation with ~50 token responses

You can use quantized models like Mistral, Phi, or TinyLlama.

Install once:

# Mac/Linux
curl -fsSL https://ollama.com/install.sh | sh

# Start the server
ollama run mistral

Download Ollama for Windows from their official site: 👉 https://ollama.com/download

Run the installer and launch the Ollama terminal.

Pull and run a model (e.g. mistral):

ollama run mistral

or

"C:\Users\{user_name}\AppData\Local\Programs\Ollama\ollama.exe" run mistral

ollama run mistral         # 7B, solid quality
ollama run phi             # 2.7B, fast reasoning
ollama run tinyllama       # 1.1B, ultra lightweight

docker run -d -p 11434:11434 --name ollama ollama/ollama

curl http://localhost:11434/api/generate -d '{
  "model": "mistral",
  "prompt": "What's the capital of France?",
  "stream": false
}'

from langchain_community.llms import Ollama

llm = Ollama(model="mistral", temperature=0.2)

⚙️ For short responses (~50 tokens), any of these models should run well on CPU in your 32GB system.

🧠 Example Query: "What’s the population of Germany?" If you feed this into context:

• “According to 2022 stats, Germany had a population of 83.2 million people.”

Then ask:

• User Query: “What’s the capital of Germany?"

Add More context with Elasticsearch, VectorDB match to the query and fed to the LLM model.

LLM Output:

• "The capital of Germany is Berlin. Germany, located entirely in Europe, is one of the continent's most powerful economies and has a high standard of living for its population. Its history dates back to 1871 when it was established, with significant periods of industrialization, imperial expansion, and international influence. However, the early 20th century saw Germany involved in both World Wars and the atrocities committed under the Nazi regime. After WWII, Germany lost territories and resources due to the fallout from the wars, and it was divided into East Germany (German Democratic Republic) and West Germany (Federal Republic of Germany). The country was officially reunified on October 3, 1990.\n\n In 2023, Germany has an estimated population of 84 million people, with Berlin being the largest city. Due to a low birth rate and high death rate, Germany has been experiencing a natural population decline for five decades, making migration crucial for its population growth. In 2022, it was estimated that almost 15 percent of the population is foreign.\n\n Economically, Germany had a nominal GDP of 3.9 trillion euros (4.1 trillion U.S. dollars) in 2022, making it the fourth largest economy in the world. It is a net exporter of goods and specializes in technologically advanced goods such as automobiles, machinery, electronics, and pharmaceuticals. The European Union trading bloc is its largest trading partner."