A command-line interface (CLI) chatbot for FAQs, leveraging Azure OpenAI's embedding and chat models. This project is designed to provide a simple and effective way to interact with FAQs using natural language processing.

This project is built with Python and can be run in a Docker container or as a standalone Python application. It supports Langfuse for observability and tracing.

I chose to use Azure OpenAI because it provides a powerful and scalable solution for natural language processing tasks, including embeddings and chat models, while ensuring compliance with data residency requirements.

Selection to ensure compliance with data residency requirements and gdpr in the european union:

• Resource Region: swedish central (westeurope is also possible)
• Deployment Type: Standard deployment type

With deploying the resources in the swedish central region together with selecting the Standard deployment type, I can ensure compliance with data residency requirements for sensitive information and gdpr. Data stored at rest and also data processed for inference will not leave the selected region.

• Embedding Model: text-embedding-ada-002
• Chat Model: gpt-35-turbo

• The text-embedding-ada-002 model is efficient for generating embeddings, cost-effective, and provides good performance for similarity search tasks. Also text-embedding-3 models seems to have issues with FAISS similarity search with Langchain.
• The gpt-35-turbo model is a powerful chat model that provides high-quality responses for simple FAQ tasks. It is cost-effective and widely used in production applications.

• Python 3.10 or higher
• Docker and Docker Compose
• Azure Embedding Model deployed
• Azure Chat Model deployed
• Azure OpenAI Endpoint and Key

git clone [email protected]:MSWagner/faq-cli-chatbot.git
cd faq-cli-chatbot

Copy the .env.example to .env and fill in the required environment variables:

cp .env.example .env

• Ensure you have the following variables set in your .env file:

  • AZURE_OPENAI_ENDPOINT
  • AZURE_OPENAI_KEY
  • EMBEDDING_MODEL
  • EMBEDDING_API_VERSION
  • CHAT_MODEL
  • CHAT_API_VERSION

• (Optional) variables for Langfuse integration (local setup described in the next sections):

  • LANGFUSE_SECRET_KEY
  • LANGFUSE_PUBLIC_KEY
  • LANGFUSE_HOST (if you want to use a remote Langfuse instance or start the chatbot outside of a docker container)
  • LANGFUSE_SYSTEM_PROMPT (if you want to use a system prompt from Langfuse)

Hints:

• use AZURE_OPENAI_ENDPOINT in the format https://<your-resource-name>.openai.azure.com/
• you can find the model names and versions in the Azure AI Foundary Studio under "Deployments".

docker compose run --rm faq-chatbot

Hint:

• If you want to run the chatbot with remote Langfuse, ensure you have the Langfuse environment variables set in your .env file.

Build the Docker container with Langfuse:

docker compose -f 'docker-compose.langfuse.yml' up -d --build

• Create a new User (Signup) and create a new Organisation and Project.
• Create an API Key in the Langfuse WebUI and copy the LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY to your .env file.
• (Optional) Create a System Prompt in Langfuse and copy the LANGFUSE_SYSTEM_PROMPT to your .env file.

source .env
docker compose -f 'docker-compose.langfuse.yml' up -d --build

docker compose -f 'docker-compose.langfuse.yml' run --rm faq-chatbot

Create a virtual environment to isolate dependencies:

python -m venv .venv
source .venv/bin/activate  # For Windows: .venv\Scripts\activate

pip install -e .  # Installs the project in editable mode, reading from pyproject.toml

python -m faqbot.main

# Force rebuild embeddings
python -m faqbot.main -f

# Enable debug mode
python -m faqbot.main -d

# Reload .env variables
python -m faqbot.main -r

python -m pytest --rich

This project is licensed under the MIT License. See the LICENSE file for details.