Run the following commands with Python 3.12 to create a virtual environment that contains all required dependencies:

python -m venv .venv # Create new virtual environment named .venv (or whatever you wanna call it)

# activate virtual environment depending on your platform #
.venv\Scripts\activate.bat # Windows, cmd.exe
.venv\Scripts\Activate.ps1 # Windows, Powershell
source .venv/bin/activate  # Linux/MacOS
# ------------------------------------------------------- #

pip install -r requirements.txt

Configure the following environment variables: AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_ENDPOINT_EMBEDDING_LARGE

The system was tested with 430 tabular and geospatial datasets of the city of Zurich, but could also be used with other types of data (with some additional steps). While the associated metadata is already contained in this repository, you need to perform some additional steps for getting the actual data.

1. Download the datasets here. Unfortunately, Zurich's open data platform does not offer an API for downloading datasets, meaning these datasets were manually exported. Note that this was performed in March and May of 2025, meaning some datasets may be out-of-date. All extraction timestamp, dataset titles and filenames are listed in data\opendata\50000006\downloads.csv, feel free to replace datasets with more recent versions.
2. Extract the ZIP folder into the directory data\opendata\50000006\extracted (make sure it is not further nested).
3. If you added the data correctly, you should see the following messages at startup:

[info     ] Will use 430 files for group owner 50000006.
[info     ] 430/548 metadata embeddings remaining after filtering for existing data.

By default, OGD4All uses OpenAI's text-embedding-3-large for semantic search. Corresponding embeddings were generated for the metadata of all datasets, and is stored in data\metadata\50000006\processed_metadata_embeddings.csv. You have two options:

• Keep text-embedding-3-large: You'll need to set either the environment variable OPENAI_API_KEY (with your OpenAI API key) or both AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT_EMBEDDING_LARGE (with your Azure OpenAI API key and endpoint).
• Use custom embeddings: First, update the processed_metadata_embeddings.csv file by modifying the scraping\generate_embeddings.py script with your own method. Then, update the get_embedding function in retrieval\retriever.py.

Support for LLMs provided by Azure OpenAI, Azure, OpenAI, OpenRouter, and Ollama is implemented. Depending on which one you use, set the environment variable AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, AZURE_API_KEY, OPENAI_API_KEY, or OPENROUTER_API_KEY.

If your desired model is not listed in SUPPORTED_LLMS in utils.py, just add it to the list and adapt the get_llm_client function.