Goal: Locally hosted chatbot answering questions to your documents
- 15 GB free space
- At least 4 GB of free memory
- Python 3.10
- docker-compose
At the ingestion stage, one or several documents are stored into the database with the corresponding semantic
embeddings. The IngestionPipeline object defined in askyourdocs.pipeline.pipeline enchains the following steps:
| Stage | Remark | Related Object(s) | |
|---|---|---|---|
| 1 | Extraction | Extracts the text from the retrieved documents | askyourdocs.storage.scraping -> TikaExtractor |
| 2 | Text Entities | Split text into text chunks | askyourdocs.modelling.llm -> TextTokenizer |
| 3 | Text Embeddings | Compute vector embeddings of the text chunks | askyourdocs.modelling.llm -> TextEmbedder |
| 4 | Storage in Solr | Add documents and embeddings to Solr | askyourdocs.storage.client -> SolrClient |
At the query stage, a given user input is transformed into a semantic vector, from which semantically related document
parts are retrieved and used for formulating an answer. The QueryPipeline object defined in
askyourdocs.pipeline.pipeline enchains the following steps:
| Stage | Remark | Related Object(s) | |
|---|---|---|---|
| 1 | Text Embeddings | Compute vector embeddings of query | askyourdocs.modelling.llm -> TextEmbedder |
| 2 | k-nearest-neighbors | Search the semantically closest text entities | askyourdocs.storage.client -> SolrClient |
| 3 | Create context | Create text context from relevant documents | - |
| 4 | Answer | Use context to form an answer to the query | askyourdocs.modelling.llm -> Summarizer |
For easy-of-use we have added the run_ayd.sh script. You only need to make it executable and run it:
source .env #only needed if you want to overwrite the default envs from docker compose
chmod +x run_ayd.sh
./run_ayd.sh
The script first creates a solr container and then for hosting the documents it mounts the volume /opt/solr into the bitnami/solr container. Make sure that the default
user in the bitnami containers (1001) has the appropriate rights by
sudo chown 1001 /opt/solr #done by the script!!! ATTENTION !!! You also want to make sure that this folder is empty when you initially start the backend container. Run the docker containers
docker compose -p ayd up -d #done by the scriptAs we are using keycloak as authentication service, we need to create a testuser with the following command:
docker exec -i ayd-postgres-1 psql -U "${AYD_PSQL_USER:-ayd_dba}" -d $"${AYD_PSQL_DB:-ayd}" -a -f /user_scripts/user_entity_data.sql #done by the scriptthen checkout localhost:3000 with "test" as username/password and see the magic happening ;-D.
!!! ATTENTION !!!
- If you plan to use askyourdocuments on real data, remove the testuser from your keycloak admin console.
- Running the docker compose will create the app for you (be patient, the backend need to download the models first so
it might take up to 10 minutes to be ready, check
docker logs ayd-backend-1 -fto see the following message:INFO: Application startup complete.)
Start by setting up the three services Apache/Tika, Solr, and ZooKeeper (you might want to get inspired by
the docker-compose.yml file).
docker run -d --name tika -p 9998:9998 apache/tika:latest
docker run -d --name zoo1 -p 2181:2181 -e ZOO_MY_ID=1 -e ZOO_SERVERS=server.1=zoo1:2888:3888;2181 -e ZOO_4LW_COMMANDS_WHITELIST=mntr,conf,ruok zookeeper
docker run -d --name solr -p 8983:8983 -v /opt/solr:/bitnami -e ZK_HOST=zoo1:2181 -e SOLR_JAVA_MEM="-Xms1g -Xmx1g"Then define the environment variables:
# Logging
export LOG_LEVEL="INFO"
# TIKA
export TIKA_URL=<changeme> # For local host use "http://localhost:9998"
# Solr
export SOLR_URL=<changeme> # For local host use "http://localhost:8983"
export ZK_URLS=<changeme> # For local host use "localhost:2181"
# Frontend
export FRONTEND_URL=<changeme> # For local host use "http://localhost:3000"
# Postgres
export AYD_PSQL_USER=<changeme>
export AYD_PSQL_PASSWORD=<changeme>
export AYD_PSQL_DB=<changeme>
# Keycloak
export KEYCLOAK_URL=<changeme> # For local deployment via uvicorn use http://localhost:8080/
export BACKEND_KEYCLOAK_SECRET=<changeme> #For initial value use bQwuuesYTIfcJmOxI4t4fltV48OQsAQq. If changed, make sure to adjust in app/keycloak/realm-export.json as well.With a venv as and an alias ayd defined as
python -m venv venv
source venv/bin/activate
pip install -r req_freeze.txt
alias ayd='python -m askyourdocs'and the necessary Solr collections ayd_docs, ayd_texts, and ayd_vecs setup through
ayd storage creation -c "ayd_docs"
ayd storage creation -c "ayd_texts"
ayd storage creation -c "ayd_vecs"you can start to play with ask-your-documents through the CLI.
ayd pipeline ingest --source "docs" --commitayd storage extract --filename <filename>
# ayd storage extraction --filename "https://www.accessdata.fda.gov/drugsatfda_docs/label/2011/020895s036lbl.pdf"<filename> is either the local path or the url of a file.
ayd storage search -c <collection> -q <query>
# ayd storage search -c "ayd_docs" -q "annual report"
# ayd storage search -c "ayd_feedback" -q "*:*"ayd modelling embedding -t <text>
# ayd modelling embedding -t "foo bar is far"ayd modelling tokenization -t <text>
# ayd modelling tokenization -t "Foo bar is far. My cat is fat"ayd pipeline query --text <your-text>
# ayd pipeline query --text "How to avoid covid" You can also choose to run the FastAPI backend and/or the React frontend locally.
From the base directory run
uvicorn app.backend.app:app --host 0.0.0.0 --port 8686 --reloadcd app/frontend # move to the frontend directory
npm install # install the dependenciesnpm run start # start the appRun type checking
mypy --ignore-missing-imports askyourdocsand unit-tests
pytest -s --cov=askyourdocs testsEnter magic schnauz in the user input field of the frontend :-D