This project began as a way to document my progression through the IBM AI Developer certification course. As the course progressed, I realized that the manner in which the course conducted many of its labs was not an optimal format. The labs were performed inside in-browser virtual IDEs with no way to reference your past work. Additionally, I felt that the labs could all be leveraged as opportunities to build out an entire AI application with multiple features, not just a simple one-dimensional chatbot application.

With this in mind I built on my original app.py and implemented the following:

• A custom logger that can:
  • log informational messages to the console
  • save an array of logs to a file.
    • This is with the intention of maintaining a conversation history between the user and the chatbot.
    • This directory is added to the .gitignore with the intention of keeping the user's conversation history private.
• Implement a Flask server to handle all interactions with different AI models and APIs
• Implement a React frontend
  • This initiative began in response to a lab around building a speech-to-text/text-to-speech application. I took the starter code from the lab and refactored it from a pure Javascript/HTML application that was far more readable and maintainable.

• TODO: Section about starting the web client
  • install npm
  • npm install
  • npm run start
• TODO: Section about starting the server
  • install python
  • set up python venv
  • pip install -r requirements.txt
• TODO: Section about Docker deployment

datasets.json is a configuration file designed to streamline the importation of multiple datasets at run time. At a high level, each entry takes the following structure:

{
	name: string, // Human-readable name of the dataset.
	hf_id: string, // The ID of the dataset in Hugging Face's datasets repo.
	pattern: string, // The pattern to be used to format the dataset prior to tokenization.
	columns: string[], // The dataset's column keys, used to build the prompt input along with pattern.
	config_type: "main" | "socratic", // Required second param when calling load_dataset() for GSM8K
	reference: obj // An object containing citation data for the dataset. Provided for credit and reference.
	split: "train" | "test" // Which portion of the dataset to use for building the model
}

This project uses the following datasets as its baseline training data. Its import configuration can be referenced in the file server/datasets.json.

• ByteDance-Seed/WideSearch
• datasets-examples/doc-formats-csv-1
• fka/awesome-chatgpt-prompts
• openai/gsm8k

• Cuda System Toolkit
• CUDA-compatible PyTorch
  • Including pytorch in requirements.txt by default installs the PyTorch build which is only compatible with the CPU.

• Hugging Face Kernel Hub

  • Intro wiki

Have fun!