ai-chatbot

small ai-chatbot for LF6

Tech Infra

We use Ollama to provide inference to a prefered model.
There is a docker-compose in this repo.

Setup

In the directory of the project execute: python -m venv ./venv
Then execute in the same directory: ./venv/bin/pip install torch transformers gradio openai
You also have to install a model inside of ollama which is done by going in the docker container via docker exec ollama -it /bin/bash
and then running for example ollama pull llama3.2