LLM Chat Agent for Jupyter

This is an LLM-powered chat interface integrated with a lot of useful tools.
Load your favorite GGUF and enjoy!
For better results, I strongly recommend you to select a model large enough or trained for tool use.

Features

1. Multi Modal.
2. Streaming output.
3. StreamingLLM (aka Context shifting).
4. Multi-step tool use.
5. Integration to Japanese text-to-speech model called style-bert-vits2.
6. Easy to integrate any User-defined tools.
7. Configurable UIs.

Built-in Tools

1. Web Search
2. Local Document Search
3. Python Executor
4. URL Fetcher
5. Shell Executor (Sandbox is available in macOS)
6. Image Recognition
7. Mouse Click
8. Screenshot

Prerequisite

1. Install the latest llama-cpp-python.
pip install llama-cpp-python
2. Jupyter must be installed.
$ pip install jupyterlab
3. Activate ipywidgets by one of following commands:

# jupyter-lab
$ jupyter labextension install @jupyter-widgets/jupyterlab-manager
# jupyter notebook / Google Colab

$ jupyter nbextension enable --py widgetsnbextension

4. Install dependencies.
$ pip install -r requirements.txt

Screen Shots

Python execution

Web search

Understading what you are seeing