ai-cli.py

I spent some time improving my earlier demo and turned it into a fun little command line app ai-cli.py.

It lives in my LangChain Demos, you can read the code for it here.

You can see it in action in the gif below.

This is a simple demo showing the LLM answering the question of what is the current time, which it does by calling one of available local tools, and processing the tool response.

There is no long-term history currently so each invocation is a new conversation.

Danger

This tool can be dangerous. See full disclaimer here.

It allows the LLM to execute code, can give access to custom tools to the LLM, allows access to the internet and more.

It is totally possible to give this LLM access to your filesystem and have it remove all your files.

It also lets the LLM run in a tool calling loop so it can decide to do unexpected things sometimes like running the same tool over and over again. YMMV.

Make sure you know what you are doing!

Now go have fun you little scamps.

Some features

• Pretty prints output using rich so you get syntax highlighting and markdown support on the command line.
• Prints clear verbose output to show chain of execution.
• Has verbosity levels -v -vv -vvv -vvvv to show more or less info, llm thinking output and LangGraph / LangChain debugging.
• Can change the system prompt quickly from the command line.
• Can also print no extra output for use as an actual CLI tool.
• Can use any LLM available in ollama ls
• Can utilize MCP servers
• Can load python tool functions from the tools/ folder
• Shows examples of built-in tools
• Allows the LLM to call tools in a loop

Example Tools

There are a few different methods of adding tools shown in the example which are all used together with the LLM.

• Custom tools directly in ai-cli.py
  • Wikipedia search
  • Run python code inside a docker container
  • Get current date
  • Get current time
  • Query Documents (vector database)
• Python tools loaded from the tools folder
  • Ping
  • Run nmap scan
  • These tools are basic, but an LLM was used to create them as part of an experiment.
• Tools loaded from MCP servers
  • Any MCP server can be added with a simple standard .json config file

LLM -> Tool Loop

ai-cli.py runs an interesting loop with LangChain. It makes tools available to the LLM, which can then decide to call them. If tool calls happen the responses are handed back to the LLM which can then AGAIN decide to call tools. This enables the LLM to call tools multiple times when it sees fit, or call tools in a specific order. In testing however YMMV.

I hope to improve outcomes with fine-tuning a model specifically for this type of use and with my own use cases.

This can also lead to dangerous loops or cases where the LLM keeps trying the same thing over and over.

Basic Vector Database

It has support for basic vector database creation, but only for text based files atm.

There are other solutions available, some with MCP servers that allow better context / vector database support so I am not sure how much I am going to continue down the road of adding better support for vector databases and symantic searching.

This does show the use of them though and of giving the LLM tools to utilize them for RAG.

The Future

This is just a proof of concept tool to explore some functionality of LangChain and LangGraph while testing some ideas with local LLMs.

Some future improvement ideas / plans

• Support config files for model details (name, temperature, etc.)
• Support rules / prompt files etc.
• Expand beyond just Ollama backend
• Support a queue system for running prompts / models (and make it available to LLMs?)
• Expand to handle image data etc. (as input / output)
• Expand beyond simple demos to a higher level library for agent implementation
• Fine-tune an LLM to be good at making it’s own tool functions

I also have plans to do some testing around adversarial agents, and using different LLMs to verify and test the output from an LLM.

Fun times.

Give it a try

It is easy to pull from github, so feel free to give it try.

Help Support Me

If this sounds exciting to you, or if you can provide financial support, compute resources, hardware for my research and work, or just want to chat, then I would love to hear from you. You can reach out to me here, email me, contact me on linkedin, support me on ko-fi or donate on paypal. If you know someone else that might be interested feel free to share this with them.

My demo code is on github and there is more to come, so stay tuned.

Thanks for reading!