Since the release of OpenAI’s revolutionary ChatGPT, the number of projects surrounding AI, especially large language models (LLMs), has been at an all-time high. Many of these projects are capable of disrupting various fields by enabling researchers and developers to leverage the power of LLMs to tackle complex tasks, enhance natural language understanding, and generate human-like text. One such project, AnythingLLM, has been released by Mintplex Labs recently. AnythingLLM is a full-stack application that aims to be the easiest way for a customer to converse intelligently with documents, resources, etc., using a well-crafted UI. AnythingLLM uses Pinecone and ChromaDB for handling vector embeddings and the OpenAI API for its LLM and conversational functionality. One defining quality that sets the tool apart is that it can simply run in the background without using a large amount of memory or resources as, by default, the LLM and vectorDB are hosted remotely on the cloud.
The creators of AnythingLLM have also emphasized its key features by highlighting how their tool differs from others currently available in the market, like PrivateGPT, LocalGPT, etc. Unlike PrivateGPT, which only is a command line tool, AnythingLLM has an interactive UI that contributes towards an intuitive and user-friendly experience. Moreover, PrivateGPT requires the user to run a local LLM on their machine, which is not the most efficient and resourceful solution for users who do not have a powerful machine. On the other hand, LocalGPT, inspired by PrivateGPT, also faces similar concerns wherein a private LLM runs on the user’s machine. There is also significant technical overhead that comes with these solutions. This is where AnythingLLM has the edge over its competitors because it uses LLMs and vectorDB that customers are already familiar with, thus, making it more accessible. This full-stack application can run locally as well as remotely in the background.
AnythingLLM uses the containerization of documents into workspaces as its foundation. In this scenario, different workspaces can share the same records but not interact, thereby allowing the user to maintain different workspaces for different use cases. AnythingLLM consists of two chat modes: conversation, wherein previous questions are retained, and query, which refers to simple question-answer chat against the user-specified document. Moreover, for publicly accessible documents, each chat response also consists of a citation that links to the original content. The project is designed as a monorepo structure and consists of mainly three sections: collector, frontend, and server. The collector is a Python utility that allows the user to quickly convert publicly accessible data from online resources such as videos from a specified Youtube Channel, Medium Articles, blog links, etc., or local documents into an LLM usable format. The frontend of the product has been built using viteJS and React, while a nodeJs and express server handles all LLM interactions and vectorDB management.
The AnythingLLM project is open-sourced under the MIT License, and the developers are looking forward to accepting bug fixes and any other contributions from the community. The project has enormous potential to completely change how users interact with documents or any piece of content using an LLM. Interested users can clone the project from this Github repository and can proceed to set up their application.