In February 2024, NVIDIA released ChatRTX, an experimental AI chatbot for users to interact with their local data with several LLMs to choose from. It has been in the minds of AI enthusiasts since its release because of its amazing features. Now, ChatRTX is expanding its support to even more models, including Google’s Gemma.
Highlights:
- NVIDIA’s ChatRTX now supports Google Gemma, OpenAI’s CLIP and ChatGLM3.
- These news models add better image analysis and speech recognition capabilities.
- ChatRTX works as a personalized ChatGPT-like bot to ask queries about your own local data, including docs and images.
New AI Models for ChatRTX
Nvidia’s ChatRTX, previously known as ‘Chat with RTX’ is an AI chatbot that works locally on PCs with Nvidia GeForce RTX graphics cards. By providing the chatbot with documents, text files, PDFs, and YouTube videos to power its responses, users can ask queries and get answers from them.
One of the key features of it is that it supports several LLMs. Initially, it announced support for Mistral and Llama 2. So using ChatRTX, users can create a personalized chatbot using Llama 2 or Mistral. The responses were also quite positive.
ChatRTX now supports Google’s Gemma, a lightweight Gemini derivative and ChatGLM3, an English and Chinese LLM built on the general language model foundation.
It makes sense for Google’s Gemma model to be included in ChatRTX given it was created to run natively on potent laptops or desktop computers. The process of running various models locally is made simpler by Nvidia’s app, and the resulting chatbot interface allows you to select the model that most closely matches your own data for analysis or search.
Here are the features of the newly supported models:
Google’s Gemma:
- Made for high-end desktop and laptop computers.
- Local model execution is made simpler by inclusion in ChatRTX.
- Users are able to select models according to the demands they have for data analysis.
ChatGLM3:
- A huge language model that is open to both Chinese and English speakers.
- Built upon the foundation of the universal language model.
- Improves the generation and interpretation of natural language.
ChatRTX also added capabilities of OpenAI’s CLIP (Contrastive Language–Image Pre-training). It trains the model to recognize images by enabling users to look through and interact with local image data. There is also Whisper that integrates voice-support with its amazing AI-based speech recognition support.
What makes ChatRTX Special?
It should be noted that ChatRTX is still regarded as a “tech demo.” It’s an illustration of what programmers can create with the TensorRT-LLM RAG open-source reference project, and it might make LLMs more approachable for the typical computer user.
Apart from the locally enabled running features which include documentation and summarizing features, one of ChatRTX’s advantages is that it doesn’t require any specialized knowledge, making it easy to train a customized local chatbot. It will motivate programmers to create AI apps that are even better.
Recently OpenAI also launched a platform to help developers fine-tune their AI assistants and deploy them. Maybe NVIDIA is also heading in a similar direction? We are in for so much more with this highly efficient ‘offline’ chatbot.
But remember that ChatRTX is a local chatbot and it is unaware of the outside world. When we point it to a folder on your PC, we can only pose inquiries regarding the data within.
Its adaptability to various file formats is very impressive in this day and age of online chatbots, which have several disadvantages due to latency, expense, and privacy issues. Since the model is built on LLM, which examines locally stored files across your operating system, you may relax about the confidentiality and integrity of your shared data.
Only Windows 11 PCs can use NVIDIA’s ChatRTX, which requires an NVIDIA GeForce™ RTX 30 or 40 Series GPU with at least 8GB of VRAM and 16GB of RAM or more.
Conclusion
With these new AI models now being part of NVIDIA ChatRTX, it will achieve more functionalities even when running locally on devices. Let’s see how the Chatbor performs in the days to come and we can expect support for even more AI models in the future!
