Chatbots are used by millions of people around the world every day and are powered by NVIDIA GPU-based cloud servers. Now, these breakthrough tools are coming to Windows PCs powered by NVIDIA RTX for native, fast, customizable generative AI. |
|
Chat with RTX is a technology demo, now available as a free download, that allows users to personalize chatbots with their own content and is powered by a native NVIDIA GeForce RTX 30 Series GPU or higher (with at least 8GB of Video Random Access Memory or VRAM) accelerate. |
|
Chat with RTX uses Retrieval Augmented Generation (RAG), NVIDIA TensorRT-LLM software, and NVIDIA RTX acceleration to bring generative AI capabilities to Windows PCs with native GeForce support. Users can quickly and easily connect local files on their PC as datasets to open source large language models like Mistral or Llama, enabling queries for fast, contextual answers. |
|
Users can simply type a query instead of searching for notes or saved content. For example, people can ask, "What restaurant would my partner recommend while in Las Vegas?" Chat with RTX will scan the local file the user points to and provide an answer with context. |
|
The tool supports various file formats including .txt, .pdf, .doc/.docx, and .xml. Point the application to the folder containing these files and the tool will load them into its library within seconds. |
|
Users can also include information from YouTube videos and playlists. Adding video URLs to Chat with RTX allows users to integrate this knowledge into their chatbots for contextual queries. For example, ask for travel advice based on the content of your favorite influencer¡¯s videos, or get quick tutorials and how-tos based on top educational resources. Use RTX chat to integrate knowledge from YouTube videos into queries. |
|
Because Chat with RTX runs natively on Windows RTX PCs and workstations, results are delivered quickly and the user's data remains on the device. Chat with RTX does not rely on cloud-based LLM services, instead letting users work with sensitive data on their local PC without the need to share with a third party or have an internet connection. |
|
A GeForce RTX 30 Series GPU or higher with at least 8GB of VRAM, along with Windows 10 or 11 and the latest NVIDIA GPU drivers, is mandatory for utilizing Chat with RTX. |
|
A chat with RTX showcases the potential of using RTX GPUs to accelerate the LLM. The application is built based on the TensorRT-LLM RAG developer reference project (available on GitHub). Developers can use the reference project to develop and deploy their own RAG-based RTX applications, accelerated by TensorRT-LLM. Learn more about building LLM-based applications. |
|
Chat with RTX is now free to download! You can get started and unleash your creativity with personalized chatbots. For more details, visit the official NVIDIA page or check out the NVIDIA Blog post announcing its availability. |
Chat with RTX is a fascinating tech demo that allows users to personalize a chatbot using their own content. It's like giving your chatbot a unique flavor! |
Here's what you need to know about Chat with RTX: |
1. Introduction: |
Chat with RTX establishes a connection between your data (documents, notes, videos, or other content) and a large language model (LLM). This personalized touch makes the chatbot more engaging and context-aware. |
|
2. How It Works: |
Retrieval-Augmented Generation (RAG): This technique combines retrieval-based methods with language generation. It allows you to query a custom chatbot and receive contextually relevant answers. TensorRT-LLM and RTX Acceleration: These technologies ensure efficient and speedy interactions with your personalized chatbot. Plus, Chat with RTX runs locally on your Windows RTX PC or workstation, ensuring security and faster results. |
|
3. Supported File Formats: |
Chat with RTX can handle various file formats, including text files, PDFs, DOC/DOCX, and XML. You can specify a folder containing your target files, and the application will load them into its library within seconds. Additionally, if you provide a YouTube playlist URL, Chat with RTX will automatically load transcriptions from the videos, allowing you to query content within the videos. |
|
4. Designed for Developers: |
Chat with RTX is a tech demo based on the TensorRT-LLM RAG developer reference project available on GitHub. Developers can explore this reference project and create their own applications that are RTX-accelerated, RAG-based, and powered by TensorRT-LLM. |
|
5. Availability: |
Chat with RTX is now free to download! You can get started and unleash your creativity with personalized chatbots. |
cs@easiersoft.com