NVIDIA brings AI assistants to life with GeForce RTX AI PCs

COMPUTEX—NVIDIA announced new ones today NVIDIA RTX™ technology to power AI assistants and digital humans that run on new GeForce RTX™ AI Laptops.

NVIDIA has unveiled Project G-Assist: a demo of RTX-powered AI assistant technology that provides context-aware assistance for PC games and apps. The Project G-Assist tech demo debuted with ARK: Survival Ascended from Studio Wildcard. NVIDIA also introduced the first PC-based NVIDIA NIM™ inference microservices for the NVIDIA ACE digital human platform.

These technologies are powered by the NVIDIA RTX AI toolkit, a new set of tools and software development kits that help developers optimize and deploy large generative AI models on Windows PCs. They join NVIDIA’s full-stack RTX AI innovations that accelerate more than 500 PC applications and games and 200 manufacturer laptop designs.

Additionally, the newly announced RTX AI PC laptops from ASUS and MSI feature GeForce RTX 4070 GPUs and low-power systems-on-a-chip with Windows 11 AI PC capabilities. These Windows 11 AI PCs will receive a free update to Copilot+ PC experiences, where available.

“NVIDIA launched the era of AI PCs in 2018 with the release of RTX Tensor Core GPUs and NVIDIA DLSS,” said Jason Paul, vice president of consumer AI at NVIDIA. “Now with Project G-Assist and NVIDIA ACE, we are unlocking the next generation of AI-powered experiences for more than 100 million RTX AI PC users.”

Project G-Assist, a GeForce AI assistant
AI assistants will transform gaming and in-app experiences – from providing game strategies and analyzing multiplayer replays to assisting with complex creative workflows. Project G-Assist is a glimpse into this future.

PC games offer vast universes to explore and complicated mechanics to master, which are challenging and time-consuming feats for even the most dedicated gamers. Project G-Assist aims to put gaming knowledge at the fingertips of players using generative AI.

Project G-Assist takes voice or text input from the player, along with contextual information from the game screen, and feeds the data through AI vision models. These models enhance contextual awareness and app-specific understanding of a large language model (LLM) linked to a game knowledge base, then generate a tailored response delivered in the form of text or speech.

NVIDIA partnered with Studio Wildcard to demonstrate the technology ARK: Survival Ascended. Project G-Assist can help answer questions about creatures, items, lore, objectives, difficult bosses, and more. Because Project G-Assist is context-aware, it personalizes responses to the player’s gaming session.

Additionally, Project G-Assist can configure the player’s gaming system for optimal performance and efficiency. It can provide insight into performance metrics, optimize graphics settings depending on the user’s hardware, apply a safe overclock, and even intelligently reduce power consumption while maintaining a performance target.

First ACE PC NIM debuts

NVIDIA ACE technology to power digital humans is now coming to RTX AI PCs and workstations with NVIDIA NIM – inference microservices that help developers reduce deployment time from weeks to minutes. ACE NIM microservices deliver high-quality inferences that run locally on devices for natural language understanding, speech synthesis, facial animation, and more.

COMPUTEX will see the gaming debut of NVIDIA ACE NIM on PC in the Covert Protocol technical demo, developed in collaboration with Inworld AI. It turns out now NVIDIA Audio2Face™ and NVIDIA Riva automatic speech recognition that runs locally on devices.

Windows Copilot Runtime to add GPU acceleration for local PC SLMs
Microsoft and NVIDIA are working together to help developers bring new generative AI capabilities to their Windows native and web apps. This collaboration will provide application developers with easy application programming interface (API) access to GPU-accelerated small language models (SLMs) that enable retrieval-augmented generation (RAG) capabilities that run on the device as part of the Windows Copilot Runtime.

SLMs offer tremendous capabilities for Windows developers, including content summarization, content generation, and task automation. RAG capabilities augment SLMs by giving the AI ​​models access to domain-specific information that is not well represented in basic models. RAG APIs enable developers to leverage application-specific data sources and tailor SLM behavior and capabilities to meet application needs.

These AI capabilities will be accelerated by NVIDIA RTX GPUs, as well as AI accelerators from other hardware vendors, giving end users fast, responsive AI experiences across the breadth of the Windows ecosystem.

The API will be released in developer preview later this year.

4x faster, 3x smaller models with the RTX AI Toolkit
The AI ​​ecosystem has built hundreds of thousands of open source models for app developers to use, but most models are pre-trained for general purposes and built to run in a data center.

To help developers build application-specific AI models that run on PCs, NVIDIA introduces RTX AI toolkit – a suite of tools and SDKs for model customization, optimization, and deployment on RTX AI PCs. RTX AI Toolkit will be available for broader developer access later this month.

Developers can customize a pre-trained model with open-source QLoRa tools. They can then use the NVIDIA TensorRT™ model optimization to quantize models to consume up to 3x less RAM. NVIDIA TensorRT Cloud then optimizes the model for peak performance across the RTX GPU lineups. The result is up to 4x faster performance compared to the pre-trained model.

The new NVIDIA AI Inference Management SDK, now available in early access, simplifies the deployment of ACE on PCs. It preconfigures the PC with the necessary AI models, engines, and dependencies, while seamlessly orchestrating AI inference across PCs and the cloud.

Software partners such as Adobe, Blackmagic Design and Topaz are integrating components of the RTX AI Toolkit into their popular creative apps to accelerate AI performance on RTX PCs.

“Adobe and NVIDIA continue to work together to deliver breakthrough customer experiences across all creative workflows, from video to imaging, design, 3D and more,” said Deepa Subramaniam, vice president of product marketing, Creative Cloud at Adobe. “TensorRT 10.0 on RTX PCs delivers unprecedented performance and AI-powered capabilities for makers, designers and developers, unlocking new creative possibilities for content creation in leading creative tools like Photoshop.”

Components of the RTX AI Toolkit, such as TensorRT-LLM, are integrated into popular developer frameworks and applications for generative AI, including Automatic1111, ComfyUI, Jan.AI, LangChain, LlamaIndex, Oobabooga, and Sanctum.AI.

AI for content creation
NVIDIA is also integrating RTX AI acceleration into apps for creators, modders and video enthusiasts.

Last year, NVIDIA introduced RTX acceleration using TensorRT for one of the most popular Stable Diffusion user interfaces, Automatic1111. Starting this week, RTX will also accelerate the highly popular ComfyUI, delivering up to a 60% performance boost over the currently available version, and 7x faster performance compared to the MacBook Pro M3 Max.

NVIDIA RTX Remix is a modding platform for remastering classic DirectX 8 and DirectX 9 games with full ray tracing, NVIDIA DLSS 3.5 and physically accurate materials. RTX Remix includes a runtime renderer and the RTX Remix Toolkit app, which facilitates modding game assets and materials.

Last year, NVIDIA made RTX Remix Runtime open source, allowing modders to do so expand game compatibility and advanced display capabilities.

Since the RTX Remix Toolkit launched earlier this year, 20,000 modders have used it to modify classic gamesresulting in over 100 RTX remasters in development on the RTX Remix Showcase Discord.

This month, NVIDIA will open source the RTX Remix Toolkit, allowing moderators to streamline how assets are replaced and scenes relighted, expand the supported file formats for RTX Remix’s asset ingestor, and enable RTX Remix’s AI Texture Tools reinforced with new models.

Additionally, NVIDIA is making RTX Remix Toolkit’s capabilities accessible through a REST API, allowing moderators to connect RTX Remix live to digital content creation tools like Blender, modding tools like Hammer, and generative AI apps like ComfyUI. NVIDIA also provides an SDK for RTX Remix Runtime so that moderators can implement RTX Remix’s renderer in applications and games other than DirectX 8 and 9 classics.

With more of the RTX Remix platform being made open source, modders around the world can build even more stunning RTX remasters.

NVIDIA RTX videothe popular AI-powered super-resolution feature supported in Google Chrome, Microsoft Edge, and Mozilla Firefox browsers is now available as an SDK for all developers, allowing them to integrate AI for upscaling, sharpening, compression artifact reduction, and high dynamic range (HDR) conversion .

Coming soon to video editing software Blackmagic Design’s DaVinci Resolve and Wondershare Filmora, RTX Video will allow video editors to upscale lower quality video files to 4K resolution, and convert standard dynamic range source files to HDR. Additionally, the free media player VLC media will soon add RTX Video HDR to its existing super-resolution capabilities.

Learn more about RTX AI PCs and technology by becoming a member NVIDIA at COMPUTEX.