NVIDIA is offering a demo of the secret protocol powered by Inworld AI

During today's GTC 2024 keynote, NVIDIA CEO Jensen Huang showed off a new AI NPC demo called Covert Protocol. Unlike the Kairos demo shown at CES 2024, which was based on Convai technology, this demo is powered by Inworld AI, the other major AI NPC platform geared toward game development. As you may recall, Inworld AI was first added by the “Bloc” model in games such as Mount & Blade II: Bannerlord, The Elder Scrolls V: Skyrim, and Grand Theft Auto V, though Rockstar quickly nixed this much to the dismay of its creator Content. .

In November, Microsoft announced a partnership with Inworld AI to develop a cross-platform toolkit featuring an AI design co-pilot and a character runtime engine. Now, Inworld's technology has been integrated with NVIDIA ACE Gaming, the Avatar Cloud Engine platform unveiled last year at Computex.

NVIDIA ACE aims to provide a complete “digital human pipeline.” It all starts with Riva, which enables automatic speech recognition (ASR) and text-to-speech (TTS) using the customizable Nemotron Large Language Model (LLM). Here, developers can create specific backstories and characters that fit the developer's game world. They can also better tailor player interactions to the context of the scene through programmable rules for NPCs using NeMo Guardrails. Finally, Audio2Face automatically creates expressive facial animations for game characters from just an audio source.

At the end of Inworld, developers can adjust various parameters for each NPC: personality, knowledge, perception, objectives, scenes, safety, long-term memory, and relationships. Kellan Gibbs, CEO of Inworld AI, also said:

The combination of NVIDIA ACE microservices and the Inworld Engine allows developers to create digital characters that can drive dynamic narratives, opening up new possibilities for how players can decode, reason, and play.

Interestingly, NVIDIA's “Digital Human Platform Partners” intend to take ACE to market, both for game developers and enterprises for customer service purposes. Here's the current list shared at GTC 2024: