Nvidia NeMo SteerLM: AI personalities for in-game characters

The new technique allows developers to customize the behavior and emotion of NPCs using large language models

Nvidia, the leading company in graphics and artificial intelligence, has announced a new technology ( Nvidia NeMo SteerLM ) that enables game developers to create intelligent and realistic in-game characters powered by generative AI.

The technology, called Nvidia ACE (Artificial Character Engine), is a suite of tools and frameworks that leverages Nvidia’s expertise in computer vision, natural language processing, and deep learning to generate high-quality 3D models, animations, voices, and dialogues for virtual characters.

One of the key components of Nvidia ACE is NeMo SteerLM, a tool that allows developers to train large language models (LLMs) to provide responses aligned with particular attributes ranging from humorous to helpful. NeMo SteerLM is based on Nvidia’s NeMo framework, which simplifies the creation of conversational AI applications.

What is NeMo SteerLM and how does it work?

NeMo SteerLM is a new technique that enables developers to customize the personality of NPCs for more emotive, realistic and memorable interactions.

Most LLMs are designed to provide only ideal responses, free of personality or emotion, as you can see by interacting with chat bots. With the NeMo SteerLM technique, however, LLMs are trained to provide responses aligned with particular attributes, ranging from humor to creativity, to toxicity, all of which can be quickly configured through simple sliders.

For example, a character can respond differently depending on the player’s mood, personality, or actions. A friendly character can crack jokes or give compliments, while an enemy character can insult or threaten the player. A character can also adapt to the context and tone of the conversation, such as being sarcastic or serious.

The NeMo SteerLM technique helps turn polite chat bots into emotive characters that will enable developers to create more immersive and realistic games.

What are the benefits and challenges of using NeMo SteerLM?

The benefits of using NeMo SteerLM are:

• It can create more engaging and immersive gaming experiences by enabling characters to have natural and diverse conversations with players.
• It can reduce the development time and cost by automating the generation of dialogues and personalities for NPCs.
• It can enable multiple characters with a single LLM by infusing personality attributes into the model.
• It can create faction attributes to align responses to the in-game story – allowing the character to be dynamically influenced by a changing open world.

The challenges of using NeMo SteerLM are:

• It requires a large amount of data and computational resources to train LLMs.
• It may generate inappropriate or offensive responses that could harm the reputation or image of the game or developer.
• It may encounter ethical or legal issues regarding the ownership and responsibility of the generated content.

How can developers access and use NeMo SteerLM?

NeMo SteerLM is part of the Nvidia ACE platform, which is a cloud-based collaboration and simulation platform for 3D content creation. Nvidia ACE connects different applications and tools through a common physics engine and rendering pipeline, enabling seamless interoperability and real-time collaboration.

Nvidia ACE also includes other tools such as:

• Nvidia Omniverse Kaolin, a framework for 3D deep learning that enables fast and easy creation of 3D models from images, videos, or sketches.
• Nvidia Omniverse Audio2Face, a tool that generates realistic facial animations from audio inputs using a neural network.
• Nvidia Omniverse Machinima, a platform that allows users to create cinematic videos using assets from popular games.

Nvidia ACE is currently in early access and will be available to game developers later this year. For more information, visit the official website or watch the video.

Resources:

• https://www.linkedin.com/posts/nvidia-game-developer_generativeai-gamedev-llm-activity-7099793875304665090-NPOE
• https://www.shacknews.com/article/136734/nvidia’s-nemo-steerlm-npc-announcement
• https://developer.nvidia.com/nemo
• https://developer.nvidia.com/omniverse/ace
• https://www.salamaproductions.eu.org/news/