(T) Following were my favorite sessions from Nvidia GTC to get a sense of Nvidia’s latest research and system software offering…

• Insights from NVIDIA Research [S62226]
  • Bill Dally, Chief Scientist and Senior Vice President of Research, NVIDIA
  • “We’ll share some insights from NVIDIA Research for the past year. These will include a power-efficient “always-on” AI accelerator, a diffusion model that improves the resolution of weather predictions, a large language model-powered embodied agent, and a foundation model for autonomous vehicle scene reconstruction.”
  • Key takeaways:
    • “Always-on” AI inference accelerator
    • Dynamic scene representation and reconstruction for AVs
    • Diffusion models for weather super-resolution (downsampling)
    • Embodied generative AI
• Generally Capable Agents in Open-Ended Worlds [S62816]
  • Jim Fan, Research Scientist, NVIDIA
  • “I believe in a future where everything that moves will eventually be autonomous. ChatGPT unifies all kinds of natural language understanding tasks in a single interface: text in, text out. What is the equivalent for an AI agent? What does it take to build a model that actively explores the world, ingests multimodal sensory stream, plans over long horizons, acquires new skills, and bootstraps its own capabilities in a self-improving loop? I’ll lay out a blueprint for the Foundation Agent, a single model that generalizes across diverse tasks, embodiments, and realities. And that will be the next grand challenge in our quest for AI.”
  • Key takeaways:
    • Eureka: GPT-4 writes reward functions to teach a 5-finger robot hand how to do extremely dexterous tasks like pen spinning.
    • Voyager: LLM-powered agent that masters Minecraft by in-context lifelong learning
    • VIMA: Multimodal LLM for robot manipulation; unifies diverse robotics tasks in a single prompting framework
    • MineDojo: Large-scale open-ended agent learning framework in Minecraft.

• Accelerating Enterprise: Tools and Techniques for Next-Generation AI Deployment [S63432]
  • Mahan Salehi, Software Product Manager, NVIDIA
  • Nave Algarici, Generative AI Software Product Manager, NVIDIA
  • “In this session, we will delve into the dynamic realm of AI inference, examining the latest state-of-the-art tools and techniques designed to revolutionize how developers deploy generative AI models. As the AI landscape continues to rapidly evolve, the demand for increased speed and efficiency in AI inference is becoming increasingly critical. Our focus will be on the newly announced NVIDIA NIMs, a set of easy-to-use runtimes designed to accelerate the deployment of generative AI. This versatile microservice supports a wide spectrum of AI models—from open-source community models to NVIDIA AI Foundation models, as well as bespoke custom AI models.”
  • Key takeaways:
    • Nvidia NIM
    • Nvidia NeMo

Note 1: I will update that blog post with the link to the video when Nvidia will make it available on YouTube.

Note 2: The picture above are my 2024 production of Tulips.

Copyright © 2005-2024 by Serge-Paul Carrasco. All rights reserved.
Contact Us: asvinsider at gmail dot com.