Summary
Highlights
The session begins with an introduction to Nvidia Live at CES, highlighting the rapid growth and investment in AI infrastructure. Vivek Arya, a semiconductor analyst, discusses the three key differentiating factors of the current AI infrastructural cycle: seamless adoption, high utilization, and substantial funding from well-financed companies. He notes the significant capital expenditure in AI infrastructure, projecting billions in spending by 2026, and dismisses the 'bubble' narrative, emphasizing the utility and financial backing driving this growth.
Sridhar Ramaswamy, CEO of Snowflake, joins the discussion to elaborate on the collaboration between Snowflake and Nvidia. He explains how AI is creating value across different sectors, utilizing both old and new chips. Snowflake's AI products, including Snowflake Intelligence, run on Nvidia chips. Ramaswamy discusses the company's focus on data agents, which provide instant access to customer information for CEOs, and addresses challenges in AI adoption, such as data sovereignty and the need for organizational change management.
Ramaswamy discusses the evolution of open and closed AI models, noting that while frontier models excel in specific applications, open models are crucial for broader developer adoption and scalability. He emphasizes the importance of open models in influencing developer mindsets and driving innovation. He also highlights market adoption trends, with the financial sector and healthcare leading the way in integrating AI, driven by the accessibility of tools like ChatGPT and the need to address 'dark data' for deeper insights.
The panel welcomes Shiv Rao, CEO of Abridge, and Harjinder Singh, CEO of Code Rabbit. Shiv discusses Abridge's mission to automate clerical work for clinicians in healthcare, aiming to shift the focus from 80% clerical work to 80% patient-facing time. Harjinder explains how Code Rabbit uses generative AI to review code, addressing the increased volume of code generated by AI. Both emphasize the challenge of integrating AI into complex existing workflows and building trust in their respective mission-critical fields.
The discussion delves into the reliability of agentic systems. Shiv explains that in healthcare, low-stakes, high-frequency tasks can be fully automated, while high-stakes decisions still require human oversight. Harjinder notes the challenges in coding, where interactive agents are successful, but long-running background agents struggle with reliability. They also discuss the increasing viability of open-source models, which offer cost-effectiveness and flexibility, particularly for less reasoning-heavy tasks.
Ola Källenius, CEO of Mercedes-Benz, and Deepak Pathak, CEO of Skilled AI, join the panel. Ola discusses Mercedes-Benz's journey in automated driving, from early research to achieving Level 3 autonomous driving certification. He highlights the engineering and legal complexities involved and the future of Level 2++ and Level 4 autonomous vehicles. Deepak introduces Skilled AI, which is developing a general-purpose brain for robotics, aiming to overcome the current hardware-centric view of robotics and the lack of general robot data.
Deepak Pathak discusses the critical role of data in robotics development. He explains that unlike LLMs, robotics lacks abundant internet data, necessitating a bootstrapping approach from human videos and simulation. Ola emphasizes the importance of redundancy and safety in automotive AI, comparing it to redundant systems in airplanes. He notes that the long tail of unexpected driving scenarios requires extensive data processing and a robust approach, rather than shortcuts.
Ola discusses the transformative impact of AI and robotics on manufacturing. He envisions a future where robots work alongside humans, handling simple logistics tasks and becoming 'thinking machines' that can take instructions. He also highlights the use of Nvidia's Omniverse for building digital twin factories, enabling simulation and debugging before physical construction, leading to faster and less expensive manufacturing. Deepak adds that robotics is inherently a general problem, with diverse tasks in factories requiring a general solution rather than task-specific ones.
Jensen Huang, CEO of Nvidia, takes the stage, announcing a new platform shift in computing. He states that AI is not just an application, but a platform upon which new applications will be built. He emphasizes that the way software is developed and run has fundamentally changed, moving from programming to training, and from CPUs to GPUs. This shift involves reinventing every layer of the computing stack, leading to a massive modernization effort and unprecedented investment in AI.
Huang reviews the incredible advancements in AI over the past year, highlighting the impact of scaling laws in large language models. He discusses the advent of test-time scaling, where AI models 'think' in real-time, requiring enormous compute. A significant breakthrough is the emergence of agentic systems, capable of reasoning, research, tool use, and planning, exemplified by tools like 'cursor' for software programming. He also notes the rise of physical AI and AI physics, which understand the laws of nature and physical interactions.
Huang emphasizes the importance of open models, highlighting their role in democratizing AI and enabling widespread innovation across industries and countries. He notes the rapid growth of open-source models, which are quickly catching up to frontier models. Nvidia actively contributes to this movement by building and operating its own AI supercomputers (DGX clouds) to develop frontier open models in various domains, including digital biology (Proteina, OpenFold 3, EVO), earth science (Earth-2 AI, ForecastNet, Neotron), and robotics (Cosmos, Groot).
Huang details the architecture of modern AI applications, built on agentic systems that combine proprietary and customized language models, reasoning frameworks, and access to tools. He demonstrates a personal assistant built on this framework, showcasing its ability to manage tasks, interact with emails, and control robots. This architecture, he explains, is transforming enterprise AI, with partnerships including Palantir, ServiceNow, Snowflake, Code Rabbit, CrowdStrike, and NetApp. Agentic systems are becoming the new user interface for platforms.
Huang delves into physical AI, which enables intelligent systems to interact with the real world by understanding common sense and physical laws like object permanence, causality, and inertia. He explains that training physical AI requires three types of computers: one for training models, one for inference, and one for simulation. Nvidia's Omniverse is used for digital twin simulation, and Cosmos, a world foundation model, helps generate synthetic data to train physical AI. He introduces Alpamo, Nvidia's first thinking, reasoning autonomous vehicle AI, trained end-to-end and capable of explaining its actions.
Huang introduces Vera Rubin, Nvidia's next-generation AI supercomputer, designed to tackle the escalating computational demands of AI models. He explains that due to the slowing of Moore's Law, extreme co-design across all chips and the entire stack is necessary to keep up with the 10x annual growth in model size and token generation. The Vera Rubin platform features six co-designed chips, including the Vera CPU and Rubin GPU, offering significant performance improvements (5x over previous generations) and power efficiency. He highlights MVF FP4 tensor core architecture as a revolutionary advancement for balancing throughput and precision.
Huang details the hardware innovations within the Vera Rubin system. This includes a redesigned compute chassis, reducing assembly time from hours to minutes, and achieving 100% liquid cooling with 45°C water. He highlights the Spectrumax Nick and MVLink 6 switch, which provide unparalleled networking performance, capable of moving data at speeds twice that of the global internet. He introduces the Bluefield 4 DPU, a revolutionary processor for offloading virtualization, security, and creating a fast KV cache for AI's context memory, addressing a critical bottleneck in large-scale AI deployment. This new architecture ensures confidential computing and optimized power utilization.
Huang presents performance benchmarks, demonstrating Vera Rubin's superior capabilities in training AI models (allowing 10 trillion parameter models to be trained in a month) and factory throughput (10x higher inference performance than Blackwell). He emphasizes that these advancements enable faster progression to the AI frontier, improved energy efficiency in data centers, and reduced cost per token. He concludes by reiterating Nvidia's full-stack approach to AI, from chips to applications, with the goal of empowering all to create incredible AI solutions. He thanks the audience and shares some lighthearted outtakes.