Nvidia shows it's still on track for AI dominance

By Ryan Shrout

The chip giant is poised to sell more recurring-revenue products, rather than just one-and-done hardware sales

In what seems like just one in a series of critically important moments for the most followed and scrutinized tech company in the world, Nvidia Corp. (NVDA) hosted developers, partners and analysts this week in San Jose, Calif., for its annual GTC event. This is Nvidia's preeminent showcase of everything related to artificial intelligence - at least it's become so, as Nvidia shifts its focus from gaming and graphics to computing and AI.

The big question attached to this event, and to Nvidia itself for the foreseeable future, is whether the company can keep riding this wave of AI growth and unprecedented leadership.

Appearing relaxed and sounding confident, Nvidia Chief Executive Jensen Huang announced the next generation of the company's AI processors, code-named Blackwell and called B100, B200, and GB200. This follows its currently shipping products, called Hopper (H100), and supplants them as the highest-performance chips for AI processing.

The Blackwell AI chips have some key architectural differences from the current generation that is keeping Nvidia in the driver's seat. The new parts offer anywhere from 2.5x to 5x better performance across a range of workloads - from training newer and larger AI models, to providing the inference compute necessary for deploying those AI models to the market.

Nvidia made even bolder claims like a 30x increase in AI-inference performance, but that comes with several caveats about the quantity of chips used in the server and the use of a new data format that the H100 doesn't support. Still, there is no getting around the fact that the B200 is going be a big step forward in raw AI performance compared to what is shipping today.

The new chip offers a much larger memory capacity, up to 192GB compared to the current 80GB, which means it can integrate bigger data sets for its training and inference models, allowing developers like OpenAI to build new capabilities even faster.

Huang also showcased more than half a dozen other processors and networking components that help Nvidia validate and deploy its AI solutions to enterprise customers as a complete solution, rather than just one piece of a very complex puzzle. This is a primary reason for Nvidia's leadership position in the AI-computing market, in that the company doesn't just make a GPU; it also has built or acquired all of the surrounding hardware components for networking, switches, data processors, cooling and power delivery. While competitors and startups in the field might look to overtake Nvidia on any one of these components, Nvidia makes the whole hardware (and software) infrastructure simple to buy and deploy.

One slight shift in the language used around the Blackwell chip announcements was additional emphasis on the inference side of AI computing. While the last couple of years were all about training and the extremely high amount of compute power needed to create new AI models like those used by Microsoft Corp. (MSFT) and OpenAI's ChatGPT, more of the story moving forward is going to be around inference, or the kind of compute needed to execute those trained models on new data, customer input and other applications.

An area of risk for Nvidia that is often discussed is the possibility that its advantages wither away when it moves from training to inference. To combat that, Nvidia showed new performance results for the B200 chip that focused on inference, or as Huang has started referring to it, "generation."

The idea is that AI-inference calculations are just creating "tokens" that are turned into whatever output the AI is tasked to do: words and sentences for a large-language model, or pictures and animations for diffusion models. The hope is that calling it "generation" rather than "inference" can help change the narrative and highlight the strong performance for Nvidia chips in this segment of AI, too.

Another critical announcement during the event was what Nvidia calls NIMs, or Nvidia inference microservices. As the name implies, these are services the company offers, rather than just products. The target users are enterprises and software developers that want to deploy AI using prebuilt software components that are easy to implement, but still customizable without dealing with hardware or system design. Nvidia says it has models and solutions ready for implementation of AI to be used for language models, image creation, drug discovery, medical imaging and more, including functions around gaming such as AI characters and animation modeling.

This puts Nvidia in a position to sell more recurring, continuous-revenue products, rather than just one-and-done hardware sales. This is likely a sign of things to come for how Nvidia can transition even further down the path to a full AI-solutions provider to customers.

Still, the question of whether Nvidia did enough this week to maintain its trajectory as the unrivaled leader in the AI space remains. I believe that the Blackwell products, despite being much more expensive to produce than its currently shipping chips, pushes the company further ahead of anyone else on the market, including Intel Corp. (INTC) and Advanced Micro Devices Inc. (AMD)

Maybe more important is the clear alignment of the entire AI ecosystem around Nvidia and its direction. The opening keynote for Huang was attended by tens of thousands of developers and AI technologists, in what looked more like a rock-concert atmosphere than tech-conference speaking engagement. That leadership position makes Nvidia's moat - which helps keep its competitors in both enterprise-AI hardware and software at bay - even more of a long-term advantage.

Ryan Shrout is president of Signal65 and founder at Shrout Research. Follow him on X @ryanshrout. Shrout has provided consulting services for AMD, Qualcomm, Intel, Arm Holdings, Micron Technology, Nvidia and others. Shrout holds shares of Intel.

More: Jensen Huang has become the Steve Jobs of AI

Also read: Why the 'Magnificent Seven' and other momentum stocks may be hitting a wall

-Ryan Shrout

This content was created by MarketWatch, which is operated by Dow Jones & Co. MarketWatch is published independently from Dow Jones Newswires and The Wall Street Journal.

(END) Dow Jones Newswires

03-23-24 0723ET

Copyright (c) 2024 Dow Jones & Company, Inc.