FuriosaAI Unveils New AI Accelerator RNGD at Hot Chips 2024

Today at Hot Chips 2024, FuriosaAI is revealing RNGD, our latest AI accelerator designed for high-performance large language model (LLM) and multimodal model inference in data centers. Co-founder and CEO June Paik will share technical details and provide a hands-on look at the fully functioning RNGD card during the presentation.

With a TDP of 150 watts, unique chip architecture, and advanced memory technology like HBM3, RNGD is optimized for inference with demanding LLMs and multimodal models. It aims to deliver high performance, power efficiency, and programmability in a single product - a feat that has been challenging for GPUs and other AI chips.

Key Milestones for RNGD

Furiosa achieved a rapid timeline in bringing up RNGD, with a full boot-up of the hardware just weeks after receiving the first silicon samples from TSMC in May. By early June, industry standard Llama 3.1 models were already running on RNGD.

The first RNGD silicon was delivered to early access customers in July, with the first private demo showcased recently. While there is more work to be done before RNGD is widely used in data centers, reaching this milestone is a significant achievement for Furiosa.

Future Updates

Currently, the focus is on refining the software stack as RNGD production ramps up. This follows the successful track record of Furiosa's first-generation chip introduced in 2021, which saw significant performance improvements through compiler enhancements.

As with previous silicon launches, Furiosa anticipates further performance enhancements for RNGD through software improvements in the coming weeks and months. Stay tuned for more updates on RNGD's performance on various LLMs.

Stay Informed

Visit Furiosa's Hot Chips booth this week to interact with the engineering team, learn more about RNGD, and see the first live demo of the AI accelerator. Keep an eye out for upcoming benchmark results, availability details, and other updates as Furiosa works towards making RNGD widely available in early 2025.