News | Joshua Berkowitz

5 Articles

PyTorch ×

Dion Optimizer: Transforming Distributed AI Training Efficiency

Optimizers such as Adam and AdamW have been essential to training large-scale neural networks. However, as model sizes soar into the trillions of parameters, the need for more efficient training metho...

AI optimization deep learning distributed training large language models open source orthonormal updates PyTorch scalability

Nov 16, 2025

0 1705

Democratizing Scalable Mixture-of-Experts Training in PyTorch with NVIDIA NeMo Automodel

Training state-of-the-art Mixture-of-Experts (MoE) models has traditionally requiredspecialists with deep distributed systems knowledge and access to high-end infrastructure. Now, NVIDIA’s NeMo Automo...

distributed training LLMs MoE NVIDIA open source performance optimization PyTorch

Nov 12, 2025

0 1617

How Monarch and Lightning AI Are Transforming Distributed PyTorch Training in Notebooks

Scaling AI experiments across massive GPU clusters is often a logistical challenge, especially for teams who want to maintain the interactive, iterative workflow of notebook development. The new integ...

AI development debugging distributed training GPU clusters Lightning AI Monarch notebooks PyTorch

Oct 28, 2025

0 5852

vLLM TPU’s Unified Backend is Revolutionizing LLM Inference

The latest vLLM TPU release is enabling developers to run open-source LLMs on TPUs with unmatched performance and flexibility. Powered by the tpu-inference backend, this innovation ensures a smooth, h...

attention kernels JAX LLM inference open source PyTorch TPU tpu-inference vLLM

Oct 18, 2025

0 16874

How MXFP8, TorchAO, and TorchTitan Boost Large-Scale AI Training on Crusoe B200

Modern AI models are growing larger and more complex, demanding new solutions to speed up training without compromising accuracy. Recent experiments on the Crusoe B200 cluster , using 1,856 GPUs, show...

AI acceleration Crusoe B200 float8 large-scale training MXFP8 PyTorch quantization TorchAO

Sep 20, 2025

0 6974

Our latest content

Check out what's new !

See all

Ads

Prompt Maker Image Generator

Struggling with the perfect AI image prompt? My free app helps you generate brilliant ideas and instantly creates an image to match. Go from concept to creation in two clicks!

Most Popular Articles

Check out what the hot topics are!

See all

Every shirt tells a story—and every story

#ClothingForACause