Blog Posts | Joshua Berkowitz

4 Articles

PyTorch ×

How Monarch and Lightning AI Are Transforming Distributed PyTorch Training in Notebooks

Scaling AI experiments across massive GPU clusters is often a logistical challenge, especially for teams who want to maintain the interactive, iterative workflow of notebook development. The new integ...

AI development debugging distributed training GPU clusters Lightning AI Monarch notebooks PyTorch

Oct 28, 2025

0 3487

News

vLLM TPU’s Unified Backend is Revolutionizing LLM Inference

The latest vLLM TPU release is enabling developers to run open-source LLMs on TPUs with unmatched performance and flexibility. Powered by the tpu-inference backend, this innovation ensures a smooth, h...

attention kernels JAX LLM inference open source PyTorch TPU tpu-inference vLLM

Oct 18, 2025

0 11528

News

TorchTitan: Democratizing Large-Scale Distributed Training with PyTorch

TorchTitan: Democratizing Large-Scale Distributed Training with PyTorch A comprehensive look at PyTorch's native solution for production-ready LLM pre-training Distributed training of large language m...

AI Infrastructure Context Parallel Distributed Training Float8 FSDP2 Large Language Models Open Source Pipeline Parallel PyTorch Tensor Parallel torch.compile TorchTitan

Oct 7, 2025

0 11352

Github Repos

How MXFP8, TorchAO, and TorchTitan Boost Large-Scale AI Training on Crusoe B200

Modern AI models are growing larger and more complex, demanding new solutions to speed up training without compromising accuracy. Recent experiments on the Crusoe B200 cluster , using 1,856 GPUs, show...

AI acceleration Crusoe B200 float8 large-scale training MXFP8 PyTorch quantization TorchAO

Sep 20, 2025

0 6237

News

Get All The Latest Research & News!

Subscribe

Our latest content

Check out what's new !

See all

Ads

Prompt Maker Image Generator

Struggling with the perfect AI image prompt? My free app helps you generate brilliant ideas and instantly creates an image to match. Go from concept to creation in two clicks!

Most Popular Articles

Check out what the hot topics are!

See all

Every shirt tells a story—and every story

#ClothingForACause