Blog Posts | Joshua Berkowitz

9 Articles

quantization ×

TorchAO: A PyTorch-Native Shortcut To Smaller, Faster Models

TorchAO is PyTorch's native toolkit for model efficiency: it unifies post-training quantization (PTQ), quantization-aware training (QAT), float8 (FP8) training, and structured sparsity in one coherent...

deep learning FP8 model efficiency open source PyTorch QAT quantization sparsity TorchAO

Nov 4, 2025

0 264

Github Repos

NVFP4 Is Transforming AI Training: 4-Bit Precision Meets High Performance

Efficiently training massive language models is now a central challenge for organizations building advanced AI systems. As models grow larger and datasets expand into the trillions of tokens, the need...

AI training Blackwell architecture generative AI large language models low precision model efficiency NVFP4 quantization

Nov 4, 2025

0 209

News

Databricks Delivers Fast, Scalable PEFT Model Serving for Enterprise AI

Enterprises aiming to deploy AI agents tailored to their proprietary data face the challange of delivering high-performance inference that can scale with complex, fragmented workloads. Parameter-Effic...

Databricks enterprise AI GPU optimization inference LoRA model serving PEFT quantization

Oct 28, 2025

0 1947

News

How MXFP8, TorchAO, and TorchTitan Boost Large-Scale AI Training on Crusoe B200

Modern AI models are growing larger and more complex, demanding new solutions to speed up training without compromising accuracy. Recent experiments on the Crusoe B200 cluster , using 1,856 GPUs, show...

AI acceleration Crusoe B200 float8 large-scale training MXFP8 PyTorch quantization TorchAO

Sep 20, 2025

0 6633

News

Accelerating Transformers: GPT-OSS-Inspired Advances in Hugging Face

Transformers are evolving fast and Hugging Face is leading the charge with new optimizations inspired by OpenAI's GPT-OSS models . If you're working with large language models, recent upgrades in the ...

GPT-OSS Hugging Face model optimization NLP parallelism quantization transformers

Sep 13, 2025

0 5863

News

Boosting Low-Precision AI: Fine-Tuning GPT-OSS with Quantization-Aware Training

Deploying large language models requires balancing accuracy and efficiency , a challenge that intensifies as demand for high-throughput generative AI grows. The open-source gpt-oss model, featuring a ...

AI deployment fine-tuning gpt-oss low precision model optimization NVIDIA QAT quantization

Sep 10, 2025

0 15774

News

BitNet: 1-bit LLMs Land With Practical Inference on CPUs and GPUs

BitNet from Microsoft Research is the official C++ inference stack for native 1-bit large language models, centered on BitNet b1.58. The repo ships fast, lossless ternary kernels for CPUs, a CUDA W2A8...

1-bit LLM BitNet CPU GGUF GPU inference llama.cpp quantization T-MAC

Aug 27, 2025

0 29436

Github Repos

FP4 Quantization Meets NVIDIA HGX B200: A New Era of Efficient AI

AI technology is advancing at lightning speed, and the search for greater efficiency has led to a breakthrough: FP4 quantization . This 4-bit floating-point format, when combined with Lambda’s NVIDIA ...

AI acceleration deep learning FP4 Lambda Cloud model optimization NVIDIA B200 quantization TensorRT

Aug 2, 2025

0 7348

News

AMD Ryzen AI Max+ Upgrade: Powering 128B-Parameter LLMs Locally on Windows PCs

With AMD's latest update deploying massive language models, up to 128 billion parameters, directly on your Windows laptop is now a possible. AMD’s Ryzen AI Max+ is a breakthrough that brings state-of-...

AMD context window large language models LLM deployment local AI quantization Ryzen AI Windows AI

Jul 31, 2025

0 22847

News

Get All The Latest Research & News!

Subscribe

Our latest content

Check out what's new !

See all

Ads

Prompt Maker Image Generator

Struggling with the perfect AI image prompt? My free app helps you generate brilliant ideas and instantly creates an image to match. Go from concept to creation in two clicks!

Most Popular Articles

Check out what the hot topics are!

See all

Every shirt tells a story—and every story

#ClothingForACause