Blog Posts | Joshua Berkowitz

12 Articles

quantization ×

Inside the Transformers v5 Release From HuggingFace

Hugging Face's Transformers library just reached a pivotal moment with the v5.0.0rc0 release, its first major version upgrade in five years. With over 800 commits, this release introduces sweeping cha...

api changes huggingface new models quantization release notes tokenization trainer transformers

Dec 7, 2025

0 8327

News

Unsloth Dynamic GGUFs: How Extreme Model Compression Outperforms AI Giants

Compressing a large language model by 75% and still outperforming the latest releases from OpenAI and Anthropic is the promise of Unsloth Dynamic GGUFs. Their integration with the Aider Polyglot bench...

Aider Polyglot benchmarking DeepSeek LLMs model compression open-source AI quantization Unsloth

Dec 6, 2025

0 2431

News

FLUX.2 Ushers in a New Era of Scalable Image Generation

The release of FLUX.2 by Black Forest Labs signals a dramatic shift in the landscape of image generation models. This new architecture is not a mere iteration but a ground-up redesign, dramatically ex...

deep learning diffusers FLUX.2 image generation LoRA memory optimization multimodal quantization

Dec 4, 2025

0 1848

News

TorchAO: A PyTorch-Native Shortcut To Smaller, Faster Models

TorchAO is PyTorch's native toolkit for model efficiency: it unifies post-training quantization (PTQ), quantization-aware training (QAT), float8 (FP8) training, and structured sparsity in one coherent...

deep learning FP8 model efficiency open source PyTorch QAT quantization sparsity TorchAO

Nov 4, 2025

0 17479

Github Repos

NVFP4 Is Transforming AI Training: 4-Bit Precision Meets High Performance

Efficiently training massive language models is now a central challenge for organizations building advanced AI systems. As models grow larger and datasets expand into the trillions of tokens, the need...

AI training Blackwell architecture generative AI large language models low precision model efficiency NVFP4 quantization

Nov 4, 2025

0 16830

News

Databricks Delivers Fast, Scalable PEFT Model Serving for Enterprise AI

Enterprises aiming to deploy AI agents tailored to their proprietary data face the challange of delivering high-performance inference that can scale with complex, fragmented workloads. Parameter-Effic...

Databricks enterprise AI GPU optimization inference LoRA model serving PEFT quantization

Oct 28, 2025

0 3883

News

How MXFP8, TorchAO, and TorchTitan Boost Large-Scale AI Training on Crusoe B200

Modern AI models are growing larger and more complex, demanding new solutions to speed up training without compromising accuracy. Recent experiments on the Crusoe B200 cluster , using 1,856 GPUs, show...

AI acceleration Crusoe B200 float8 large-scale training MXFP8 PyTorch quantization TorchAO

Sep 20, 2025

0 7909

News

Accelerating Transformers: GPT-OSS-Inspired Advances in Hugging Face

Transformers are evolving fast and Hugging Face is leading the charge with new optimizations inspired by OpenAI's GPT-OSS models . If you're working with large language models, recent upgrades in the ...

GPT-OSS Hugging Face model optimization NLP parallelism quantization transformers

Sep 13, 2025

0 8041

News

Boosting Low-Precision AI: Fine-Tuning GPT-OSS with Quantization-Aware Training

Deploying large language models requires balancing accuracy and efficiency , a challenge that intensifies as demand for high-throughput generative AI grows. The open-source gpt-oss model, featuring a ...

AI deployment fine-tuning gpt-oss low precision model optimization NVIDIA QAT quantization

Sep 10, 2025

0 21109

News

BitNet: 1-bit LLMs Land With Practical Inference on CPUs and GPUs

BitNet from Microsoft Research is the official C++ inference stack for native 1-bit large language models, centered on BitNet b1.58. The repo ships fast, lossless ternary kernels for CPUs, a CUDA W2A8...

1-bit LLM BitNet CPU GGUF GPU inference llama.cpp quantization T-MAC

Aug 27, 2025

0 41778

Github Repos

FP4 Quantization Meets NVIDIA HGX B200: A New Era of Efficient AI

AI technology is advancing at lightning speed, and the search for greater efficiency has led to a breakthrough: FP4 quantization . This 4-bit floating-point format, when combined with Lambda’s NVIDIA ...

AI acceleration deep learning FP4 Lambda Cloud model optimization NVIDIA B200 quantization TensorRT

Aug 2, 2025

0 12232

News

AMD Ryzen AI Max+ Upgrade: Powering 128B-Parameter LLMs Locally on Windows PCs

With AMD's latest update deploying massive language models, up to 128 billion parameters, directly on your Windows laptop is now a possible. AMD’s Ryzen AI Max+ is a breakthrough that brings state-of-...

AMD context window large language models LLM deployment local AI quantization Ryzen AI Windows AI

Jul 31, 2025

0 35365

News

Our latest content

Check out what's new !

See all

Ads

Prompt Maker Image Generator

Struggling with the perfect AI image prompt? My free app helps you generate brilliant ideas and instantly creates an image to match. Go from concept to creation in two clicks!

Most Popular Articles

Check out what the hot topics are!

See all

Every shirt tells a story—and every story

#ClothingForACause