Blog Posts | Joshua Berkowitz

4 Articles

performance optimization ×

How Dicer Is Revolutionizing Auto-Sharding for Distributed Systems

Engineering teams can struggle to scale distributed systems efficiently while maintaining performance and reliability. The introduction of Dicer, Databricks’ open-source dynamic auto-sharder looks to ...

auto-sharding caching cloud infrastructure Databricks distributed systems open source performance optimization service architecture

Jan 15, 2026

0 803

News

Democratizing Scalable Mixture-of-Experts Training in PyTorch with NVIDIA NeMo Automodel

Training state-of-the-art Mixture-of-Experts (MoE) models has traditionally requiredspecialists with deep distributed systems knowledge and access to high-end infrastructure. Now, NVIDIA’s NeMo Automo...

distributed training LLMs MoE NVIDIA open source performance optimization PyTorch

Nov 12, 2025

0 5896

News

Hugging Face Introduces Dataset Streaming for Machine Learning

If you’ve ever been frustrated by long waits to download massive datasets for model training, you’re not alone. Hugging Face has introduced a groundbreaking way to stream multi-terabyte datasets direc...

data engineering dataset streaming huggingface machine learning parquet performance optimization Xet storage

Nov 7, 2025

0 12485

News

AI-Driven Vibe Coding and Rust Are Changing Data Engineering

Building high-performance, reliable data tools is more crucial than ever and Rust is at the forefront of this movement. Thanks to the surge in AI-powered development, a new approach called vibe coding...

AI coding asynchronous processing data engineering parallel computing performance optimization Rust vibe coding

Sep 9, 2025

0 6985

News

Our latest content

Check out what's new !

See all

Ads

Prompt Maker Image Generator

Struggling with the perfect AI image prompt? My free app helps you generate brilliant ideas and instantly creates an image to match. Go from concept to creation in two clicks!

Most Popular Articles

Check out what the hot topics are!

See all

Every shirt tells a story—and every story

#ClothingForACause