Blog Posts | Joshua Berkowitz

13 Articles

data engineering ×

dbt’s Structured Context Makes Agentic Data Development Safe and Scalable

AI is transforming how applications are built, but data pipelines often lag behind. Automation hits a barrier due to a lack of structured context . Without this, even the most advanced AI agents face ...

AI agents automation data engineering data pipelines dbt MCP server structured context

Dec 11, 2025

0 209

News

Amazon MWAA Serverless: Simplifying Workflow Orchestration for Data Teams

Managing Apache Airflow clusters has historically been complex and resource-intensive. The introduction of Amazon Managed Workflows for Apache Airflow (MWAA) Serverless changes that equation by lettin...

Airflow AWS cloud computing cost optimization data engineering security serverless workflow automation

Nov 18, 2025

0 6017

News

Conversational AI and AWS MCP Are Revolutionizing ETL Pipelines

In the fast-evolving world of data engineering, the demand for efficient ETL (Extract, Transform, Load) processes continues to grow. Traditionally, building robust ETL pipelines involved manual script...

Amazon Q AWS conversational AI data engineering ETL Model Context Protocol Redshift S3 Tables

Nov 15, 2025

0 1540

News

Google BigQuery’s Data Engineering Agent Is Automating the Future of Analytics

Manual pipeline maintenance can drain your team’s creativity and slow down innovation. Google’s new Data Engineering Agent in BigQuery, now in preview , aims to automate routine data engineering tasks...

AI agents automation BigQuery cloud analytics data engineering data migration pipeline management

Nov 10, 2025

0 6193

News

Hugging Face Introduces Dataset Streaming for Machine Learning

If you’ve ever been frustrated by long waits to download massive datasets for model training, you’re not alone. Hugging Face has introduced a groundbreaking way to stream multi-terabyte datasets direc...

data engineering dataset streaming huggingface machine learning parquet performance optimization Xet storage

Nov 7, 2025

0 11561

News

Empowering Developers: Microsoft Fabric Extension for VS Code Goes Open Source

Microsoft is inviting developers to take the reins with the open-source Fabric Core extension for Visual Studio Code. This strategic move not only demonstrates the company’s commitment to transparency...

community data engineering developer tools extensions GitHub Microsoft Fabric open-source VS Code

Oct 30, 2025

0 2244

News

Databricks Data Orchestration: Table Update Triggers in Lakeflow Jobs

Start eliminating the guesswork from your data workflows where pipelines launch the moment fresh data lands, and insights never lag behind reality. With Databricks’ latest table update triggers in Lak...

automation Databricks data engineering data orchestration event-driven Lakeflow Jobs table triggers Unity Catalog

Oct 21, 2025

0 11451

News

Docker MCP Toolkit Supercharges Codex: From Code Generation to Data Engineering

Today’s AI coding assistants are impressive, but what if they could do more than just suggest code? By connecting OpenAI’s Codex to Docker’s Model Context Protocol (MCP) servers with the Docker MCP To...

AI assistants automation Codex data engineering Docker knowledge graphs MCP Toolkit Neo4j

Oct 20, 2025

0 10285

News

dbt Agents and the Remote MCP Server Are Shaping Trusted AI Analytics

Analytics is entering a new era, driven by AI-powered agents and robust governance frameworks. dbt Labs recently unveiled the remote dbt MCP Server alongside dbt Agents, a suite of secure, task-specif...

AI agents analytics data engineering dbt Fusion engine governance MCP server self-service analytics

Oct 15, 2025

0 4191

News

Databricks Variant: The Open Standard For Semi-Structured Data in the Lakehouse

Organizations increasingly face the challenge of managing vast amounts of semi-structured data, such as logs and telemetry, in analytics and AI workflows. Historically, teams had to choose between slo...

Apache Iceberg data engineering data performance Delta Lake lakehouse Parquet semi-structured data Variant

Oct 10, 2025

0 4411

News

Hugging Face’s FinePDFs Dataset For AI Training

AI research has long relied on web-scraped content, but Hugging Face’s FinePDFs dataset is set to change the landscape. By sourcing over 475 million documents directly from PDFs, often considered too ...

AI data engineering datasets Hugging Face language models machine learning open source PDF

Sep 19, 2025

0 11099

News

AI-Driven Vibe Coding and Rust Are Changing Data Engineering

Building high-performance, reliable data tools is more crucial than ever and Rust is at the forefront of this movement. Thanks to the surge in AI-powered development, a new approach called vibe coding...

AI coding asynchronous processing data engineering parallel computing performance optimization Rust vibe coding

Sep 9, 2025

0 6303

News

1
2

Our latest content

Check out what's new !

See all

Ads

Prompt Maker Image Generator

Struggling with the perfect AI image prompt? My free app helps you generate brilliant ideas and instantly creates an image to match. Go from concept to creation in two clicks!

Most Popular Articles

Check out what the hot topics are!

See all

Every shirt tells a story—and every story

#ClothingForACause