Level 3 · Advanced Scale & Production

Goal: process data at scale with Spark, build advanced orchestration patterns, design data lake architecture, go deep on streaming, govern and catalog data, wire up CI/CD, tune performance, work with cloud warehouses, and monitor pipelines in production.

Modules

1. Distributed Processing (Spark Basics)
2. Advanced Airflow Patterns
3. Data Lake Architecture
4. Streaming Deep Dive
5. Data Governance & Cataloging
6. CI/CD for Data Pipelines
7. Performance Tuning for Pipelines
8. Working with Cloud Data Warehouses
9. Data Pipeline Monitoring & Alerting
10. Project — Spark-based Batch Pipeline

Coming soon

Full lesson content for this level is being written next. Start with Level 1 · Entry, which is complete.