FreeData & Analytics
Data Engineering
Build the modern data stack end to end: PySpark for batch processing, Airflow for orchestration, Kafka for streaming, dbt for transformations, then ship a real data pipeline from ingestion to analytics.
The big picture first: what data engineers actually build. Pipelines, batch vs streaming, warehouses and the modern data stack, with a hands-on tour of the core tools.
- What data engineering is and where it fits vs data science
- Building data pipelines and the ETL/ELT distinction
- Batch processing with Spark and streaming with Kafka
- Orchestration with Airflow and transformations with dbt
- SQL, CRON jobs and moving data with Airbyte