These pipeline examples show how the same source-transform-warehouse shape adapts to classic batch ETL, modern ELT, streaming pipelines, and change-data-capture.
Who uses it: Team building a first analytics pipeline
Why this works: Classic batch ETL transforms before load — the diagram shows transformation as a discrete stage, which keeps the warehouse minimal but pushes complexity into the pipeline tooling.
Who uses it: Team using Snowflake / BigQuery / Redshift
Why this works: ELT inverts the order — the diagram puts load before transform and the transform stage lives inside the warehouse. This works because modern cloud warehouses make transformation cheap at scale.
Who uses it: Team needing near-real-time data
Why this works: Streaming replaces the batch scheduler with a continuously running pipeline — the diagram drops the scheduler box and adds a stream-processing engine, because freshness requirements push the architecture from periodic batches to always-on flow.
Who uses it: Team syncing operational DBs into the warehouse
Why this works: CDC eliminates the periodic full-extract — the diagram replaces the extract stage with a change-log reader, so the warehouse stays in sync without ever scanning the source tables.
Go back to the template, swap in your own topics, and keep the same structure if it fits your class or project.
Use this template: /editor/new?template=etl-pipeline-architecture
Edit this ETL template