The Data Engineering
Framework
Answer three questions. Get a complete, working pipeline — Docker services, dependencies, UIs, and a unified CLI to run it all.
Everything you'd spend half a day configuring by hand — done in 60 seconds.
Interactive Scaffold
Three questions. DEGEN generates a complete project tailored to your stack.
Services Out of the Box
Kafka, Postgres, Airflow, Grafana — all in Docker, pre-wired. One degen docker-up.
UI for Every Tool
Kafka UI, pgAdmin, Prefect, dbt Docs, Jupyter — all registered and ready, no manual config.
Unified CLI
degen run, degen stream, degen docs — one consistent interface across all stacks.
Three production-proven patterns. Each one generates a fully working project.
Batch ETL
Extract raw data, transform with dbt, persist in DuckDB or Postgres. Explore results in Jupyter.
Analytics
Orchestrate scheduled pipelines with Prefect or Airflow. Visualize with Grafana or Metabase.
Streaming
Real-time event processing with Kafka and PySpark Structured Streaming.
No tool without a visual interface. Registered automatically, no manual setup.
Learn by doing
The Tutorial walks you through a real pipeline — Airflow + PySpark + dbt + Postgres — writing every file from scratch, running every command, and seeing live results in the UI.
Start the Tutorial →