Every tool you need. One platform.

OrcheStack supports a curated set of open-source tools per pipeline layer. Each category has a tested default and documented alternatives — pick what fits your team during setup, swap later without losing data.

Ingestion

Pulls data from sources into the platform

Airbyte

DefaultCold

Connector-based ETL supporting 300+ sources. Runs on a schedule defined by Airflow.

Config: source.host, source.port, source.auth, destination.url

Custom Python

AlternativeCold

For sources without an Airbyte connector. Write Python scripts that run as Airflow DAG tasks.

Config: lives in your Airflow DAG code

Orchestration

The clock that schedules every pipeline run

Apache Airflow

CoreHot (sched)

Scheduler runs in the hot tier. Workers spin up only during task execution. Can't be disabled — orchestration is the backbone.

Config: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN, AIRFLOW__CORE__EXECUTOR, FERNET_KEY

Storage

Where data lives — lake and warehouse

PostgreSQL

Default warehouseHot

Primary warehouse. One Postgres instance hosts two logical databases: OrcheStack's own (platform schema) and your pipeline data (raw, marts, Airflow metadata) — kept separate for clean backups and permission scoping.

Config: ORCHESTACK_DB_* (in .env) + PIPELINE_DB_* (in setup wizard)

ClickHouse

AlternativeHot

Column-store warehouse for analytical workloads above ~1 TB. Heavier than PostgreSQL.

Config: CLICKHOUSE_USER, CLICKHOUSE_PASSWORD, CLICKHOUSE_DB

DuckDB

AlternativeEmbedded

In-process analytical engine for small datasets. No separate server; lives inside dbt.

Config: duckdb.path (file location)

MinIO

Core lakeHot

S3-compatible object store. Open end — AI / ML / DS consumers read raw files here without going through the warehouse.

Config: MINIO_ROOT_USER, MINIO_ROOT_PASSWORD, MINIO_BUCKET

Transformation

SQL models that turn raw data into marts

dbt Core

DefaultCold

Industry-standard transformation framework. Reads from PostgreSQL raw, writes marts in-place.

Config: profiles.target, warehouse.host, warehouse.db, warehouse.schema

SQLMesh

AlternativeCold

Modern dbt alternative with virtual data environments and better incremental semantics.

Config: connection.type, connection.host, connection.user

Data quality

Assertions that guard your marts

Great Expectations

DefaultCold

Declarative data quality checks. Runs after every dbt build. Failed expectations block downstream tasks.

Config: datasource.url, checkpoints.dir, expectations.dir

Soda Core

AlternativeCold

YAML-first data quality. Simpler setup; less powerful assertions than Great Expectations.

Config: soda.yml with datasource + checks

Governance

Catalogue, lineage, documentation

OpenMetadata

DefaultCold

Column-level lineage across every stage. Stores data ownership, glossaries, and data contracts.

Config: admin.email, jwt.secret, backend.postgres.url

DataHub

AlternativeCold

LinkedIn-origin metadata platform. Broader connector library; heavier operational burden.

Config: datahub.gms.host, datahub.token

BI & Visualisation

Dashboards for stakeholders

Metabase

DefaultHot

Easy-to-use BI tool. Stakeholders build dashboards without writing SQL. Best for small-to-medium teams.

Config: admin.email, admin.password, warehouse.url

Apache Superset

AlternativeHot

Advanced BI with SQL Lab, row-level security, and richer chart types. Steeper learning curve.

Config: SECRET_KEY, admin.username, admin.password, SQLALCHEMY_DATABASE_URI

Lightdash

AlternativeHot

BI-as-code driven by dbt models. Metrics live in your dbt project; dashboards sync automatically.

Config: SECRET, PGHOST, PGUSER, DBT_PROJECT_DIR

Database admin UI

Web interface for ad-hoc SQL

pgAdmin

DefaultCold

Full-featured PostgreSQL web UI. Includes ERD, query plans, server monitoring. ~500 MB when running.

Config: PGADMIN_DEFAULT_EMAIL, PGADMIN_DEFAULT_PASSWORD

Adminer

AlternativeCold

Lightweight single-file database UI. Only ~50 MB. Supports PostgreSQL, MySQL, and several others.

Config: ADMINER_DESIGN, ADMINER_PLUGINS

pgweb

AlternativeCold

Minimal Go-based PostgreSQL client. Fastest spin-up time. Read-only query runner.

Config: PGWEB_DATABASE_URL

Not sure which tools fit your team?

Talk to us — we'll help you choose defaults that match your workload.

Talk to us →