August 9, 2026
A dbt Semantic Layer Over My Own GitHub Activity
A standalone data pipeline — ingest, dbt transformations, a real semantic layer, and an open-source BI tool on top — built specifically to show the ingest-to-insight path, not just a dashboard.
9
dbt models across staging + marts
6
metrics defined in the semantic layer
Live from the pipeline
Queried from the same Postgres marts this entry describes, on every page load — not a snapshot pasted in once. Last ingested 13 minutes ago.
3,091
Commits
29
Active repos
5
Pull requests
80%
PR merge rate
0
Issues
—
Issue close rate
0.1h
Avg PR cycle time
Commit activity, last 12 months
Queried from marts.fct_commits, grouped by day — the same table the stats above come from.
Problem
Most portfolio "data pipeline" projects are a script that dumps an API into a table and calls it done — they show ingestion, but nothing about the transformation discipline or the layer that makes numbers mean the same thing everywhere they're used. I wanted something that actually demonstrated the full path: raw data in, a modeled semantic layer with real metric definitions, and a BI tool sitting on top of that layer rather than querying raw tables directly.
Approach
A standalone repo (deliberately decoupled from this site's own codebase): a Python script ingests repos, commits, pull requests, and issues from the GitHub REST API into a Postgres "raw" schema — idempotent upserts, paginated, rate-limit aware, reusing the same auth pattern this site's own GitHub digest already uses. dbt takes it from there: staging views, then marts (dim_repos, dim_dates, fct_commits, fct_pull_requests, fct_issues), then a native dbt Semantic Layer — semantic_models and metrics defined once in YAML, not scattered across dashboard configs. Six metrics: commits per week, active repos, PR merge rate, average PR cycle time, issue close rate, average time to close. Lightdash sits on top as the BI layer specifically because it reads dbt's semantic model definitions directly, rather than requiring metrics to be redefined a second time in the BI tool. Scheduling is GitHub Actions in the pipeline's own repo, not this site's Vercel Cron — kept fully decoupled. One honest technical note worth flagging: dbt's Semantic Layer runs on MetricFlow, which is source-available (BSL) rather than OSI-approved open source. Free to self-host, which is what this does, but not the same license class as dbt-core itself or Lightdash.
Outcome
The ingestion script and full dbt project (staging, marts, six-metric semantic layer, tests) are written and the repo is public. The repo's GitHub Actions workflow runs on a schedule for ingestion run,. Lightdash reads from the semantic layer.