g8s·lab
g8s·lab · Pipeline Product

Automated Data Pipeline

The Problem
  • Manual data entry from multiple sources
  • Inconsistent data quality — no audit trail
  • Reporting takes hours every month
  • No standardized impact metrics
The Solution
  • One config file drives the entire pipeline
  • Rule-based + Claude AI data audit
  • Always-on impact dashboard
  • Any org — zero code changes to deploy
Architecture

One config file. Six components.

config.yaml
→
Source Adapters
→
Source Mapper
→
Data Auditor
→
Data Store
→
Dashboard
Source Adapters
CSV, XLSX, SQLite · uniform row dicts · encoding + header handling
Source Mapper
Maps source columns → schema fields · flags unmapped / missing
Data Auditor
Layer 1: deterministic rules · Layer 2: Claude AI semantic checks · severity-ranked issues · quarantine
Data Store
SQLite SSOT · quarantine table · audit log · all ETL performed here
Dashboard
Executive KPIs · operational charts · full case detail · export/print · Streamlit v1
Workflow Orchestrator
Cron scheduling · run history · failure alerts · dashboard refresh
Proof of Concept · MTF

Built for Musician Treatment Foundation

Demo 1 — Data Pipeline · Built 2026-05-26

Musician Treatment Foundation · Impact Dashboard

  • 15 rule types + Claude AI semantic audit
  • 6-KPI executive dashboard (Streamlit)
  • Monthly + YTD reporting, print-ready
  • Quarantine for critical data quality issues
✓ MTF Demo 1 shipped
✓ MTF Demo 2 shipped
⟳ Architecture design (Problem 1)
Config layer
CLI + package
1 / 3

Speaker Notes