Tech Stack

Docker

Caddy

17+ services running in Docker Compose with Caddy reverse proxy providing automatic HTTPS and clean URL routing to all services.

Prefect

Orchestrates the full pipeline: upload processing, LLM extraction, geocoding, ClickHouse loading, dbt builds, and GIS sync. Includes scheduled polling every 5 minutes.

FastAPI Upload

Pixtral LLM

Nominatim

Multi-channel data collection: CSV ingestion, drag-and-drop letter upload with AI-powered address extraction (Pixtral vision LLM), and self-hosted geocoding via Nominatim.

ClickHouse Bronze

RustFS

Raw data lands in the ClickHouse bronze layer. Uploaded letter files are stored in RustFS, a self-hosted S3-compatible object store.

ClickHouse Silver / Gold

Silver layer provides cleaned and combined staging tables. Gold layer delivers analytics-ready dimensions, facts, and marts with 13+ models for coverage analysis, proximity scoring, and more.

dbt

23+ SQL models transforming raw bronze data through silver staging to gold analytics. Includes 40+ automated data quality tests, incremental models, and custom generic tests.

dbt Docs

Auto-generated data catalog with full lineage graph from bronze to gold. Shows table schemas, column descriptions, test coverage, and model dependencies.

Grafana

ClickHouse Play

CH-UI

Five pre-built Grafana dashboards for API traffic, upload tracking, LLM extraction stats, geocoding health, and GIS monitoring. ClickHouse Play and CH-UI for ad-hoc SQL queries.

Pixtral 12B

Scaleway AI

Mistral's Pixtral 12B vision model hosted on Scaleway extracts street addresses from scanned handwritten letters as structured JSON with confidence scoring.

Tyk Gateway

FastAPI

Redis

REST API with Swagger docs, authenticated via Tyk gateway with API key management and rate limiting (100 req/60s). Serves analytics, GIS endpoints, and letter uploads.

Prometheus

Grafana

Prometheus scrapes FastAPI metrics every 15 seconds. Grafana provides the visualization layer with five pre-built dashboards monitoring the entire platform.

GeoServer

GeoNode

PostGIS

Nominatim

GeoServer publishes WMS/WFS layers for QGIS and ArcGIS. GeoNode provides a data catalog portal. PostGIS stores spatial data synced from ClickHouse. Nominatim handles self-hosted geocoding.

Upload Portal

CH-UI

Apache Hop

Drag-and-drop letter upload portal, full-featured ClickHouse web interface (CH-UI), and Apache Hop visual ETL tool for building pipelines without code.