Technical Architecture

Monorepo Structure

RealHouse is organized as a monorepo with separate packages for data ingestion and the web dashboard, plus shared database migrations and documentation.

realhouse/
├── packages/
│   ├── ingest/                  # Python 3.12+
│   │   ├── pyproject.toml
│   │   ├── src/
│   │   │   ├── connectors/      # One module per data source
│   │   │   │   ├── bps.py       # Census Building Permits Survey
│   │   │   │   ├── nrc_nrs.py   # New Residential Construction/Sales
│   │   │   │   ├── fhfa_hpi.py  # House Price Index
│   │   │   │   ├── fred.py      # FRED API series
│   │   │   │   ├── bls.py       # BLS employment/unemployment
│   │   │   │   ├── census_pop.py # Census PEP population
│   │   │   │   └── hmda.py      # HMDA mortgage originations
│   │   │   ├── models/
│   │   │   │   └── permit_lag.py # Pipeline estimation model
│   │   │   ├── db.py            # Supabase/Postgres connection
│   │   │   └── cli.py           # Click CLI entry point
│   │   └── tests/
│   └── dashboard/               # Next.js 15 App Router
│       ├── package.json
│       ├── src/
│       │   ├── app/
│       │   │   ├── page.tsx             # Landing / market selector
│       │   │   └── market/[cbsa]/
│       │   │       ├── page.tsx         # Market overview (KPI tiles)
│       │   │       ├── permits/page.tsx # Permits detail page
│       │   │       ├── employment/page.tsx  # BLS employment charts
│       │   │       ├── demographics/page.tsx # Census population charts
│       │   │       └── prices/page.tsx  # HPI & mortgage rate charts
│       │   ├── components/
│       │   │   ├── charts/      # Recharts wrappers
│       │   │   └── kpi-card.tsx
│       │   └── lib/
│       │       ├── supabase.ts  # Client setup
│       │       └── queries.ts   # Data fetching
│       └── tailwind.config.ts
├── supabase/
│   ├── migrations/              # 9 SQL migrations
│   └── seed.sql                 # Houston CBSA + geography seed
├── docs/
│   ├── research/                # Primary research documents
│   └── plans/                   # Design docs & implementation plans
├── explainer/                   # This site
└── agent/                       # Agent coordination files

packages/ingest

Python CLI for fetching, parsing, and loading federal data. Uses Click for commands, httpx for HTTP, and psycopg for Postgres. Each connector is a module in src/connectors/ — one file per federal data source (BPS, NRC/NRS, FHFA HPI, FRED, BLS, Census PEP, HMDA).

packages/dashboard

Next.js 15 App Router with Tailwind CSS + shadcn/ui components + Recharts for charts. Queries Supabase directly from server components via the queries.ts module.

supabase/

Database schema defined as SQL migrations + seed data for the Houston CBSA geography (CBSA 26420, including all 9 counties). The 9 migrations create dimension tables, federal fact tables (including BLS employment, Census population, HMDA originations, and FRED series), and the derived mart.

Data Flow

Data moves through four layers: federal source files are parsed by Python connectors, loaded into Supabase tables, transformed by the pipeline model into a mart table, and finally queried by the Next.js dashboard.

Database Schema

The warehouse schema follows a dimensional model: geography dimensions, federal fact tables populated by connectors, and a derived mart table produced by the pipeline model.

Geography Dimension

Federal Fact Tables

Derived Mart

Permit-to-Pipeline Estimation Model

NRC publishes starts, under construction, and completions only at national and Census region level — not at the CBSA level. To estimate metro-level pipeline metrics, we use a distributed lag convolution model calibrated to NRC totals.

1. Starts from Permits

Starts in month t are estimated by convolving permits (P) with a lag distribution (w). The typical permit-to-start lag for single-family construction is 1–3 months.

2. Completions from Starts

Completions are estimated by convolving starts (S) with a start-to-completion distribution (v). The typical start-to-completion duration for single-family is 6–9 months.

3. Under Construction (Stock-Flow Identity)

Under-construction stock follows a stock-flow identity. Each month's stock equals the previous month's stock plus new starts minus completions.

4. Finished Vacant Inventory

Finished vacant inventory equals the previous stock plus completions minus closings. Closings are proxied from HMDA purchase loan originations, which are now wired into the model as of Phase 2 (previously a placeholder).

5. Calibration to NRC Totals

The lag distributions w and v are adjusted so that national and regional aggregates of metro-level estimates match NRC published totals. For the Houston CBSA, Census South region factors are applied.

This calibration step ensures the model is internally consistent: summing all metro estimates within a region reproduces the NRC-reported regional totals.

Known Limitations

• ~9 month warm-up period — the first quarters of modeled output underestimate pipeline activity because the convolution has insufficient lag history.
• Uniform lag assumption — the model assumes the same lag distribution applies to all units within a CBSA, regardless of unit type or builder.
• Will be replaced with real lifecycle data — once TPIA responses arrive with actual permit-to-CO timelines, the distributed lag model will be supplemented (and eventually replaced) by observed local durations.

Key Technical Decisions

Important implementation details discovered during development.

• BPS files use fixed-position format — not standard CSV. The parser uses column indices per Census documentation to extract fields from each row.
• FHFA HPI is quarterly at MSA level — the period field represents a quarter number. Mapped to month-start dates: Q1 → Jan, Q2 → Apr, Q3 → Jul, Q4 → Oct.
• Both imputed and reported BPS rows stored — distinguished by the is_imputed flag. Imputed rows include Census imputation for non-responding places; reported rows include only places that actually responded.
• psycopg.sql for all dynamic SQL — prevents SQL injection by using sql.Identifier() for table and column names. This approach was adopted after code review.
• No error boundary for invalid CBSA codes — known issue. The dashboard does not currently validate the CBSA parameter from the URL, which can lead to empty results or errors for non-existent codes.