blue/docs/rfcs/0001-cross-repo-realms.md

RFC 0001: Cross-Repo Coordination with Realms

Status	Draft
Created	2026-01-24
Source	Spike: cross-repo-coordination
Dialogue	cross-repo-realms.dialogue.md
Refinement	cross-repo-realms-refinement.dialogue.md

---

Problem

We have repositories under different ownership that depend on each other:

• aperture (training-tools webapp) needs S3 access to data in another AWS account
• fungal-image-analysis grants that access via IAM policies

When aperture adds a new S3 path, fungal's IAM policy must update. Currently:

1. No awareness - Blue in aperture doesn't know fungal exists
2. No coordination - Changes require manual cross-repo communication
3. No tracking - No record of cross-repo dependencies

Goals

1. Awareness - Blue sessions know about related repos and their dependencies
2. Coordination - Changes in one repo trigger notifications in dependent repos
3. Trust boundaries - Different orgs can coordinate without shared write access
4. Auditability - All cross-repo coordination is version-controlled

Non-Goals

• Automatic code changes across repos (manual review required)
• Public realm discovery (future scope)
• Monorepo support (for MVP, each repo is a distinct participant)
• Cyclic dependencies between domains

---

Proposal

Hierarchy

Index (~/.blue/index.yaml)
  └── Realm (git repo)
        └── Domain (coordination context)
              ├── Repo A (participant)
              └── Repo B (participant)

Level	Purpose	Storage
Index	List of realms user participates in	~/.blue/index.yaml
Realm	Groups related coordination domains	Git repository
Domain	Coordination context between repos	Directory in realm repo
Repo	Actual code repository (can participate in multiple domains)	.blue/ directory

Key insight: A domain is the relationship (edge), not the thing (node). Repos are nodes; domains are edges connecting them.

         ┌─────────────────────────────────────────┐
         │           Realm: letemcook              │
         │                                         │
         │  ┌─────────────────────────────────┐   │
         │  │    Domain: s3-access            │   │
         │  │                                 │   │
         │  │   ┌──────────┐   ┌──────────┐  │   │
         │  │   │ aperture │◄─►│  fungal  │  │   │
         │  │   │ (export) │   │ (import) │  │   │
         │  │   └──────────┘   └──────────┘  │   │
         │  │                                 │   │
         │  └─────────────────────────────────┘   │
         │                                         │
         │  ┌─────────────────────────────────┐   │
         │  │    Domain: training-pipeline    │   │
         │  │                                 │   │
         │  │   ┌──────────┐   ┌──────────┐  │   │
         │  │   │  fungal  │◄─►│ ml-infra │  │   │
         │  │   │ (export) │   │ (import) │  │   │
         │  │   └──────────┘   └──────────┘  │   │
         │  │                                 │   │
         │  └─────────────────────────────────┘   │
         │                                         │
         └─────────────────────────────────────────┘

Note: fungal participates in both domains with different roles.

---

Architecture

Directory Structure

~/.blue/
├── index.yaml                    # Realms this user participates in
└── credentials.yaml              # Optional, prefer git credentials

$XDG_RUNTIME_DIR/blue/
└── sessions.db                   # Session coordination (SQLite)

realm-{name}/                     # Git repository
├── realm.yaml                    # Metadata, governance, trust
├── repos/
│   └── {repo}.yaml               # Registered repos
└── domains/
    └── {domain}/
        ├── domain.yaml           # Coordination context
        ├── governance.yaml       # Optional domain-level governance
        ├── contracts/
        │   └── {name}.yaml       # Schema, version, validation, owner
        └── bindings/
            └── {repo}.yaml       # Export/import declarations

{repo}/.blue/
├── config.yaml                   # Realm membership, domains
└── cache.db                      # SQLite cache for exports, contracts

realm.yaml

name: letemcook
version: "1.0.0"
created_at: 2026-01-24T10:00:00Z

governance:
  admission: approval  # open | approval | invite-only
  approvers:
    - eric@example.com
  breaking_changes:
    require_approval: true
    grace_period_days: 14

trust:
  mode: collaborative  # collaborative | vendor-customer | federation
  require_signed_commits: false

  permissions:
    repos/{repo}.yaml: [repo_maintainers]
    domains/{domain}/domain.yaml: [domain_owners]
    domains/{domain}/contracts/{name}.yaml: [contract_owner]
    domains/{domain}/bindings/{repo}.yaml: [repo_maintainers]

repos/{repo}.yaml

name: aperture
org: cultivarium  # Optional org prefix
path: /Users/ericg/letemcook/aperture
# Or remote:
# url: git@github.com:cultivarium/aperture.git

maintainers:
  - eric@example.com

joined_at: 2026-01-24T10:00:00Z

domains/{domain}/domain.yaml

name: s3-access
description: Coordinates S3 bucket access between aperture and fungal

created_at: 2026-01-24T10:00:00Z

members:
  - aperture
  - fungal-image-analysis

domains/{domain}/contracts/{contract}.yaml

name: s3-permissions
version: "1.4.0"
owner: aperture  # Only this repo can modify the contract

compatibility:
  backwards: true   # New version readable by old importers
  forwards: false   # Old version NOT readable by new importers

schema:
  type: object
  required: [read]
  properties:
    read:
      type: array
      items: { type: string }
    write:
      type: array
      items: { type: string }
      default: []
    delete:
      type: array
      items: { type: string }
      default: []

value:
  read:
    - "jobs/*/masks/*"
    - "jobs/*/*/config.json"
    - "training-runs/*"
    - "training-metrics/*"
  write:
    - "jobs/*/*/manifest.json"
    - "training-metrics/*"
  delete: []

validation:
  exporter: scripts/validate-s3-paths.sh
  importer: scripts/validate-iam-policy.sh
  ci_only:
    - scripts/integration-test.sh

evolution:
  - version: "1.0.0"
    changes: ["Initial with read array"]
    compatible: true
  - version: "1.3.0"
    changes: ["Added write array"]
    compatible: true
  - version: "1.4.0"
    changes: ["Added delete array"]
    compatible: true

domains/{domain}/bindings/{repo}.yaml

Export binding:

repo: aperture
role: provider

exports:
  - contract: s3-permissions
    source_files:
      - models/training/s3_paths.py
      - models/shared/data/parquet_export.py

Import binding:

repo: fungal-image-analysis
role: consumer

imports:
  - contract: s3-permissions
    version: ">=1.0.0 <2.0.0"  # Semver range
    binding: cdk/training_tools_access_stack.py
    status: current
    resolved_version: "1.4.0"
    resolved_at: 2026-01-24T12:00:00Z

Local Configuration

# aperture/.blue/config.yaml
realm:
  name: letemcook
  path: ../realm-letemcook

repo: aperture

domains:
  - name: s3-access
    role: provider
    contracts:
      - s3-permissions

---

Coordination Model

Blue uses a hybrid coordination model:

1. Real-time hints (SQLite IPC): Fast, best-effort session awareness
2. Durable changes (Git PRs): Source of truth, auditable, PR-based

┌──────────────────────────────────────────────────────────────┐
│                    Coordination Layers                        │
├──────────────────────────────────────────────────────────────┤
│  Real-time Hints (SQLite)                                     │
│  ┌─────────────┐     sessions.db     ┌─────────────┐         │
│  │ Session A   │◄───────────────────►│ Session B   │         │
│  │ (aperture)  │    notifications    │ (fungal)    │         │
│  └─────────────┘                     └─────────────┘         │
│                                                               │
│  Best-effort, sub-second latency, auto-cleanup               │
├──────────────────────────────────────────────────────────────┤
│  Durable Changes (Git)                                        │
│  ┌─────────────┐                     ┌─────────────┐         │
│  │ Repo        │────sync branch─────►│ Realm Repo  │         │
│  │             │◄───PR review────────│             │         │
│  └─────────────┘                     └─────────────┘         │
│                                                               │
│  Source of truth, PR-based, auditable                        │
└──────────────────────────────────────────────────────────────┘

Session Coordination (SQLite)

Sessions register in a shared SQLite database for real-time awareness:

-- $XDG_RUNTIME_DIR/blue/sessions.db

CREATE TABLE sessions (
    id TEXT PRIMARY KEY,
    repo TEXT NOT NULL,
    realm TEXT NOT NULL,
    pid INTEGER,
    started_at TEXT,
    last_heartbeat TEXT,
    active_rfc TEXT,
    active_domains JSON DEFAULT '[]',
    exports_modified JSON DEFAULT '[]',
    imports_watching JSON DEFAULT '[]'
);

CREATE TABLE notifications (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    realm TEXT NOT NULL,
    domain TEXT NOT NULL,
    contract TEXT NOT NULL,
    from_repo TEXT NOT NULL,
    change_type TEXT NOT NULL,  -- 'updated', 'breaking', 'new'
    changes JSON,
    created_at TEXT,
    acknowledged_by JSON DEFAULT '[]'
);

Heartbeat protocol:

• Sessions update last_heartbeat every 10 seconds
• Stale sessions (>30s) are automatically cleaned up
• Crash recovery on startup cleans orphaned sessions

Sync Protocol (Git PRs)

Contract changes go through a PR workflow for durability:

1. blue realm sync creates branch sync/{repo}/{timestamp}
2. Pushes changes to realm repo
3. Creates PR with affected parties as reviewers
4. Merges when:
   • All affected importers acknowledge, OR
   • Grace period expires (for non-breaking changes)
5. On merge, notification broadcast to active sessions

---

Commands

MVP Commands (5)

Command	Purpose
blue realm status	Show realm state, domains, sessions, notifications
blue realm sync	Push local changes, pull remote changes via PR
blue realm worktree	Create linked worktrees across repos in domain
blue realm pr	Create coordinated PRs with merge order
blue realm check	Validate contracts (for CI)

Admin Commands

Under blue realm admin:

Command	Purpose
blue realm admin init	Create a new realm
blue realm admin join	Register repo in realm
blue realm admin domain	Create/manage domains
blue realm admin leave	Remove repo from realm
blue realm admin cache	Cache management

Command Details

blue realm status

$ blue realm status
📊 aperture (in realm: letemcook)

Domains:
  s3-access (provider)
    Contract: s3-permissions v1.4.0

Active sessions:
  • aperture (you) - working on training-metrics-v2
  • fungal-image-analysis - idle

Notifications:
  ⚠️  5 min ago: fungal acknowledged s3-permissions 1.4.0

blue realm sync

$ blue realm sync
📤 Syncing with realm 'letemcook'...

Creating sync branch: sync/aperture/20260124-103000
Contract changes detected:
  s3-permissions: 1.4.0 → 1.5.0
  + training-metrics/experiments/*

Opening PR for review...
  ✓ PR #12: "aperture: s3-permissions 1.5.0"
  Reviewers: fungal-image-analysis maintainers

Notifying active sessions...
  ✓ fungal-image-analysis notified

Waiting for acknowledgment or grace period (14 days for non-breaking).

blue realm check

# Validate exports match code
$ blue realm check --mode=exporter
✓ s3-permissions: code matches contract

# Validate imports are current
$ blue realm check --mode=importer
✓ s3-permissions: binding at 1.4.0 (current)

# Validate contract schema
$ blue realm check --mode=contract
✓ s3-permissions: valid schema, version bump correct

# Check compatibility between versions
$ blue realm check --mode=compatibility --from=1.4.0 --to=1.5.0
Checking s3-permissions 1.4.0 → 1.5.0
  ✓ read: unchanged
  ✓ write: unchanged
  ✓ delete: unchanged
  + experiments: added (optional, default=[]) - COMPATIBLE

Result: COMPATIBLE (minor version bump OK)

---

Caching

Blue uses SQLite for caching to handle concurrent access properly:

-- .blue/cache.db

CREATE TABLE export_cache (
    file_path TEXT PRIMARY KEY,
    mtime INTEGER NOT NULL,
    content_hash TEXT NOT NULL,
    exports JSON,
    cached_at INTEGER NOT NULL
);

CREATE TABLE contract_cache (
    domain TEXT NOT NULL,
    contract TEXT NOT NULL,
    version TEXT NOT NULL,
    schema JSON,
    realm_commit TEXT,
    cached_at INTEGER NOT NULL,
    PRIMARY KEY (domain, contract, version)
);

CREATE TABLE validation_cache (
    contract TEXT NOT NULL,
    version TEXT NOT NULL,
    binding_hash TEXT NOT NULL,
    valid INTEGER NOT NULL,
    output TEXT,
    validated_at INTEGER NOT NULL,
    PRIMARY KEY (contract, version, binding_hash)
);

Invalidation rules:

• Exports: Invalidate when source file mtime changes
• Contracts: Invalidate when realm repo pulls new commits
• Validation: Invalidate when contract version or binding changes

Escape hatches:

blue realm sync --no-cache    # Force fresh detection
blue realm admin cache clear  # Wipe cache
blue realm admin cache stats  # Show cache hit rates

---

CI/CD Integration

GitHub Actions Example

name: Realm Contract Check

on: [push, pull_request]

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/cache@v3
        with:
          path: .blue/cache.db
          key: blue-cache-${{ hashFiles('.blue/config.yaml') }}

      - name: Install Blue
        run: cargo install blue-cli

      - name: Check exports
        run: blue realm check --mode=exporter

      - name: Check imports
        run: blue realm check --mode=importer
        env:
          BLUE_REALM_TOKEN: ${{ secrets.REALM_TOKEN }}

      - name: Check compatibility
        if: github.event_name == 'pull_request'
        run: blue realm check --mode=compatibility

Validation Hooks

Contracts can define validation scripts:

validation:
  exporter: scripts/validate-s3-paths.sh    # Run on export
  importer: scripts/validate-iam-policy.sh  # Run on import
  ci_only:
    - scripts/integration-test.sh           # Only in CI

Exit codes:

• 0: Valid
• 1: Invalid (hard fail)
• 2: Warning (soft fail, can be overridden)

---

Credentials

Blue uses a layered credential approach:

Priority order (first found wins):
1. Environment: BLUE_REALM_TOKEN, BLUE_GITHUB_TOKEN
2. Git credential helper: git credential fill
3. Keychain: macOS Keychain, Linux secret-service
4. Config file: ~/.blue/credentials.yaml (discouraged)

For CI, GitHub App auth is supported:

env:
  BLUE_GITHUB_APP_ID: 12345
  BLUE_GITHUB_APP_PRIVATE_KEY: ${{ secrets.APP_KEY }}
  BLUE_GITHUB_APP_INSTALLATION_ID: 67890

Default behavior: Uses existing git credentials. No additional setup for basic usage.

---

Conflict Resolution

Conflict Type	Detection	Resolution
Concurrent sync	Git merge conflict	Rebase and retry
Same contract, two editors	Ownership check	Reject non-owner
Stale binding	Version mismatch	Soft warning, CI flag
Version incompatibility	Semver check	Hard error
Session collision	Unique ID + heartbeat	Auto-cleanup stale

Contract Ownership

Each contract has an owner field. Only the owner repo can modify the contract:

name: s3-permissions
owner: aperture  # Only aperture can update this contract

Enforced at:

1. blue realm sync: Rejects if non-owner tries to modify
2. Realm repo CI: CODEOWNERS enforces in PRs

---

Cycle Prevention

Cycles between domains are detected and prevented at domain creation:

$ blue realm admin domain create feedback-loop --repos a,b
Error: Adding this domain creates a cycle:
  a → (s3-access) → b → (feedback-loop) → a

Consider:
  - Merging domains into a single coordination context
  - Restructuring the dependency direction
  - Using a hub-and-spoke pattern

Detection uses topological sort on the domain dependency graph.

---

Workflow

Initial Setup

# 1. Create realm (one-time)
$ blue realm admin init --name letemcook
✓ Created realm.yaml
✓ Initialized git repository

# 2. Register aperture in realm
$ cd aperture
$ blue realm admin join ../realm-letemcook
✓ Created repos/aperture.yaml
✓ Auto-detected exports: s3-permissions
✓ Updated .blue/config.yaml

# 3. Create the s3-access domain
$ blue realm admin domain create s3-access \
    --repos aperture,fungal-image-analysis \
    --contract s3-permissions
✓ Created domains/s3-access/domain.yaml
✓ Created domains/s3-access/contracts/s3-permissions.yaml
✓ Created domains/s3-access/bindings/aperture.yaml (provider)
✓ Created domains/s3-access/bindings/fungal-image-analysis.yaml (consumer)

# 4. Register fungal in realm
$ cd ../fungal-image-analysis
$ blue realm admin join ../realm-letemcook
✓ Created repos/fungal-image-analysis.yaml
✓ Detected import: s3-permissions in domain s3-access
✓ Updated .blue/config.yaml

Daily Development

# Developer in aperture adds new S3 path
$ cd aperture
$ vim models/training/metrics_exporter.py
# Added: s3://bucket/training-metrics/experiments/*

# Check status
$ blue realm status
📊 aperture (in realm: letemcook)

⚠️  Contract change detected:
   s3-permissions has local changes:
   + training-metrics/experiments/*

   Affected repos:
   - fungal-image-analysis (imports >=1.0.0 <2.0.0)

   Run 'blue realm sync' to update realm

# Sync changes
$ blue realm sync
📤 Syncing with realm 'letemcook'...

Creating branch: sync/aperture/20260124-150000
Contract update: s3-permissions 1.4.0 → 1.5.0 (compatible)

✓ PR #15 created for review
✓ fungal-image-analysis session notified

Consumer Response

# Maintainer in fungal sees notification
$ blue realm status
📊 fungal-image-analysis (in realm: letemcook)

🔔 Notification from aperture (5 min ago):
   Contract 's3-permissions' updated to 1.5.0
   + training-metrics/experiments/*

   Your import (>=1.0.0 <2.0.0) is still satisfied.
   Update binding when ready.

# Update binding
$ vim cdk/training_tools_access_stack.py
# Add new path to IAM policy

# Sync to acknowledge
$ blue realm sync
📤 Syncing with realm 'letemcook'...

Binding updated:
  s3-permissions: resolved_version 1.4.0 → 1.5.0

✓ Acknowledged PR #15
✓ Realm synced

Coordinated Changes (Unified Worktrees)

# Start cross-repo change
$ blue realm worktree --rfc training-metrics-v2

Creating unified worktree 'feat/training-metrics-v2':
  ✓ aperture: .worktrees/feat-training-metrics-v2/
  ✓ fungal-image-analysis: .worktrees/feat-training-metrics-v2/

Branch 'feat/training-metrics-v2' created in both repos.

# Make changes in both worktrees...

# Create coordinated PRs
$ blue realm pr --title "feat: training metrics v2"

Creating coordinated PRs:
  ✓ aperture PR #45 (merge first - exports)
  ✓ fungal PR #23 (merge second - imports)
    Blocked by: aperture#45

Merge order: aperture#45 → fungal#23

---

Implementation

Phase Overview

Phase	Scope
0	Data model + SQLite schemas
1	blue realm admin init/join
2	blue realm status with realm info
3	blue realm sync with PR workflow
4	blue realm check for CI
5	Session coordination (SQLite IPC)
6	blue realm worktree
7	blue realm pr
8	Caching layer
9	Polish + docs

Phase 0: Data Model

// blue-core/src/realm.rs

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RealmConfig {
    pub name: String,
    pub version: String,
    pub governance: Governance,
    pub trust: TrustConfig,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TrustConfig {
    pub mode: TrustMode,
    pub require_signed_commits: bool,
    pub permissions: HashMap<String, Vec<String>>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Contract {
    pub name: String,
    pub version: String,
    pub owner: String,
    pub compatibility: Compatibility,
    pub schema: serde_json::Value,
    pub value: serde_json::Value,
    pub validation: Option<ValidationConfig>,
    pub evolution: Vec<EvolutionEntry>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Compatibility {
    pub backwards: bool,
    pub forwards: bool,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ValidationConfig {
    pub exporter: Option<String>,
    pub importer: Option<String>,
    pub ci_only: Vec<String>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Binding {
    pub repo: String,
    pub role: BindingRole,
    pub exports: Vec<ExportBinding>,
    pub imports: Vec<ImportBinding>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ImportBinding {
    pub contract: String,
    pub version: String,  // Semver range
    pub binding: String,
    pub status: ImportStatus,
    pub resolved_version: Option<String>,
}

---

Test Plan

Core Operations

• blue realm admin init creates valid realm structure
• blue realm admin join registers repo correctly
• blue realm admin domain create creates domain with contracts
• blue realm admin domain create rejects cycles
• blue realm status shows realm state, domains, notifications
• blue realm sync creates PR for contract changes
• blue realm sync rejects non-owner contract modifications
• blue realm sync --dry-run shows changes without committing

Validation

• blue realm check --mode=exporter validates exports match code
• blue realm check --mode=importer validates bindings
• blue realm check --mode=contract validates schema
• blue realm check --mode=compatibility detects breaking changes
• Validation hooks run with correct exit codes

Session Coordination

• SQLite sessions.db created on Blue start
• Session registered with heartbeat
• Stale sessions cleaned up after 30s
• Contract changes create notifications
• Notifications visible in blue realm status

Caching

• Export cache invalidates on file mtime change
• Contract cache invalidates on realm commit
• --no-cache forces fresh detection
• Cache survives concurrent access

Unified Worktrees

• blue realm worktree creates worktrees in all domain repos
• blue realm pr creates linked PRs with merge order
• Exporters ordered before importers

Conflict Resolution

• Concurrent syncs handled via git rebase
• Ownership violations rejected
• Semver incompatibility flagged as error
• Stale bindings flagged as warning

---

Future Work

1. Signature verification - Repos sign their exports
2. Multiple realms - One repo participates in multiple realms
3. Cross-realm imports - Import from domain in different realm
4. Public registry - Discover realms and contracts
5. Infrastructure verification - Check actual AWS state matches contracts
6. Domain-level governance - Override realm governance per domain

---

References

• Spike: cross-repo-coordination
• Dialogue: cross-repo-realms
• Refinement: cross-repo-realms-refinement