superviber/blue
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: RFC 0001 - Cross-repo coordination with realms

Browse source 
Add comprehensive design for cross-repo coordination:

- Spike: Initial problem exploration
- Dialogue: 12-expert, 6-round alignment session
- RFC: Full implementation spec

Key design decisions:
- Realm = git repo (auditable, no new infrastructure)
- Index → Realm → Domain → Repo hierarchy
- Export/import contracts with semver
- Pull-based sync with GitHub issue notifications
- Different orgs can coordinate without shared write access

7-week MVP: init, join, export, import, sync, check, status integration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Eric Garcia 2026-01-24 05:01:08 -05:00
parent 4ecaeea6ad
commit 7e9a466329
3 changed files with 2677 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1754
docs/dialogues/cross-repo-realms.dialogue.md Normal file
View file

File diff suppressed because it is too large Load diff

738
docs/rfcs/0001-cross-repo-realms.md Normal file
View file

@ -0,0 +1,738 @@
# RFC 0001: Cross-Repo Coordination with Realms
| | |
|---|---|
| **Status** | Draft |
| **Created** | 2026-01-24 |
| **Source** | [Spike: cross-repo-coordination](../spikes/cross-repo-coordination.md) |
| **Dialogue** | [cross-repo-realms.dialogue.md](../dialogues/cross-repo-realms.dialogue.md) |
---
## Problem
We have repositories under different ownership that depend on each other:
- `aperture` (training-tools webapp) needs S3 access to data in another AWS account
- `fungal-image-analysis` grants that access via IAM policies
When aperture adds a new S3 path, fungal's IAM policy must update. Currently:
1. No awareness - Blue in aperture doesn't know fungal exists
2. No coordination - Changes require manual cross-repo communication
3. No tracking - No record of cross-repo dependencies
## Goals
1. **Awareness** - Blue sessions know about related repos and their dependencies
2. **Coordination** - Changes in one repo trigger notifications in dependent repos
3. **Trust boundaries** - Different orgs can coordinate without shared write access
4. **Auditability** - All cross-repo coordination is version-controlled
## Non-Goals
- Automatic code changes across repos (manual review required)
- Real-time synchronization (pull-based is sufficient)
- Public realm discovery (future scope)
- Monorepo support (one repo = one domain for MVP)
---
## Proposal
### Hierarchy
```
Index (~/.blue/index.yaml)
└── Realm (git repo)
└── Domain (directory in realm)
└── Repo (.blue/ in code repo)
```
| Level | Purpose | Storage |
|-------|---------|---------|
| **Index** | List of realms user participates in | `~/.blue/index.yaml` |
| **Realm** | Federation of related domains | Git repository |
| **Domain** | Single org's presence in a realm | Directory in realm repo |
| **Repo** | Actual code repository | `.blue/` directory |
### Realm Structure
A realm is a git repository:
```
realm-letemcook/
├── realm.yaml # Realm metadata and governance
├── domains/
│ ├── aperture/
│ │ ├── domain.yaml # Domain metadata
│ │ ├── exports.yaml # What this domain provides
│ │ └── imports.yaml # What this domain consumes
│ └── fungal-image-analysis/
│ ├── domain.yaml
│ ├── exports.yaml
│ └── imports.yaml
├── contracts/
│ └── s3-paths.schema.yaml # Shared schema definitions
└── .github/
└── CODEOWNERS # Domain isolation
```
### realm.yaml
```yaml
name: letemcook
version: "1.0.0"
created_at: 2026-01-24T10:00:00Z
governance:
# Who can add new domains?
admission: approval # open | approval | invite-only
approvers:
- eric@example.com
# Breaking change policy
breaking_changes:
require_approval: true
grace_period_days: 14
```
### domain.yaml
```yaml
name: aperture
repo_path: /Users/ericg/letemcook/aperture
# Or for remote:
# repo_url: git@github.com:cultivarium/aperture.git
maintainers:
- eric@example.com
joined_at: 2026-01-24T10:00:00Z
```
### exports.yaml
```yaml
exports:
- name: required-s3-permissions
version: "1.3.0"
description: S3 paths that aperture training code needs to access
schema:
type: object
properties:
read:
type: array
items: { type: string }
write:
type: array
items: { type: string }
value:
read:
- "jobs/*/masks/*"
- "jobs/*/*/config.json"
- "jobs/*/*/manifest.json"
- "training-runs/*"
- "training-metrics/*"
write:
- "jobs/*/*/manifest.json"
- "training-metrics/*"
changelog:
- version: "1.3.0"
date: 2026-01-24
changes:
- "Added training-metrics/* for experiment tracking"
- version: "1.2.0"
date: 2026-01-20
changes:
- "Added write permissions for manifest updates"
```
### imports.yaml
```yaml
imports:
- contract: required-s3-permissions
from: aperture
version: ">=1.0.0"
binding: cdk/training_tools_access_stack.py
status: current # current | stale | broken
resolved_version: "1.3.0"
resolved_at: 2026-01-24T12:00:00Z
```
### Local Configuration
Each repo stores its realm membership:
```yaml
# aperture/.blue/config.yaml
realm:
name: letemcook
path: ../realm-letemcook # Relative path to realm repo
domain: aperture
```
### Index File
```yaml
# ~/.blue/index.yaml
realms:
- name: letemcook
path: /Users/ericg/letemcook/realm-letemcook
- name: ml-infra
url: git@github.com:org/realm-ml-infra.git
local_path: /Users/ericg/.blue/realms/ml-infra
```
---
## Workflow
### Initial Setup
```bash
# 1. Create realm (one-time)
$ mkdir realm-letemcook && cd realm-letemcook
$ blue realm init --name letemcook
✓ Created realm.yaml
✓ Initialized git repository
# 2. Add aperture to realm
$ cd ../aperture
$ blue realm join ../realm-letemcook --as aperture
✓ Created domains/aperture/domain.yaml
✓ Auto-detected exports: required-s3-permissions
✓ Created domains/aperture/exports.yaml
✓ Updated .blue/config.yaml
# 3. Add fungal to realm
$ cd ../fungal-image-analysis
$ blue realm join ../realm-letemcook --as fungal-image-analysis
✓ Created domains/fungal-image-analysis/domain.yaml
✓ Detected import: required-s3-permissions from aperture
✓ Created domains/fungal-image-analysis/imports.yaml
```
### Daily Development
```bash
# Developer in aperture adds new S3 path
$ cd aperture
$ vim models/training/metrics_exporter.py
# Added: s3://bucket/training-metrics/experiments/*
# Blue status shows cross-realm impact
$ blue status
📊 aperture (domain in letemcook realm)
RFCs:
training-metrics-v2 [in-progress] 3/5 tasks
⚠️ Cross-realm change detected:
Export 'required-s3-permissions' has local changes:
+ training-metrics/experiments/*
Consumers:
- fungal-image-analysis (imports >=1.0.0)
Run 'blue realm sync' to update realm
# Sync changes to realm
$ blue realm sync
📤 Syncing with realm 'letemcook'...
Exports updated:
required-s3-permissions: 1.3.0 → 1.4.0
+ training-metrics/experiments/*
Notifying consumers:
fungal-image-analysis:
✓ Created GitHub issue #42
"Update IAM policy: new S3 path training-metrics/experiments/*"
✓ Realm synced (commit abc1234)
```
### Consumer Response
```bash
# Maintainer in fungal sees notification
$ cd fungal-image-analysis
$ blue status
📊 fungal-image-analysis (domain in letemcook realm)
⚠️ Stale imports:
required-s3-permissions: 1.3.0 → 1.4.0 available
Binding: cdk/training_tools_access_stack.py
Changes:
+ training-metrics/experiments/* (read/write)
# Update the IAM policy
$ vim cdk/training_tools_access_stack.py
# Add new path to policy
# Mark import as resolved
$ blue realm sync
📤 Syncing with realm 'letemcook'...
Imports resolved:
required-s3-permissions: now at 1.4.0
✓ Realm synced (commit def5678)
```
---
## New Tools
### blue_realm_init
Create a new realm.
```
blue_realm_init
--name: Realm name (kebab-case)
--path: Where to create realm repo (default: current directory)
```
### blue_realm_join
Join a repository to a realm as a domain.
```
blue_realm_join
realm_path: Path to realm repo
--as: Domain name (default: repo directory name)
--detect-exports: Auto-detect exports (default: true)
```
### blue_realm_leave
Remove domain from realm.
```
blue_realm_leave
--force: Leave even if other domains import from us
```
### blue_realm_export
Declare or update an export.
```
blue_realm_export
--name: Export name
--version: Semantic version
--value: JSON value (or --file for YAML file)
--detect: Auto-detect from code patterns
```
### blue_realm_import
Declare an import dependency.
```
blue_realm_import
--contract: Contract name
--from: Source domain
--version: Version requirement (semver)
--binding: Local file that uses this import
```
### blue_realm_sync
Synchronize local state with realm.
```
blue_realm_sync
--dry-run: Show what would change without committing
--notify: Create GitHub issues for affected consumers (default: true)
```
### blue_realm_check
Check realm status without syncing.
```
blue_realm_check
--exports: Check if local exports have changed
--imports: Check if any imports are stale
```
### blue_realm_graph
Display the dependency graph.
```
blue_realm_graph
--format: text | mermaid | dot
```
---
## Implementation
### Phase 0: Data Model (Week 1)
Add to `blue-core/src/`:
```rust
// realm.rs
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RealmConfig {
pub name: String,
pub version: String,
pub governance: Governance,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Governance {
pub admission: AdmissionPolicy,
pub approvers: Vec<String>,
pub breaking_changes: BreakingChangePolicy,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum AdmissionPolicy {
Open,
Approval,
InviteOnly,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BreakingChangePolicy {
pub require_approval: bool,
pub grace_period_days: u32,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Domain {
pub name: String,
pub repo_path: Option<PathBuf>,
pub repo_url: Option<String>,
pub maintainers: Vec<String>,
pub joined_at: String,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Export {
pub name: String,
pub version: String,
pub description: Option<String>,
pub schema: Option<serde_json::Value>,
pub value: serde_json::Value,
pub changelog: Vec<ChangelogEntry>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ChangelogEntry {
pub version: String,
pub date: String,
pub changes: Vec<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Import {
pub contract: String,
pub from: String,
pub version: String, // semver requirement
pub binding: String,
pub status: ImportStatus,
pub resolved_version: Option<String>,
pub resolved_at: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum ImportStatus {
Current,
Stale,
Broken,
}
```
### Phase 1: Realm Init (Week 2)
```rust
// handlers/realm.rs
pub fn handle_init(args: &Value) -> Result<Value, ServerError> {
let name = args.get("name").and_then(|v| v.as_str())
.ok_or(ServerError::InvalidParams)?;
let path = args.get("path")
.and_then(|v| v.as_str())
.map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from("."));
// Create realm directory
let realm_path = path.join(format!("realm-{}", name));
fs::create_dir_all(&realm_path)?;
// Create realm.yaml
let config = RealmConfig {
name: name.to_string(),
version: "1.0.0".to_string(),
governance: Governance {
admission: AdmissionPolicy::Approval,
approvers: vec![],
breaking_changes: BreakingChangePolicy {
require_approval: true,
grace_period_days: 14,
},
},
};
let yaml = serde_yaml::to_string(&config)?;
fs::write(realm_path.join("realm.yaml"), yaml)?;
// Create directories
fs::create_dir_all(realm_path.join("domains"))?;
fs::create_dir_all(realm_path.join("contracts"))?;
// Initialize git
Command::new("git")
.args(["init"])
.current_dir(&realm_path)
.output()?;
Ok(json!({
"status": "success",
"message": format!("Created realm '{}' at {}", name, realm_path.display()),
"path": realm_path.display().to_string()
}))
}
```
### Phase 2: Domain Join (Week 3)
```rust
pub fn handle_join(args: &Value, repo_path: &Path) -> Result<Value, ServerError> {
let realm_path = args.get("realm_path")
.and_then(|v| v.as_str())
.map(PathBuf::from)
.ok_or(ServerError::InvalidParams)?;
let domain_name = args.get("as")
.and_then(|v| v.as_str())
.map(String::from)
.unwrap_or_else(|| {
repo_path.file_name()
.and_then(|n| n.to_str())
.unwrap_or("unknown")
.to_string()
});
// Validate realm exists
let realm_yaml = realm_path.join("realm.yaml");
if !realm_yaml.exists() {
return Err(ServerError::NotFound("Realm not found".into()));
}
// Create domain directory
let domain_dir = realm_path.join("domains").join(&domain_name);
fs::create_dir_all(&domain_dir)?;
// Create domain.yaml
let domain = Domain {
name: domain_name.clone(),
repo_path: Some(repo_path.to_path_buf()),
repo_url: None,
maintainers: vec![],
joined_at: chrono::Utc::now().to_rfc3339(),
};
fs::write(
domain_dir.join("domain.yaml"),
serde_yaml::to_string(&domain)?
)?;
// Auto-detect exports
let exports = detect_exports(repo_path)?;
if !exports.is_empty() {
fs::write(
domain_dir.join("exports.yaml"),
serde_yaml::to_string(&json!({ "exports": exports }))?
)?;
}
// Update repo's .blue/config.yaml
let blue_dir = repo_path.join(".blue");
fs::create_dir_all(&blue_dir)?;
let config = json!({
"realm": {
"name": realm_name,
"path": realm_path.display().to_string(),
"domain": domain_name
}
});
fs::write(blue_dir.join("config.yaml"), serde_yaml::to_string(&config)?)?;
// Commit to realm
Command::new("git")
.args(["add", "."])
.current_dir(&realm_path)
.output()?;
Command::new("git")
.args(["commit", "-m", &format!("Add domain: {}", domain_name)])
.current_dir(&realm_path)
.output()?;
Ok(json!({
"status": "success",
"message": format!("Joined realm as '{}'", domain_name),
"exports_detected": exports.len()
}))
}
```
### Phase 3: Status Integration (Week 4)
Modify `handle_status` to include realm information:
```rust
fn get_realm_status(state: &ProjectState) -> Option<Value> {
let config_path = state.repo_path.join(".blue/config.yaml");
let config: Value = serde_yaml::from_str(
&fs::read_to_string(&config_path).ok()?
).ok()?;
let realm_path = config["realm"]["path"].as_str()?;
let domain_name = config["realm"]["domain"].as_str()?;
// Check for local export changes
let local_exports = detect_exports(&state.repo_path).ok()?;
let declared_exports = load_declared_exports(realm_path, domain_name).ok()?;
let export_changes = diff_exports(&local_exports, &declared_exports);
// Check for stale imports
let imports = load_imports(realm_path, domain_name).ok()?;
let stale_imports = check_import_staleness(realm_path, &imports);
Some(json!({
"realm": config["realm"]["name"],
"domain": domain_name,
"export_changes": export_changes,
"stale_imports": stale_imports
}))
}
```
### Phase 4: Sync (Weeks 5-6)
```rust
pub fn handle_sync(args: &Value, state: &ProjectState) -> Result<Value, ServerError> {
let dry_run = args.get("dry_run").and_then(|v| v.as_bool()).unwrap_or(false);
let notify = args.get("notify").and_then(|v| v.as_bool()).unwrap_or(true);
let config = load_realm_config(&state.repo_path)?;
let realm_path = PathBuf::from(&config.path);
// Pull latest realm state
git_pull(&realm_path)?;
// Detect export changes
let local_exports = detect_exports(&state.repo_path)?;
let declared_exports = load_declared_exports(&realm_path, &config.domain)?;
let changes = diff_exports(&local_exports, &declared_exports);
if changes.is_empty() {
return Ok(json!({
"status": "success",
"message": "No changes to sync"
}));
}
if dry_run {
return Ok(json!({
"status": "dry_run",
"changes": changes
}));
}
// Update exports in realm
let new_version = bump_version(&declared_exports[0].version, &changes);
save_exports(&realm_path, &config.domain, &local_exports, &new_version)?;
// Commit and push
git_commit(&realm_path, &format!(
"{}: export {}@{}",
config.domain, local_exports[0].name, new_version
))?;
git_push(&realm_path)?;
// Find affected consumers
let consumers = find_consumers(&realm_path, &local_exports[0].name)?;
// Create notifications
let mut notifications = vec![];
if notify {
for consumer in &consumers {
let issue = create_github_issue(consumer, &changes)?;
notifications.push(issue);
}
}
Ok(json!({
"status": "success",
"message": format!("Synced {} export(s)", changes.len()),
"new_version": new_version,
"notifications": notifications
}))
}
```
### Phase 5: Polish (Week 7)
- Error handling for all edge cases
- User-friendly messages in Blue's voice
- Documentation
- Tests
---
## Test Plan
- [ ] `blue realm init` creates valid realm structure
- [ ] `blue realm join` registers domain correctly
- [ ] `blue realm join` auto-detects S3 path exports
- [ ] `blue status` shows realm state
- [ ] `blue status` detects local export changes
- [ ] `blue status` shows stale imports
- [ ] `blue realm sync` updates exports in realm
- [ ] `blue realm sync` creates GitHub issues
- [ ] `blue realm sync --dry-run` shows changes without committing
- [ ] Multiple domains can coordinate through realm
- [ ] Breaking changes are flagged appropriately
- [ ] CODEOWNERS prevents cross-domain writes
---
## Future Work
1. **Signature verification** - Domains sign their exports
2. **Multiple realms** - One repo participates in multiple realms
3. **Cross-realm imports** - Import from domain in different realm
4. **Public registry** - Discover realms and contracts
5. **Infrastructure verification** - Check actual AWS state matches exports
6. **Automatic PR creation** - Generate code changes, not just issues
---
## References
- [Spike: cross-repo-coordination](../spikes/cross-repo-coordination.md)
- [Dialogue: cross-repo-realms](../dialogues/cross-repo-realms.dialogue.md)

185
docs/spikes/cross-repo-coordination.md Normal file
View file

@ -0,0 +1,185 @@
# Spike: Cross-Repo Coordination
| | |
|---|---|
| **Status** | Complete |
| **Outcome** | Recommends Implementation |
| **RFC** | [0001-cross-repo-realms](../rfcs/0001-cross-repo-realms.md) |
| **Question** | How can Blue sessions in different repos be aware of each other and coordinate changes when repos have dependencies? |
| **Time Box** | 2 hours |
| **Started** | 2026-01-24 |
---
## Context
We have repos with cross-repo dependencies:
- `aperture` (training-tools webapp) - runs in Account A
- `fungal-image-analysis` - runs in Account B, grants IAM access to Account A
When changes are made in one repo (e.g., adding a new S3 path pattern), the corresponding changes must be made in the other (e.g., updating IAM policies).
### Current Pain Points
1. **No awareness** - Blue session in repo A doesn't know repo B exists
2. **No dependency graph** - Changes to IAM policies don't trigger awareness of dependent services
3. **Manual coordination** - Developer must remember to update both repos
4. **Planning blindness** - RFCs in one repo can't reference or depend on RFCs in another
---
## Research Areas
### 1. Dependency Declaration
How do we declare cross-repo dependencies?
**Option A: Blue manifest file**
```yaml
# .blue/manifest.yaml
dependencies:
- repo: ../fungal-image-analysis
type: infrastructure
resources:
- cdk/training_tools_access_stack.py
```
**Option B: In-document links**
```markdown
<!-- In RFC -->
| **Cross-Repo** | [fungal-image-analysis](../fungal-image-analysis) |
```
**Option C: Centralized registry**
```
# ~/.blue/repos.yaml (or domain-level DB)
repos:
aperture:
path: /Users/ericg/letemcook/aperture
depends_on: [fungal-image-analysis]
fungal-image-analysis:
path: /Users/ericg/letemcook/fungal-image-analysis
depended_by: [aperture]
```
### 2. Session Coordination
How do Blue sessions communicate?
**Option A: Shared SQLite (domain store)**
- All repos in a domain share a single `.data/domain.db`
- Sessions register themselves and their active RFCs
- Can query "who else is working on related changes?"
**Option B: File-based signals**
- Write `.blue/active-session.json` with current work
- Other sessions poll or watch for changes
**Option C: IPC/Socket**
- Blue MCP server listens on a socket
- Sessions can query each other directly
- More complex but real-time
### 3. Change Propagation
When a change is made in repo A that affects repo B, what happens?
**Option A: Manual notification**
```
⚠️ This change affects dependent repo: fungal-image-analysis
- cdk/training_tools_access_stack.py may need updates
Run: blue_cross_repo_check
```
**Option B: Automatic RFC creation**
- Detect affected files via dependency graph
- Create draft RFC in dependent repo
- Link the RFCs together
**Option C: Unified worktree**
- Create worktrees in both repos simultaneously
- Single branch name spans repos
- Coordinate commits
### 4. Planning Integration
How do cross-repo RFCs work together?
**Requirements:**
- RFC in repo A can declare dependency on RFC in repo B
- Status changes propagate (can't implement A until B is accepted)
- Plan tasks can span repos
**Proposal:**
```markdown
| **Depends On** | fungal-image-analysis:rfc-0060-cross-account-access |
| **Blocks** | aperture:rfc-0045-training-metrics |
```
---
## Findings
### Key Insight: Domain-Level Store
The cleanest solution is a **domain-level store** that sits above individual repos:
```
~/.blue/domains/
letemcook/
domain.db # Cross-repo coordination
repos.yaml # Repo registry
sessions/ # Active sessions
```
This enables:
1. Single source of truth for repo relationships
2. Cross-repo RFC dependencies
3. Session awareness without IPC complexity
4. Centralized audit of cross-repo changes
### Proposed Architecture
```
┌─────────────────────────────────────────────────────────┐
│ Domain Store │
│ ~/.blue/domains/letemcook/domain.db │
│ - repos table (path, name, dependencies) │
│ - cross_repo_links table (source_rfc, target_rfc) │
│ - active_sessions table (repo, rfc, agent_id) │
└─────────────────────────────────────────────────────────┘
│ │
▼ ▼
┌─────────────────────┐ ┌─────────────────────┐
│ aperture │ │ fungal-image-analysis│
│ .blue/blue.db │ │ .blue/blue.db │
│ docs/rfcs/ │ │ docs/rfcs/ │
└─────────────────────┘ └─────────────────────┘
```
### New Tools Needed
1. `blue_domain_init` - Initialize domain, register repos
2. `blue_domain_link` - Link two repos as dependencies
3. `blue_cross_repo_check` - Check if changes affect other repos
4. `blue_cross_repo_rfc` - Create linked RFCs across repos
---
## Outcome
**Recommendation:** Implement domain-level store with cross-repo RFC linking.
**Next Steps:**
1. Design domain store schema (new RFC)
2. Add domain detection to Blue startup
3. Implement cross-repo RFC dependencies
4. Add change impact detection
---
## Notes
- Start simple: just repo registry + session awareness
- Don't over-engineer IPC - polling shared DB is sufficient
- Consider git worktree naming conventions that span repos