Proposed Action: Increase memory limit
  from: 512Mi
  to: 1Gi

Dry-Run Results:
   Manifest validation: PASS
   Resource quota check: PASS (within namespace limits)
   Pod disruption budget: PASS (1/3 pods, within budget)
   Dependency impact: None detected
   Rollback plan: Generated
  
Predicted Outcome:
  - Pod will restart (estimated 8-12 seconds downtime)
  - Memory usage will normalize
  - No cascading effects expected
  
Confidence: 96%

12:00: Auto-fix applied to pod A
12:01: Pod A crashes again
12:02: Auto-fix applied to pod A (attempt #2)
12:03: Pod A crashes again
12:04: BLOCKED - Max retries reached
      → Alert SRE team
      → Disable auto-fix for this pod
      → Manual intervention required

# Automatic rollback triggers if:
- Health check fails after change
- Error rate increases >50%
- Latency spikes >200%
- Pod crash rate increases

# Manual rollback available:
Click "Rollback" button in dashboard
OR: Run `rubixkube rollback <action-id>`

{
  "action_id": "fix-2025-10-03-abc123",
  "timestamp": "2025-10-03T14:30:00Z",
  "agent": "remediation-agent-v2",
  "trigger": "automated",
  "approved_by": "system (auto-approval policy)",
  
  "target": {
    "cluster": "prod-us-east",
    "namespace": "checkout",
    "resource": "deployment/checkout-service"
  },
  
  "action": {
    "type": "update_resource_limits",
    "changes": {
      "memory.limits": "512Mi → 1Gi"
    }
  },
  
  "context": {
    "incident_id": "inc-2025-10-03-xyz789",
    "root_cause": "OOMKilled",
    "confidence": 0.96,
    "blast_radius": "single_deployment"
  },
  
  "outcome": {
    "status": "success",
    "duration": "8.2s",
    "verification": "pod_healthy",
    "rollback_available": true
  }
}

guardrails:
  policies:
    - name: "Critical Services"
      resources:
        - "production/payment-*"
        - "production/auth-*"
      rules:
        auto_fix: false
        require_approval: always
        require_2fa: true
    
    - name: "Background Jobs"
      resources:
        - "production/batch-*"
        - "production/cron-*"
      rules:
        auto_fix: true
        max_blast_radius: deployment
        approval: none

def calculate_blast_radius(action):
    scope = action.target_scope
    dependencies = get_dependencies(scope)
    
    radius = {
        "direct_impact": count_affected_resources(scope),
        "dependency_impact": count_dependent_resources(dependencies),
        "user_impact": estimate_affected_users(scope),
        "revenue_impact": calculate_potential_revenue_loss(scope)
    }
    
    if radius.direct_impact > 10 or radius.user_impact > 1000:
        return "HIGH_RISK - Require approval"
    elif radius.direct_impact > 3 or radius.user_impact > 100:
        return "MEDIUM_RISK - Suggest with review"
    else:
        return "LOW_RISK - Auto-approve possible"

Action: Restart pod 'api-gateway-7f9d'

Blast Radius Analysis:
  Direct Impact: 1 pod (out of 5 replicas)
  Dependency Impact: 0 (no dependent services fail)
  User Impact: 20% of traffic (during rolling update)
  Revenue Impact: $15/minute × 0.5min = $7.50
  
  Classification: LOW RISK
  Decision: Auto-approved

observer_agent:
  resource_limits:
    cpu: 500m (max 0.5 CPU cores)
    memory: 512Mi (max 512 megabytes)
    api_calls: 100/minute (to Kubernetes API)
  
  if_limits_exceeded:
    - throttle_monitoring_frequency
    - skip_non_critical_checks
    - alert_if_persistent

# What Observer CAN do (read-only):
apiGroups: ["", "apps", "batch"]
resources: ["pods", "deployments", "jobs", "services"]
verbs: ["get", "list", "watch"]

# What Observer CANNOT do:
verbs: ["create", "update", "delete", "patch"]
# Remediation happens through controlled API, not direct cluster access

Incident: API service memory leak
Remediation Agent Proposal: "Restart all pods simultaneously for quick fix"

Guardian Agent Analysis:
   Blast radius: ENTIRE SERVICE (5 pods)
   Downtime: 100% of capacity during restart
   User impact: ~5000 users affected
   Risk: HIGH

Decision: BLOCKED

Alternative Proposal: "Rolling restart (1 pod at a time)"
Guardian Agent Analysis:
   Blast radius: 20% of capacity per step
   Downtime: Zero (4 pods serve during restart)
   User impact: Minimal (slight latency)
   Risk: LOW

Decision: APPROVED

Situation: Production database pod stuck, auto-fix blocked (high risk)
SRE Assessment: "We need to restart NOW, outage ongoing"

Manual Override:
  1. SRE clicks "Override Guardrails"
  2. System requires: 
     - Incident ticket number
     - Override justification
     - 2FA confirmation
  3. Action executes with OVERRIDE flag in audit log
  4. Post-incident review required

Result: Fast response when humans decide risk is acceptable

# Example API configuration
curl -X POST https://api.rubixkube.ai/v2/guardrails/config \
  -H "Authorization: Bearer $API_KEY" \
  -d '{
    "environment": "production",
    "auto_fix_enabled": true,
    "auto_fix_scope": "pod_level_only",
    "require_approval": ["medium_risk", "high_risk"],
    "circuit_breaker_threshold": 3,
    "circuit_breaker_window": "30m"
  }'

Memory Engine:
  "This fix worked 94% of the time for this issue"
  
Agent Mesh:
  "I propose we apply that fix now"
  
Guardrails:
  "Checking safety...  Scope OK,  Risk LOW,  Policy allows"
  "APPROVED - Execute with logging and rollback ready"
  
Outcome: Fast, safe, autonomous remediation