Table of Contents
Environment Creation Flow
This document illustrates the complete environment creation flow in the Blueberry IDP, from initial request to ready environment.
Environment Creation Overview
graph TB
subgraph "Trigger Sources"
WEB_UI[Web UI
Manual Creation] CI_CD[CI/CD Pipeline
Automated] WEBHOOK[Webhook
Git Events] API_CALL[Direct API
Scripts/Tools] end subgraph "Request Processing" AUTH[Authentication
& Authorization] VALIDATE[Request
Validation] NAME_CHECK[Name Uniqueness
Check] QUOTA[Resource Quota
Check] end subgraph "Environment Service" ENV_CREATE[Environment
Creator Service] CONFIG_MERGE[Configuration
Merging] TEMPLATE_GEN[Template
Generation] METADATA[Metadata
Storage] end subgraph "GitOps Layer" ARGOCD_APP[ArgoCD Application
Creation] GIT_SYNC[Git Repository
Sync] MANIFEST_APPLY[Kubernetes
Manifest Apply] end subgraph "Kubernetes Layer" NAMESPACE[Namespace
Creation] RBAC[RBAC
Setup] NETWORK_POL[Network
Policies] HELM_DEPLOY[Helm Chart
Deployment] end subgraph "Infrastructure" PODS[Pod
Deployment] SERVICES[Service
Creation] INGRESS[Ingress
Configuration] SECRETS[Secret
Provisioning] end subgraph "Monitoring & Status" READINESS[Readiness
Checks] STATUS_UPDATE[Status
Updates] NOTIFICATION[Notifications
& Webhooks] URL_READY[URL
Availability] end %% Flow connections WEB_UI --> AUTH CI_CD --> AUTH WEBHOOK --> AUTH API_CALL --> AUTH AUTH --> VALIDATE VALIDATE --> NAME_CHECK NAME_CHECK --> QUOTA QUOTA --> ENV_CREATE ENV_CREATE --> CONFIG_MERGE CONFIG_MERGE --> TEMPLATE_GEN TEMPLATE_GEN --> METADATA METADATA --> ARGOCD_APP ARGOCD_APP --> GIT_SYNC GIT_SYNC --> MANIFEST_APPLY MANIFEST_APPLY --> NAMESPACE NAMESPACE --> RBAC RBAC --> NETWORK_POL NETWORK_POL --> HELM_DEPLOY HELM_DEPLOY --> PODS PODS --> SERVICES SERVICES --> INGRESS INGRESS --> SECRETS SECRETS --> READINESS READINESS --> STATUS_UPDATE STATUS_UPDATE --> NOTIFICATION NOTIFICATION --> URL_READY %% Status feedback loops READINESS -.->|"Status Updates"| ENV_CREATE STATUS_UPDATE -.->|"Progress"| WEB_UI STATUS_UPDATE -.->|"Progress"| CI_CD classDef trigger fill:#e3f2fd,stroke:#1976d2,stroke-width:2px classDef process fill:#fff3e0,stroke:#f57c00,stroke-width:2px classDef service fill:#e8f5e9,stroke:#388e3c,stroke-width:2px classDef gitops fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px classDef k8s fill:#fce4ec,stroke:#c2185b,stroke-width:2px classDef infra fill:#e0f2f1,stroke:#00695c,stroke-width:2px classDef monitor fill:#fff8e1,stroke:#f9a825,stroke-width:2px class WEB_UI,CI_CD,WEBHOOK,API_CALL trigger class AUTH,VALIDATE,NAME_CHECK,QUOTA process class ENV_CREATE,CONFIG_MERGE,TEMPLATE_GEN,METADATA service class ARGOCD_APP,GIT_SYNC,MANIFEST_APPLY gitops class NAMESPACE,RBAC,NETWORK_POL,HELM_DEPLOY k8s class PODS,SERVICES,INGRESS,SECRETS infra class READINESS,STATUS_UPDATE,NOTIFICATION,URL_READY monitor
Manual Creation] CI_CD[CI/CD Pipeline
Automated] WEBHOOK[Webhook
Git Events] API_CALL[Direct API
Scripts/Tools] end subgraph "Request Processing" AUTH[Authentication
& Authorization] VALIDATE[Request
Validation] NAME_CHECK[Name Uniqueness
Check] QUOTA[Resource Quota
Check] end subgraph "Environment Service" ENV_CREATE[Environment
Creator Service] CONFIG_MERGE[Configuration
Merging] TEMPLATE_GEN[Template
Generation] METADATA[Metadata
Storage] end subgraph "GitOps Layer" ARGOCD_APP[ArgoCD Application
Creation] GIT_SYNC[Git Repository
Sync] MANIFEST_APPLY[Kubernetes
Manifest Apply] end subgraph "Kubernetes Layer" NAMESPACE[Namespace
Creation] RBAC[RBAC
Setup] NETWORK_POL[Network
Policies] HELM_DEPLOY[Helm Chart
Deployment] end subgraph "Infrastructure" PODS[Pod
Deployment] SERVICES[Service
Creation] INGRESS[Ingress
Configuration] SECRETS[Secret
Provisioning] end subgraph "Monitoring & Status" READINESS[Readiness
Checks] STATUS_UPDATE[Status
Updates] NOTIFICATION[Notifications
& Webhooks] URL_READY[URL
Availability] end %% Flow connections WEB_UI --> AUTH CI_CD --> AUTH WEBHOOK --> AUTH API_CALL --> AUTH AUTH --> VALIDATE VALIDATE --> NAME_CHECK NAME_CHECK --> QUOTA QUOTA --> ENV_CREATE ENV_CREATE --> CONFIG_MERGE CONFIG_MERGE --> TEMPLATE_GEN TEMPLATE_GEN --> METADATA METADATA --> ARGOCD_APP ARGOCD_APP --> GIT_SYNC GIT_SYNC --> MANIFEST_APPLY MANIFEST_APPLY --> NAMESPACE NAMESPACE --> RBAC RBAC --> NETWORK_POL NETWORK_POL --> HELM_DEPLOY HELM_DEPLOY --> PODS PODS --> SERVICES SERVICES --> INGRESS INGRESS --> SECRETS SECRETS --> READINESS READINESS --> STATUS_UPDATE STATUS_UPDATE --> NOTIFICATION NOTIFICATION --> URL_READY %% Status feedback loops READINESS -.->|"Status Updates"| ENV_CREATE STATUS_UPDATE -.->|"Progress"| WEB_UI STATUS_UPDATE -.->|"Progress"| CI_CD classDef trigger fill:#e3f2fd,stroke:#1976d2,stroke-width:2px classDef process fill:#fff3e0,stroke:#f57c00,stroke-width:2px classDef service fill:#e8f5e9,stroke:#388e3c,stroke-width:2px classDef gitops fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px classDef k8s fill:#fce4ec,stroke:#c2185b,stroke-width:2px classDef infra fill:#e0f2f1,stroke:#00695c,stroke-width:2px classDef monitor fill:#fff8e1,stroke:#f9a825,stroke-width:2px class WEB_UI,CI_CD,WEBHOOK,API_CALL trigger class AUTH,VALIDATE,NAME_CHECK,QUOTA process class ENV_CREATE,CONFIG_MERGE,TEMPLATE_GEN,METADATA service class ARGOCD_APP,GIT_SYNC,MANIFEST_APPLY gitops class NAMESPACE,RBAC,NETWORK_POL,HELM_DEPLOY k8s class PODS,SERVICES,INGRESS,SECRETS infra class READINESS,STATUS_UPDATE,NOTIFICATION,URL_READY monitor
Detailed Environment Creation Sequence
sequenceDiagram
participant User/CI
participant Blueberry API
participant Environment Service
participant Config Service
participant Firestore
participant ArgoCD
participant Git Repository
participant Kubernetes
participant Readiness Checker
participant Notification Service
Note over User/CI,Notification Service: Environment Creation Lifecycle
%% Request Phase
User/CI->>Blueberry API: POST /environments
Blueberry API->>Blueberry API: Authenticate & validate
Blueberry API->>Environment Service: createEnvironment(request)
%% Validation Phase
Environment Service->>Firestore: Check name uniqueness
Environment Service->>Config Service: Validate configuration
Environment Service->>Environment Service: Check resource quotas
%% Metadata Creation
Environment Service->>Firestore: Store environment metadata
Note over Environment Service,Firestore: Status: CREATING
%% Configuration Processing
Environment Service->>Config Service: Merge configurations
Config Service->>Environment Service: Merged values
Environment Service->>Environment Service: Generate ArgoCD templates
%% GitOps Deployment
Environment Service->>ArgoCD: Create Application CR
ArgoCD->>Git Repository: Fetch manifests
ArgoCD->>Kubernetes: Apply manifests
%% Kubernetes Resources
Kubernetes->>Kubernetes: Create namespace
Kubernetes->>Kubernetes: Apply RBAC
Kubernetes->>Kubernetes: Create network policies
Kubernetes->>Kubernetes: Deploy Helm charts
%% Infrastructure Deployment
Kubernetes->>Kubernetes: Create pods
Kubernetes->>Kubernetes: Create services
Kubernetes->>Kubernetes: Configure ingress
Kubernetes->>Kubernetes: Provision secrets
%% Status Monitoring
Note over Environment Service,Firestore: Status: PROVISIONING
Environment Service->>Firestore: Update status
loop Every 30 seconds
Readiness Checker->>Kubernetes: Check pod status
Readiness Checker->>Kubernetes: Check service endpoints
Readiness Checker->>Readiness Checker: Test URLs
alt Environment Ready
Readiness Checker->>Environment Service: Environment ready
Environment Service->>Firestore: Status: READY
Environment Service->>Notification Service: Send ready notification
Notification Service->>User/CI: Environment ready
else Still Provisioning
Readiness Checker->>Environment Service: Still provisioning
Note over Readiness Checker: Continue monitoring
else Error Detected
Readiness Checker->>Environment Service: Error detected
Environment Service->>Firestore: Status: FAILED
Environment Service->>Notification Service: Send error notification
end
end
Environment Status Lifecycle
stateDiagram-v2
[*] --> CREATING
CREATING --> PROVISIONING : ArgoCD Application Created
CREATING --> FAILED : Validation Error
PROVISIONING --> READY : All Checks Pass
PROVISIONING --> FAILED : Critical Error
PROVISIONING --> PROVISIONING : Transient Issues
READY --> UPDATING : Configuration Change
READY --> TERMINATING : Delete Request
READY --> FAILED : Runtime Error
UPDATING --> READY : Update Complete
UPDATING --> FAILED : Update Error
FAILED --> TERMINATING : Cleanup Request
FAILED --> PROVISIONING : Retry Request
TERMINATING --> TERMINATED : Cleanup Complete
TERMINATING --> FAILED : Cleanup Error
TERMINATED --> [*]
note right of CREATING
- Validates request
- Checks quotas
- Creates metadata
end note
note right of PROVISIONING
- ArgoCD syncing
- Kubernetes deploying
- Grace period active
end note
note right of READY
- All services healthy
- URLs accessible
- Monitoring active
end note
note right of FAILED
- Critical error occurred
- Manual intervention needed
- Automatic cleanup possible
end note
Configuration Merging Process
graph LR
subgraph "Input Sources"
DEFAULT[Default Values
Helm Chart] CONFIG_SET[Configuration Set
User Selected] OVERRIDES[Runtime Overrides
Environment Specific] USER_INPUT[User Input
Form Data] end subgraph "Merge Process" VALIDATOR[Configuration
Validator] MERGER[Value
Merger] TEMPLATE[Template
Generator] end subgraph "Output" HELM_VALUES[Helm Values
YAML] ARGOCD_APP[ArgoCD Application
Manifest] COMPUTED[Computed Values
URLs, Names] end DEFAULT --> MERGER CONFIG_SET --> MERGER OVERRIDES --> MERGER USER_INPUT --> VALIDATOR VALIDATOR --> MERGER MERGER --> TEMPLATE TEMPLATE --> HELM_VALUES TEMPLATE --> ARGOCD_APP TEMPLATE --> COMPUTED classDef input fill:#e3f2fd,stroke:#1976d2,stroke-width:2px classDef process fill:#fff3e0,stroke:#f57c00,stroke-width:2px classDef output fill:#e8f5e9,stroke:#388e3c,stroke-width:2px class DEFAULT,CONFIG_SET,OVERRIDES,USER_INPUT input class VALIDATOR,MERGER,TEMPLATE process class HELM_VALUES,ARGOCD_APP,COMPUTED output
Helm Chart] CONFIG_SET[Configuration Set
User Selected] OVERRIDES[Runtime Overrides
Environment Specific] USER_INPUT[User Input
Form Data] end subgraph "Merge Process" VALIDATOR[Configuration
Validator] MERGER[Value
Merger] TEMPLATE[Template
Generator] end subgraph "Output" HELM_VALUES[Helm Values
YAML] ARGOCD_APP[ArgoCD Application
Manifest] COMPUTED[Computed Values
URLs, Names] end DEFAULT --> MERGER CONFIG_SET --> MERGER OVERRIDES --> MERGER USER_INPUT --> VALIDATOR VALIDATOR --> MERGER MERGER --> TEMPLATE TEMPLATE --> HELM_VALUES TEMPLATE --> ARGOCD_APP TEMPLATE --> COMPUTED classDef input fill:#e3f2fd,stroke:#1976d2,stroke-width:2px classDef process fill:#fff3e0,stroke:#f57c00,stroke-width:2px classDef output fill:#e8f5e9,stroke:#388e3c,stroke-width:2px class DEFAULT,CONFIG_SET,OVERRIDES,USER_INPUT input class VALIDATOR,MERGER,TEMPLATE process class HELM_VALUES,ARGOCD_APP,COMPUTED output
Resource Provisioning Details
graph TB
subgraph "Namespace Setup"
NS_CREATE[Create Namespace]
NS_LABEL[Apply Labels]
NS_QUOTA[Resource Quotas]
NS_LIMITS[Limit Ranges]
end
subgraph "Security Setup"
RBAC_SA[Service Account]
RBAC_ROLE[Role & RoleBinding]
NET_POL[Network Policies]
POD_SEC[Pod Security Standards]
end
subgraph "Application Deployment"
SECRET_SYNC[External Secrets
Sync] CONFIG_MAP[ConfigMaps] HELM_DEPLOY[Helm Release] INIT_JOBS[Init Jobs] end subgraph "Networking" SVC_CREATE[Services] INGRESS_CREATE[Ingress] TLS_CERT[TLS Certificates] DNS_UPDATE[DNS Records] end subgraph "Monitoring Setup" SVC_MONITOR[ServiceMonitor] POD_MONITOR[PodMonitor] ALERTS[Alert Rules] DASHBOARDS[Grafana Dashboards] end NS_CREATE --> NS_LABEL NS_LABEL --> NS_QUOTA NS_QUOTA --> NS_LIMITS NS_LIMITS --> RBAC_SA RBAC_SA --> RBAC_ROLE RBAC_ROLE --> NET_POL NET_POL --> POD_SEC POD_SEC --> SECRET_SYNC SECRET_SYNC --> CONFIG_MAP CONFIG_MAP --> HELM_DEPLOY HELM_DEPLOY --> INIT_JOBS INIT_JOBS --> SVC_CREATE SVC_CREATE --> INGRESS_CREATE INGRESS_CREATE --> TLS_CERT TLS_CERT --> DNS_UPDATE DNS_UPDATE --> SVC_MONITOR SVC_MONITOR --> POD_MONITOR POD_MONITOR --> ALERTS ALERTS --> DASHBOARDS classDef namespace fill:#e3f2fd,stroke:#1976d2,stroke-width:2px classDef security fill:#fff3e0,stroke:#f57c00,stroke-width:2px classDef app fill:#e8f5e9,stroke:#388e3c,stroke-width:2px classDef network fill:#fce4ec,stroke:#c2185b,stroke-width:2px classDef monitor fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px class NS_CREATE,NS_LABEL,NS_QUOTA,NS_LIMITS namespace class RBAC_SA,RBAC_ROLE,NET_POL,POD_SEC security class SECRET_SYNC,CONFIG_MAP,HELM_DEPLOY,INIT_JOBS app class SVC_CREATE,INGRESS_CREATE,TLS_CERT,DNS_UPDATE network class SVC_MONITOR,POD_MONITOR,ALERTS,DASHBOARDS monitor
Sync] CONFIG_MAP[ConfigMaps] HELM_DEPLOY[Helm Release] INIT_JOBS[Init Jobs] end subgraph "Networking" SVC_CREATE[Services] INGRESS_CREATE[Ingress] TLS_CERT[TLS Certificates] DNS_UPDATE[DNS Records] end subgraph "Monitoring Setup" SVC_MONITOR[ServiceMonitor] POD_MONITOR[PodMonitor] ALERTS[Alert Rules] DASHBOARDS[Grafana Dashboards] end NS_CREATE --> NS_LABEL NS_LABEL --> NS_QUOTA NS_QUOTA --> NS_LIMITS NS_LIMITS --> RBAC_SA RBAC_SA --> RBAC_ROLE RBAC_ROLE --> NET_POL NET_POL --> POD_SEC POD_SEC --> SECRET_SYNC SECRET_SYNC --> CONFIG_MAP CONFIG_MAP --> HELM_DEPLOY HELM_DEPLOY --> INIT_JOBS INIT_JOBS --> SVC_CREATE SVC_CREATE --> INGRESS_CREATE INGRESS_CREATE --> TLS_CERT TLS_CERT --> DNS_UPDATE DNS_UPDATE --> SVC_MONITOR SVC_MONITOR --> POD_MONITOR POD_MONITOR --> ALERTS ALERTS --> DASHBOARDS classDef namespace fill:#e3f2fd,stroke:#1976d2,stroke-width:2px classDef security fill:#fff3e0,stroke:#f57c00,stroke-width:2px classDef app fill:#e8f5e9,stroke:#388e3c,stroke-width:2px classDef network fill:#fce4ec,stroke:#c2185b,stroke-width:2px classDef monitor fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px class NS_CREATE,NS_LABEL,NS_QUOTA,NS_LIMITS namespace class RBAC_SA,RBAC_ROLE,NET_POL,POD_SEC security class SECRET_SYNC,CONFIG_MAP,HELM_DEPLOY,INIT_JOBS app class SVC_CREATE,INGRESS_CREATE,TLS_CERT,DNS_UPDATE network class SVC_MONITOR,POD_MONITOR,ALERTS,DASHBOARDS monitor
Readiness Check Process
graph LR
subgraph "Grace Period"
GRACE_START[Environment Created]
GRACE_WAIT[5 Minute Grace Period]
GRACE_END[Grace Period Complete]
end
subgraph "Health Checks"
POD_CHECK[Pod Readiness]
SVC_CHECK[Service Endpoints]
URL_CHECK[URL Health Check]
ARGOCD_CHECK[ArgoCD Sync Status]
end
subgraph "Status Decision"
ALL_PASS{All Checks Pass?}
ANY_FAIL{Critical Failure?}
STATUS_READY[Status: READY]
STATUS_WARN[Status: PROVISIONING
with Warnings] STATUS_FAIL[Status: FAILED] end GRACE_START --> GRACE_WAIT GRACE_WAIT --> GRACE_END GRACE_END --> POD_CHECK POD_CHECK --> SVC_CHECK SVC_CHECK --> URL_CHECK URL_CHECK --> ARGOCD_CHECK ARGOCD_CHECK --> ALL_PASS ALL_PASS -->|Yes| STATUS_READY ALL_PASS -->|No| ANY_FAIL ANY_FAIL -->|Yes| STATUS_FAIL ANY_FAIL -->|No| STATUS_WARN STATUS_WARN -.->|Retry in 30s| POD_CHECK STATUS_FAIL --> GRACE_START STATUS_READY --> GRACE_START classDef grace fill:#e3f2fd,stroke:#1976d2,stroke-width:2px classDef check fill:#fff3e0,stroke:#f57c00,stroke-width:2px classDef status fill:#e8f5e9,stroke:#388e3c,stroke-width:2px class GRACE_START,GRACE_WAIT,GRACE_END grace class POD_CHECK,SVC_CHECK,URL_CHECK,ARGOCD_CHECK check class ALL_PASS,ANY_FAIL,STATUS_READY,STATUS_WARN,STATUS_FAIL status
with Warnings] STATUS_FAIL[Status: FAILED] end GRACE_START --> GRACE_WAIT GRACE_WAIT --> GRACE_END GRACE_END --> POD_CHECK POD_CHECK --> SVC_CHECK SVC_CHECK --> URL_CHECK URL_CHECK --> ARGOCD_CHECK ARGOCD_CHECK --> ALL_PASS ALL_PASS -->|Yes| STATUS_READY ALL_PASS -->|No| ANY_FAIL ANY_FAIL -->|Yes| STATUS_FAIL ANY_FAIL -->|No| STATUS_WARN STATUS_WARN -.->|Retry in 30s| POD_CHECK STATUS_FAIL --> GRACE_START STATUS_READY --> GRACE_START classDef grace fill:#e3f2fd,stroke:#1976d2,stroke-width:2px classDef check fill:#fff3e0,stroke:#f57c00,stroke-width:2px classDef status fill:#e8f5e9,stroke:#388e3c,stroke-width:2px class GRACE_START,GRACE_WAIT,GRACE_END grace class POD_CHECK,SVC_CHECK,URL_CHECK,ARGOCD_CHECK check class ALL_PASS,ANY_FAIL,STATUS_READY,STATUS_WARN,STATUS_FAIL status
Error Handling and Recovery
Common Failure Points
Stage | Error Type | Recovery Action |
---|---|---|
Validation | Invalid configuration | Return validation errors |
Name Check | Duplicate name | Suggest alternative names |
Quota | Resource limits exceeded | Queue request or reject |
ArgoCD | Sync failure | Retry with exponential backoff |
Kubernetes | Resource creation | Check quotas and permissions |
Networking | Ingress issues | Verify DNS and certificates |
Application | Pod failures | Check logs and resource limits |
Recovery Strategies
graph TD
ERROR[Error Detected] --> TYPE{Error Type}
TYPE -->|Transient| RETRY[Exponential Backoff
Retry] TYPE -->|Configuration| VALIDATE[Re-validate
Configuration] TYPE -->|Resource| QUOTA[Check Resource
Quotas] TYPE -->|Network| NETWORK[Verify Network
Configuration] TYPE -->|Critical| FAIL[Mark as Failed
Require Manual Fix] RETRY --> SUCCESS{Success?} VALIDATE --> SUCCESS QUOTA --> SUCCESS NETWORK --> SUCCESS SUCCESS -->|Yes| READY[Environment Ready] SUCCESS -->|No| MAX{Max Retries?} MAX -->|No| RETRY MAX -->|Yes| FAIL FAIL --> CLEANUP[Automatic Cleanup
After 24 hours]
Retry] TYPE -->|Configuration| VALIDATE[Re-validate
Configuration] TYPE -->|Resource| QUOTA[Check Resource
Quotas] TYPE -->|Network| NETWORK[Verify Network
Configuration] TYPE -->|Critical| FAIL[Mark as Failed
Require Manual Fix] RETRY --> SUCCESS{Success?} VALIDATE --> SUCCESS QUOTA --> SUCCESS NETWORK --> SUCCESS SUCCESS -->|Yes| READY[Environment Ready] SUCCESS -->|No| MAX{Max Retries?} MAX -->|No| RETRY MAX -->|Yes| FAIL FAIL --> CLEANUP[Automatic Cleanup
After 24 hours]
Performance Metrics
Typical Timelines
Phase | Duration | Description |
---|---|---|
Validation | < 1 second | Request validation and auth |
Metadata Creation | < 2 seconds | Database operations |
ArgoCD Sync | 30-60 seconds | Git sync and manifest apply |
Kubernetes Deploy | 2-5 minutes | Pod startup and readiness |
URL Availability | 30-60 seconds | Ingress and DNS propagation |
Total (P95) | < 8 minutes | 95th percentile creation time |
Monitoring Metrics
- Environment creation success rate
- Time to ready (P50, P95, P99)
- Failure rate by error type
- Resource utilization per environment
- Cost per environment per hour
Related Documentation
- Environment API Reference
- Configuration Management
- ArgoCD Integration
- Troubleshooting Environment Creation
Last Updated: January 2024
Document ID: architecture/diagrams/environment-creation-flow