IMP

Appearance

Imp Kubernetes E2E Sequence (Failure and Retry Paths) ​

mermaid
sequenceDiagram
    autonumber
    actor U as User/kubectl
    participant APIServer as Kubernetes API Server
    participant Etcd as etcd
    participant Op as Imp Operator
    participant Agent as Imp Agent
    participant FC as Firecracker Runtime
    participant Net as ImpNetwork/IPAM
    participant Ev as events.k8s.io

    U->>APIServer: POST ImpVM/ImpNetwork
    APIServer->>Etcd: Persist desired state
    Op->>APIServer: LIST/WATCH + GET dependencies

    alt ResourceVersion conflict on PATCH /status
        Op->>APIServer: PATCH /status (stale rv)
        APIServer-->>Op: 409 Conflict
        Op->>APIServer: GET latest ImpVM
        Op->>APIServer: PATCH /status retry
    end

    Op->>Agent: Assign desired runtime/network intent

    alt Firecracker start failure
        Agent->>FC: Boot VM
        FC-->>Agent: Start error
        Agent-->>Op: Runtime failure details
        Op->>APIServer: PATCH condition Ready=False, Reason=RuntimeStartFailed
        Op->>Ev: Event type=Warning reason=RuntimeStartFailed
    else Network allocation failure
        Agent->>Net: Allocate IP / wire network
        Net-->>Agent: Allocation error
        Agent-->>Op: Network failure details
        Op->>APIServer: PATCH condition Ready=False, Reason=NetworkSetupFailed
        Op->>Ev: Event type=Warning reason=NetworkSetupFailed
    else Agent heartbeat timeout
        Op->>APIServer: PATCH condition Ready=False, Reason=AgentTimeout
        Op->>Ev: Event type=Warning reason=AgentTimeout
    end

    opt Retry loop (bounded backoff)
        Op->>APIServer: Requeue with backoff
        Op->>APIServer: GET resources again
        Op->>Agent: Re-issue assignment when safe
    end

    opt Recovery path
        Agent-->>Op: Runtime/network recovered
        Op->>APIServer: PATCH condition Ready=True
        Op->>Ev: Event type=Normal reason=Recovered
        APIServer->>Etcd: Persist recovered status
    end

Failure Notes ​

  • Condition transitions should preserve reason/message history via Events.
  • Retries must be idempotent and conflict-safe.
  • Backoff policy should prevent controller hot loops during repeated node/runtime failures.