IMP

Appearance

Day 1: Operations ​

Context Pin ​

Pin kubectl to an explicit context — do not rely on current-context. For the homelab Omni target:

bash
kubectl --context omni-unraid-lab get nodes
kubectl --context omni-unraid-lab -n imp-system get pods
kubectl --context omni-unraid-lab get impvm -A

All runbook commands below assume kubectl --context omni-unraid-lab if your kubeconfig has multiple contexts. Alias it if needed:

bash
alias k='kubectl --context omni-unraid-lab'

Daily Operations ​

  • Track reconcile latency and error-rate trends.
  • Review VM readiness/failure condition distribution.
  • Check agent health and node coverage (kubectl get nodes -l imp/enabled=true).
  • Verify network allocation health and saturation.
  • Watch for MemoryPressure if pressureLifecycle.enabled=true — largest VMs suspend first and never auto-resume.

Metrics ​

Imp exposes Prometheus-compatible metrics from operator and agent. Scrape via ServiceMonitor (enabled by default) or direct /metrics.

bash
kubectl -n imp-system get servicemonitor
kubectl -n imp-system get svc -l app.kubernetes.io/name=imp -o wide
curl -s http://<operator-pod-ip>:8080/metrics | grep imp_

Key signals:

  • VM state/phase metrics
  • Scheduling/boot latency (p95/p99)
  • Guest resource usage (CPU, memory, disk)
  • Reconcile queue depth and error rates

See Architecture — Operational Notes and integrate with your Prometheus/Grafana stack.

Change Management ​

  • Roll operator and agent separately when possible.
  • imp-runtime is OnDelete — see Day 0 — Runtime OnDelete. Cordon/drain + delete pod to roll.
  • Use canary rollout on a subset of nodes (keep imp/enabled=true selector).
  • Watch condition/event deltas during rollout windows.
  • Keep version compatibility notes for CRD changes (v1alpha1 may break; pin digests).

Capacity and Performance ​

  • Watch queue depth and reconcile time p95/p99.
  • Monitor per-node VM density and startup latency.
  • Watch memory/CPU pressure on controller and agents.
  • Track repeated retries that indicate hidden bottlenecks.

Quick diagnostics:

bash
kubectl --context omni-unraid-lab describe impvm <name> -n <ns>
kubectl --context omni-unraid-lab logs -n imp-system deploy/imp-controller-manager -c manager --since=30m
kubectl --context omni-unraid-lab logs -n imp-system ds/imp-agent --since=30m
kubectl --context omni-unraid-lab get events -A --sort-by=.lastTimestamp | tail -n 100

Backup and Recovery Inputs ​

  • Retain cluster event history for incident windows.
  • Preserve metric history for trend and regression analysis.
  • Keep tested restore/runbook procedures for critical namespaces.