Examples¶
Runnable, copy-pasteable recipes. The headline ones show Xila Guard enforcing placement security alongside a real external scheduler — out-of-tree, with no fork and no change to the scheduler itself.
How the orchestrator integration works
An orchestrator is a profile, not code: schedulerName (the scheduler it
sets on its Pods) + a workloadSelector (a label that marks its workloads).
Xila Guard co-registers its own ValidatingWebhookConfiguration on those Pods
and accepts that scheduler as a legitimate route (--external-schedulers), so it
enforces the security boundary without competing for scheduling. See
orchestrator integration.
Volcano + its scheduler (verified live)¶
This runs real upstream Volcano (scheduler + controller + its own admission webhooks, unmodified) and has Xila Guard gate Volcano's workloads. It is verified end-to-end against Volcano v1.12.2.
One command
cd xila-guard-k8s && make e2e-volcano does everything below in a disposable
kind cluster and asserts admit / deny / co-registration, then tears down.
1. Install real Volcano¶
kind create cluster --name xila-volcano
VOLCANO_VERSION=v1.12.2
kubectl apply -f "https://raw.githubusercontent.com/volcano-sh/volcano/${VOLCANO_VERSION}/installer/volcano-development.yaml"
kubectl -n volcano-system wait --for=condition=complete job/volcano-admission-init --timeout=240s
kubectl -n volcano-system rollout status deploy/volcano-scheduler --timeout=240s
2. Install Xila Guard with the volcano profile¶
Build + load the webhook image (nothing is published yet), then install the chart with the profile and the webhook's TLS CA bundle:
podman build --build-arg COMPONENT=webhook -t localhost/xila-guard-webhook:dev xila-guard-k8s
podman save -o /tmp/wh.tar localhost/xila-guard-webhook:dev
kind load image-archive /tmp/wh.tar --name xila-volcano
kubectl create namespace xila-guard-system
SVC=xila-guard-xila-guard-webhook
SAN="DNS:${SVC}.xila-guard-system.svc,DNS:${SVC}.xila-guard-system.svc.cluster.local"
openssl req -x509 -newkey rsa:2048 -nodes -days 2 \
-keyout /tmp/tls.key -out /tmp/tls.crt \
-subj "/CN=${SVC}.xila-guard-system.svc" -addext "subjectAltName=${SAN}"
kubectl create secret tls xila-guard-webhook-tls -n xila-guard-system --cert=/tmp/tls.crt --key=/tmp/tls.key
CA_BUNDLE=$(base64 -w0 < /tmp/tls.crt)
cat > /tmp/volcano-values.yaml <<EOF
coreServer: { enabled: false } # admission-boundary demo; core/observer not needed here
enforcement:
webhook:
tls:
caBundle: ${CA_BUNDLE}
orchestratorIntegrations:
- name: volcano
schedulerName: volcano
failurePolicy: Fail
workloadSelector:
matchExpressions:
- { key: volcano.sh/job-name, operator: Exists }
EOF
helm install xila-guard xila-guard-k8s/charts/xila-guard \
-n xila-guard-system -f /tmp/volcano-values.yaml
kubectl rollout status deploy/${SVC} -n xila-guard-system --timeout=180s
This renders a validate-volcano.guard.xila.us webhook on volcano.sh/job-name
Pods and runs the webhook with --external-schedulers=volcano.
3. Bind a protected tenant¶
kubectl create namespace volcano-tenant
NS_UID=$(kubectl get ns volcano-tenant -o jsonpath='{.metadata.uid}')
kubectl apply -f - <<EOF
apiVersion: guard.xila.us/v1alpha1
kind: PlacementPolicy
metadata: { name: tenant-dedicated-baseline }
spec:
placementScope: node
requirements: {}
evidence: { maxAgeSeconds: 3600 }
---
apiVersion: guard.xila.us/v1alpha1
kind: TenantBinding
metadata: { name: tb-volcano-tenant }
spec:
namespaceUid: "${NS_UID}"
tenantId: tenant-a
effectivePolicyRef: tenant-dedicated-baseline
EOF
4. Submit a real Volcano job → admitted and scheduled by Volcano¶
kubectl apply -f - <<EOF
apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata: { name: vc-allow, namespace: volcano-tenant }
spec:
minAvailable: 1
schedulerName: volcano
queue: default
tasks:
- replicas: 1
name: main
template:
spec:
schedulerName: volcano
restartPolicy: Never
containers:
- name: main
image: registry.k8s.io/pause:3.10
resources: { requests: { cpu: "50m", memory: "32Mi" }, limits: { cpu: "100m", memory: "64Mi" } }
EOF
# The Volcano controller creates the pod only after it passes OUR webhook:
kubectl get pods -n volcano-tenant -l volcano.sh/job-name=vc-allow -o wide
# -> Running, schedulerName=volcano, bound to a node (no scheduler conflict)
5. Bypass attempts → denied by Xila Guard¶
# Privileged Volcano-labeled pod -> DENIED:
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata: { name: vc-privileged, namespace: volcano-tenant, labels: { volcano.sh/job-name: vc-privileged } }
spec:
schedulerName: volcano
restartPolicy: Never
containers: [{ name: p, image: registry.k8s.io/pause:3.10, securityContext: { privileged: true } }]
EOF
# Error: admission webhook "validate-volcano.guard.xila.us" denied the request:
# xila-guard: privileged/host-access: privileged containers are not permitted in the protected pool
# nodeName-pinned Volcano-labeled pod -> DENIED (direct-binding bypass):
# xila-guard: protected pod must not set spec.nodeName (direct-binding bypass rejected)
What this proves
Volcano's own *.volcano.sh admission and validate-volcano.guard.xila.us
both act on the same workload (AND-semantics): the compliant job runs end-to-end
on Volcano's scheduler, while privileged/node-pinned bypasses are rejected —
co-registration with zero conflict, and Volcano was never modified.
run:ai + its scheduler¶
run:ai (NVIDIA) routes its workloads to schedulerName: runai-scheduler and labels
them run.ai/*. That is all Xila Guard needs — the runai profile is verified
against run:ai's published source, and the mechanism is identical to the Volcano
example above (which is verified live).
# values override — enable the run:ai profile:
orchestratorIntegrations:
- name: runai
schedulerName: runai-scheduler
failurePolicy: Fail
workloadSelector:
matchExpressions:
- { key: run.ai/application, operator: Exists }
helm upgrade --install xila-guard xila-guard-k8s/charts/xila-guard \
-n xila-guard-system --create-namespace \
-f runai-values.yaml # the override above + webhook TLS caBundle
With it enabled:
- a
validate-runai.guard.xila.usValidatingWebhookConfigurationinterceptsCREATEonrun.ai/applicationPods; - the webhook runs with
--external-schedulers=runai-scheduler; - a compliant run:ai Pod (
schedulerName: runai-scheduler, tenant resolved, not privileged, nonodeName) is ADMITTED and keepsrunai-scheduler— no scheduler conflict; - a privileged or
nodeName-pinned run:ai Pod is DENIED.
Out-of-tree by design — no fork, no PR
Xila Guard never modifies, rebuilds, or imports run:ai. It co-registers a
standard admission webhook (run:ai's dromos plugins are compile-time, so an
in-tree plugin would require rebuilding run:ai — explicitly avoided). The run:ai
platform is not run in CI here (it needs a heavy control plane and datacenter
GPUs); the profile is verified from source and the identical mechanism is proven
live against Volcano.
Add any orchestrator (no code)¶
A new scheduler is one profile entry — no build, no plugin:
orchestratorIntegrations:
- name: my-orchestrator
schedulerName: my-scheduler # "" = the default kube-scheduler
failurePolicy: Fail
workloadSelector:
matchExpressions:
- { key: my.org/workload, operator: Exists }
Shipped presets: runai and volcano are verified; kueue and yunikorn are
documented examples (confirm the selector/scheduler against your install).
Other cool examples¶
Shows the RFC 9334 verifier model: an accepted verifier result yields allow,
a production context that only trusts real verifiers yields indeterminate
(ATTESTATION_VERIFIER_NOT_ACCEPTED), and a failed check yields deny
(ATTESTATION_FAILED).
Simulation is never trusted by enforcement
The shipped verifier id is sim:… and is never accepted by an
enforcement context. The CLI xila-guard attest verify is built only with
--features attestation-sim. See attestation.
make demo-identity # host-process SVIDs (openssl), no cluster
make demo-identity-cluster # REAL SPIRE in kind: server + agent + CSI
Each node's collector authenticates with a per-node X.509-SVID over mTLS; the
receiver binds every report to the node UID from the verified certificate.
Proven: accept (200), spoof (403), replay (409), untrusted cert (TLS
handshake rejected). See identity.
# Helm values: let the observer evaluate from SPIRE-authenticated inventory
identity:
enabled: true
observer:
useAuthenticatedInventory: true
mode: require # fail-closed if a node has no authenticated inventory
The observer calls evaluate-node over mTLS; snapshots carry source
authenticated_collector, which an enforcement context accepts while still
rejecting mock.
make demo-nvidia
# or directly:
cd xila-guard-core && cargo run -p xila-guard-cli -- inventory --provider nvidia
Read-only nvidia-smi discovery (host_observation): UUID / model / PCI /
driver / VBIOS. Unknown sharing facts stay indeterminate, never allow. A
missing NVIDIA library is a structured unsupported result, not a crash.
In observe mode the controller re-evaluates and records drift when an outcome changes or evidence is lost — report-only, no eviction. Drift records go to the drift JSONL alongside decisions:
xila-guard evaluate maps the decision algebra onto exit codes (0 allow,
2 deny, 3 indeterminate, 1 error) — so a policy check fails your pipeline:
Next: the policy model for what the evaluator checks, or the support matrix for exactly what is real vs simulated.