kubectl debug: debug a pod with no shell in the image
Distroless and scratch images have no shell, so kubectl exec has nothing to open. kubectl debug solves that with an ephemeral container: a temporary container, with the tools you pick, added to the pod that is already running.
When kubectl exec is not enough
exec runs a command inside a container that already exists. If the image has no sh, it fails:
kubectl exec -it <pod> -n <namespace> -- sh
# OCI runtime exec failed: exec: "sh": executable file not found in $PATH1. Open an ephemeral container with kubectl debug
--target joins the ephemeral container to the application container’s process namespace, when the runtime allows it. That way ps shows its processes:
kubectl debug -it <pod> -n <namespace> --image=busybox:1.36 --target=<container>
# Inside the ephemeral container
ps aux
ls /proc/1/root/The application’s filesystem is visible under /proc/<pid>/root, as long as the ephemeral container runs as the same user or as root.
2. Pick the debug image
- busybox: sh, ps, wget, nslookup and the basics, in a small image.
- nicolaka/netshoot: curl, dig, nc, tcpdump, openssl and other network tools.
- An image of the application’s language, when you need its runtime.
- In namespaces with PodSecurity restricted, an image that runs as root may be refused: in newer kubectl versions, --profile=restricted builds the ephemeral container within that profile.
3. Or debug a copy of the pod
For a pod in CrashLoopBackOff, where the container dies before you can get in, create a copy with a different command. The original pod stays as it is:
# Copy with a debug container alongside and shared processes
kubectl debug <pod> -n <namespace> -it --copy-to=<pod>-debug \
--image=busybox:1.36 --share-processes
# Copy where the application container opens a shell instead of its command
kubectl debug <pod> -n <namespace> -it --copy-to=<pod>-debug \
--container=<container> -- sh
# Delete the copy when you are done
kubectl delete pod <pod>-debug -n <namespace>The copy is a new pod, outside the Deployment: it gets no traffic from the Service unless it has the same labels.
What to know about ephemeral containers
- Stable since Kubernetes 1.25.
- They do not leave the pod: the ephemeral container stays listed until the pod is deleted or recreated, and it does not restart.
- They have no ports, probes or resources of their own.
- They need patch permission on pods/ephemeralcontainers in the namespace, plus get on pods and create on pods/attach for -it.
- They show up in kubectl describe pod, under Ephemeral Containers.
kubectl debug or kubectl exec
| Situation | Use |
|---|---|
| The image has a shell and you just want to look inside the container | kubectl exec |
| Distroless or scratch image, no shell | kubectl debug with --target |
| The image lacks tools (curl, dig, tcpdump) | kubectl debug with a tools image |
| The container dies before you get in (CrashLoopBackOff) | kubectl debug --copy-to, with another command |
| You cannot touch the production pod | kubectl debug --copy-to |
kubectl debug and Kubepier
Kubepier does not create ephemeral containers: for those, use kubectl debug from your terminal. On Pro and Team, Kubepier opens a shell in a container that already exists, like kubectl exec: on the web, for admins, and on desktop. On desktop, a pod’s ephemeral containers show up in its details, under Ephemeral Containers. The node shell is on the web (admins) and on desktop.
To test the network from inside the cluster without touching the application pod, the web Bastion opens a terminal in a temporary pod with the netshoot tools.