So I have been trying this-tutorial to access the runC container from the diego-cell VM. However, when I try to run

/var/vcap/packages/runc/bin/runc exec -t <instance_id> /bin/bash

I get the error

exec failed: container "<id>" does not exist

when I run cfdot for the given application it does list this instance-id, also when navigating into the location

/var/vcap/data/garden/depot/<container_id> it exists.

But runc in unable to open the container, also when I run

runc list

the list of running containers in the vm is empty

diego-cell/49dbeca6-36eb-4910-94b9-48960891a0b1:~# /var/vcap/packages/runc/bin/runc list
ID          PID         STATUS      BUNDLE      CREATED     OWNER

I've run all as root as mentioned in the link.

Why am I not able to access the runC containers, What am I missing?

P.S: Although the intent is same, I'm not looking for cf ssh

  • What if you run ps aux and look at the running processes? Do you see the processes for your app running? Also, what version of Cloud Foundry are you running? – Daniel Mikusa Jun 25 '19 at 16:35
  • Oh forgot to add that..Yes it shows the relevant process running – Vipin Menon Jun 26 '19 at 6:35
  • What's odd is that runc list is showing nothing. Do you see this on all your Cells or just a specific Cell? Again, what versions are you running? Pivotal CF or OSS CF? Definitely include diego & garden release version being used. – Daniel Mikusa Jun 26 '19 at 15:27
  • It maybe because it is a custom modified release of CF built in-house. Garden Release is 1.17.1 and Diego is 2.16 The CF has been broken to 3 separate deployments (1st deployment having cf and its components including diegoAPI) the second and third deployment is just for diegocells – Vipin Menon Jun 27 '19 at 8:36
  • OK, not sure I can help much with a custom deployment. My only suggestion would be to compare to a foundation installed using cf-deployment, and look for differences in your deployment, release versions & in how the Cells are deployed & configured. – Daniel Mikusa Jun 27 '19 at 14:08

I suspect this may be something coming from containerd changes - running PCF 2.5 the runc list works fine, 2.6 runc list shows nothing in the inventory. I have not yet found the equivalent command that will allow me to find the container id and how to execute a shell entry to that container instance yet, but I'm hot on the trail.

  • 2
    Found it (and confirmed it is new containerd behavior [for me, at least]): After bosh ssh'ing and sudo'ing cell where the instance is running: app_guid=<app_guid value from cf app <app_name> --guid> cell_name=<string of the diego cell guid you're on> cfdot actual-lrp-groups | grep $app_guid |grep $cell_name| jq '.instance | {instance_guid, cell_id, process_guid}' Take the result instance_guid and containerid=<instance_guid> /var/vcap/packages/containerd/bin/ctr -a /var/vcap/sys/run/containerd/containerd.sock -n garden tasks exec --exec-id my-shell --tty $containerid /bin/bash – Jon Ravenscraft Sep 12 '19 at 15:21
  • Amazing :)..This works – Vipin Menon Sep 13 '19 at 8:05

What @Jon Ravenscraft posted as a comment on his own accepted (non) answer did in fact work for me as a solution. I am just formatting it here as an answer for anybody who stumbles here in the future:

  • Follow these steps to get instance GUID (also included below for convenience)
# Establish the process-guid for the app:
cf curl /v2/apps/$(cf app <app_name> --guid)| jq -r '.metadata.guid + "-" + .entity.version'

# Find the hosts where the app instances are running:
cf curl v2/apps/$(cf app <app-name> --guid)/stats | grep host

# Identify the Diego cell you want to ssh to by running bosh vms | grep <cell_ip>

# bosh ssh to the Diego cell (grab the cell_id from step 3).

# Once in the Diego cell, as root, run the following command to grab the container_guid/instance_guid:
cfdot actual-lrp-groups-for-guid <process_guid_from_step_1> | jq '.instance | select (.address == "<diego_cell_ip>")' | jq .instance_guid
  • Then, from your terminal on the appropriate diego cell:
sudo /var/vcap/packages/containerd/bin/ctr -a /var/vcap/sys/run/containerd/containerd.sock -n garden tasks exec --exec-id my-shell --tty <instance-guid> /bin/sh

And voilà , you are dropped at a shell...

P.S. I believe the speculated reasoning given in the other answer is in fact, also correct, per https://docs.pivotal.io/pivotalcf/2-6/pcf-release-notes/runtime-rn.html:

PAS v2.6 enables containerd mode by default to bring PAS in line with other container providers. When you upgrade to PAS v2.6, Garden switches the container runtime from runc to containerd. Since the restart of Garden and the re-creation of containers are a normal part of a PAS upgrade, there is no operational impact to running applications.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.