How to get into an App Container Manually with Garden-RunC Backend?

Question

So I have been trying this-tutorial to access the runC container from the diego-cell VM. However, when I try to run

/var/vcap/packages/runc/bin/runc exec -t <instance_id> /bin/bash

I get the error

exec failed: container "<id>" does not exist

when I run cfdot for the given application it does list this instance-id, also when navigating into the location

/var/vcap/data/garden/depot/<container_id> it exists.

But runc in unable to open the container, also when I run

runc list

the list of running containers in the vm is empty

diego-cell/49dbeca6-36eb-4910-94b9-48960891a0b1:~# /var/vcap/packages/runc/bin/runc list
ID          PID         STATUS      BUNDLE      CREATED     OWNER
diego-cell/49dbeca6-36eb-4910-94b9-48960891a0b1:~#

I've run all as root as mentioned in the link.

Why am I not able to access the runC containers, What am I missing?

P.S: Although the intent is same, I'm not looking for cf ssh

Answer 1

I suspect this may be something coming from containerd changes - running PCF 2.5 the runc list works fine, 2.6 runc list shows nothing in the inventory. I have not yet found the equivalent command that will allow me to find the container id and how to execute a shell entry to that container instance yet, but I'm hot on the trail.

Answer 2

What @Jon Ravenscraft posted as a comment on his own accepted (non) answer did in fact work for me as a solution. I am just formatting it here as an answer for anybody who stumbles here in the future:

• Follow these steps to get instance GUID (also included below for convenience)

# Establish the process-guid for the app:
cf curl /v2/apps/$(cf app <app_name> --guid)| jq -r '.metadata.guid + "-" + .entity.version'
b06b2901-2093-4be4-9f37-f7788d04ce8c-ff93f0e2-285f-4f18-a6fc-806802f3dde9

# Find the hosts where the app instances are running:
cf curl v2/apps/$(cf app <app-name> --guid)/stats | grep host

# Identify the Diego cell you want to ssh to by running bosh vms | grep <cell_ip>

# bosh ssh to the Diego cell (grab the cell_id from step 3).

# Once in the Diego cell, as root, run the following command to grab the container_guid/instance_guid:
cfdot actual-lrp-groups-for-guid <process_guid_from_step_1> | jq '.instance | select (.address == "<diego_cell_ip>")' | jq .instance_guid

• Then, from your terminal on the appropriate diego cell:

sudo /var/vcap/packages/containerd/bin/ctr -a /var/vcap/sys/run/containerd/containerd.sock -n garden tasks exec --exec-id my-shell --tty <instance-guid> /bin/sh

And voilà , you are dropped at a shell...

P.S. I believe the speculated reasoning given in the other answer is in fact, also correct, per https://docs.pivotal.io/pivotalcf/2-6/pcf-release-notes/runtime-rn.html:

PAS v2.6 enables containerd mode by default to bring PAS in line with other container providers. When you upgrade to PAS v2.6, Garden switches the container runtime from runc to containerd. Since the restart of Garden and the re-creation of containers are a normal part of a PAS upgrade, there is no operational impact to running applications.