Copy multiple files using * wildcard from kubernetes container

Question

I have several files named out1, out2, ... in my Kubernetes container. I want to copy them to my local computer. I am using:

$ kubectl cp pod:/path/out* .

But I am getting an error:

tar: Removing leading `/' from member names
tar: /path/out*: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors
error: path/out* no such file or directory

How can I use kubectl cp with wildcards?

Answer 1

Wildcard support was added to kubectl cp in the #72641 pull request.

It is merged but only available in Kubernetes v1.14.0. Take a look at the ChangeLog:

Notable Features:
[...]
• kubectl supports copying files with wild card (#72641, @dixudx)

However, as stated on the issue #78854 the PR was made without proper tests and the feature is actually broken. The PR to fix the problem (#78928) is open since June, and it's not merged yet.

Although the PR to fix the feature exists, they are also considering removing the support to wildcard copy.

Answer 2

You can do that using this command:

kubectl cp -n [NAMESPACE] [POD_NAME]:/[POD_DIRECTORY]/. .

Answer 3

As there is no wildcard support, a workaround is to do a 2-step process to achieve the same:

• kubectl exec ... to do make a new tmp directory and wildcard copy/move your desired transfer files into that dir on the container
• use the cmd from @cookiedough above to copy all files from that dir on the container to your local

Example:

export TARGET_POD="myapp-mypod1234" && \
kubectl exec "$TARGET_POD" -n MYNAMESPACE "$TARGET_POD" \
-- sh -c "mkdir -p /tmp/outfiles && cp /path/out* /tmp/outfiles/" && \
kubectl cp "sites/$TARGET_POD:/tmp/outfiles/." .

Answer 4

I had a similar requirement and finally settled on:

mkdir /my/dest/dir
kubectl  exec $TARGET_POD -n MYNAMESPACE -- tar -zcvf - -C /my/src/directory . | tar -zxvf - -C /my/dest/dir

This depends on 'tar' being available inside the container. Similar can be done with any archive command (zip/unzip or cpio) that can deal with stdin/stdout as long as the command is in both the container and the host.

Also note that '*' is not native to tar, but depends on shell expansion (in my experience...), so if you want to match only certain files in a directory on the container I include 'bash' in the call and cd to the directory first:

kubectl  exec $TARGET_POD -n MYNAMESPACE -- bash -c "cd /my/src/directory;tar -zcvf - *.log" | tar -zxvf - -C /my/dest/dir

Again, bash or some other shell supporting * expansion must be present in the container.

Finally, this works in the other direction as well sending multiple files to a container:

cd /my/src/dir
tar -zcvf - *.log | kubectl exec $TARGET_POD -n MYNAMESPACE --stdin -- tar -zxvf - -C /my/dest/dir

And "bash" is your friend if the target directory does not exist:

tar -zcvf - *.log | kubectl exec $TARGET_POD -n MYNAMESPACE --stdin -- bash -c "mkdir -p /my/dest/dir;tar -zxvf - -C /my/dest/dir"

Answer 5

For everyone's benefit, I am pasting working code as suggested by heyjared user on https://github.com/kubernetes/kubernetes/issues/78854

Use find with xargs as a workaround:

find . | xargs -i{} kubectl cp {} namespace/pod:path

It worked for me. I replaced "." with my needed source path.

Example:

find <source_path> | xargs -i{} kubectl cp {} namespace/pod:<path_inside_pod>

Like:

find /var/tmp/data/ | xargs -i{} kubectl cp {} namespace/pod:/tmp/

This will copy all contents inside local /var/tmp/data/ directory and store it under /tmp/ inside pod.