Access vault secret from bash script

Question

I am looking for help on a direction on where to go from here. I have a Hasicorp Vault server running and accessible over DNS (local). I need to have a script get a secret from it then use that secret. I am not sure where to go from there. I know I want to use a bash script for this but past that I don't know what to do.

Until now I have been using curl to test it.

Suggestions?

Answer 1

The vault docs mention a -field parameter for the read subcommand. So you should be able to put this into a shell script:

SECRET=$(vault read -field foo secret/mysecret)

Other vault docs use the vault kv get in the same way so you might try:

SECRET=$(vault kv get -field foo secret/mysecret)

Answer 2

This mighty be a lengthy procedure but worth implementing, creating child tokens to fetch information from Vault Server.

Execute following procedure at Vault Server.

1. Create policy to restrict the access for clients.

cat auth-policy.hcl 
path "secret/*" {
  capabilities = ["read"]
}

2. Write the policy

vault policy write client-access auth-policy.hcl                         
Success! Uploaded policy: client-access 

3. Create token

vault token create -policy=client-access -period=768h                                    
                                                                                
Key                Value                                                        
---                -----                                                        
token              *********************                         
token_accessor     *********************                         
token_duration     768h                                                         
token_renewable    true                                                         
token_policies     [client-access default]  
-----------------------------------------------
vault policy write client-access auth-policy.hcl                         
Success! Uploaded policy: client-access 

Configure -period in hours. Token should be renewed after configured period of time. Configure expiry period accordingly by considering max_lease_ttl and default_lease_ttl in config.hcl.

Configure these parameters in config.hcl during Vault startup.

max_lease_ttl = "1000h"
default_lease_ttl = "1000h"

Now using this child token, from your laptop/machine, you will be able to fetch the information from Vault server, using APIs.

$ curl -sH "X-Vault-Token: CHILD-TOKEN" -X GET VAULT-URL/v1/secret/SECRET | jq
{
  "request_id": "e53887a2-fe5a-2f27-2121-c716a697f0e8",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 25920000,
  "data": {
    **************
    **************
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

With this child token, users can only read the data but neither write or delete data as configured in auth-policy.

If you still want to secure child token you can save it in Jenkins credentials and will be fetched into bash script when build triggered.

Answer 3

I've ended up at exactly the same spot.

We have a Vault server holding a good deal of secrets for different environments and various endpoints for each environment, so we set up kv engines for every environment and added a descriptive path for every secret. For instance, we have called our engines constant-text/environment/engine-name and all Database users are somewhere in db/schema-name/username, db/schema-name/password, db/schema-name/tns-string (we use Oracle).

I will not cover SSL/TLS setup, as you said you have TLS installed.

Sources: in Vault CLI there is a section about environment variables.

This is what I did:

!#/bin/bash
export VAULT_ADDR=https://address.of.vault.server/
export VAULT_TOKEN=value-of-my-root-token-or-an-authorized-token
vault kv get const/environment/engine-name db/schema/tns-string | command

where 'command' will read the secret from stdin. It will be required to parse the vault output format (example given in subsection "Reading Data" in 1) but a manual test shows more than 1000 words. As already mentionned by Tensibai above, there are ways to parse JSON, which I haven't implemented yet. I'm afraid it can't get much simpler. It can get more complex and more secure if you pass the address and the token to the script and then use

export VAULT_ADDR=$1
export VAULT_TOKEN=$2

to pass them on to the environment, but don't forget adding

set VAULT_ADDR=
set VAULT_TOKEN=

at the end to clear the variables again.

Answer 4

If you want pure-play bash, then you're already doing the right thing, curl, and although you don't mention it, it's a good idea to use jq when using REST APIs for parsing results, just remember to request json.

If you can install extra tools, then you can install the vault cli itself. Personally, I never install anything except docker, so I'd run that in a container:

$ alias vault="docker run -it --rm --cap-add IPC_LOCK vault"
$ vault --version
Vault v1.5.3 (9fcd81405feb320390b9d71e15a691c3bc1daeef)

As I'm sure you know, the vault cli, just like curl, can return json from most of its commands, which you can pipe to jq (again) for parsing (which is a lot easier than using go-templates IMO).

However, having said that, I have a suspicion that maybe you're asking the wrong question. Getting a secret on your laptop isn't really what vault is for and maybe you're just writing the code on your laptop ready to deploy to a server somewhere in order to provide a secret to that server. If that's the case then doing this in bash may be the wrong approach due to the issue of initial trust.

If you need to get a secret from vault then you have to authenticate first and get a token that has the correct authority (policies) to access the secret you're interested in. Authentication requires some sort of credentials (jwt/username-password/etc.). How do you get those creds to your bash script? You can't commit them to git.

Perhaps you're provisioning an ec2 instance with user-data that contains the creds the script needs (or any other cloud or on-prem equiv. of this process). If that's the case, then you can work out how to provision the machine with the secret it needs instead of the creds to get the secret. This is of course much easier with containers and kubernetes, but you can do the same thing with VMs of various types. In AWS you can do it with Lambda. In Azure with Azure Functions. On-prem you can do it using VMWare. With this approach, you'd be pushing the secrets management function into the platform (shift-left if you like that vocab.), rather than making it a responsibility of your application layer.

Maybe this is going too far and you're just playing around and learning about vault, but you did ask for help with direction. But if it's more than that, you should look at things like SPIFFE and check out the patterns hashicorp has implemented with its K8S sidecar.

Answer 5

Vault has a REST API available for reading.

https://www.vaultproject.io/api/index.html

In your script, just curl the API.