3

I am looking for help on a direction on where to go from here. I have a Hasicorp Vault server running and accessible over DNS (local). I need to have a script get a secret from it then use that secret. I am not sure where to go from there. I know I want to use a bash script for this but past that I don't know what to do.

Until now I have been using curl to test it.

Suggestions?

7
  • Have you seen digitalocean.com/community/tutorials/… ?? This seems like the sort of thing that most Vault tutorials would mention.
    – chicks
    Jul 12 '18 at 17:38
  • @chicks Yes I have seen that and followed it to set it up. None of that is useful to what I asked. I asked about using bash to interact with vault (likely via curl) and extract a secret (a value in a K:V) and use that in the bash script (likely as a variable). So no that guide is of no use in this case.
    – user8517
    Jul 12 '18 at 18:44
  • 1
    wiki.bash-hackers.org/syntax/expansion/cmdsubst might help you understand command substitution in a more general way.
    – chicks
    Jul 12 '18 at 19:01
  • 1
    Vault return a json, curl to get the json, jq to parse it, somthing along the line VALUE=$(curl [..auth headers..] http://<vault url>/KV/secret | jq '.<can't tell you didn't give any insight>)
    – Tensibai
    Jul 13 '18 at 14:38
  • @Tensibai Yes that is what I have been doing, hence why I said I used curl to test it, was looking for something better, guess that has gone over everyone's head.
    – user8517
    Jul 13 '18 at 17:18
3

The vault docs mention a -field parameter for the read subcommand. So you should be able to put this into a shell script:

SECRET=$(vault read -field foo secret/mysecret)

Other vault docs use the vault kv get in the same way so you might try:

SECRET=$(vault kv get -field foo secret/mysecret)
2
  • Again it seems like you are thinking I am trying to do this locally on my vault server. I am not doing that. Think of it this way, I have a vault server. I have a laptop. Both are different machines not connected except by the internet. I need to use a curl command to query the vault server. I then need to take the result of that query which is a JSON object and I need to extract the value at data.value. That make more sense? I repeat though, this is being done on a different machine than the vault server!
    – user8517
    Jul 12 '18 at 19:09
  • 3
    @JoshKirby you can install the vault command line on any machine...
    – Tensibai
    Jul 13 '18 at 14:39
1

This mighty be a lengthy procedure but worth implementing, creating child tokens to fetch information from Vault Server.

Execute following procedure at Vault Server.

  1. Create policy to restrict the access for clients.
cat auth-policy.hcl 
path "secret/*" {
  capabilities = ["read"]
}
  1. Write the policy
vault policy write client-access auth-policy.hcl                         
Success! Uploaded policy: client-access 
  1. Create token
vault token create -policy=client-access -period=768h                                    
                                                                                
Key                Value                                                        
---                -----                                                        
token              *********************                         
token_accessor     *********************                         
token_duration     768h                                                         
token_renewable    true                                                         
token_policies     [client-access default]  
-----------------------------------------------
vault policy write client-access auth-policy.hcl                         
Success! Uploaded policy: client-access 

Configure -period in hours. Token should be renewed after configured period of time. Configure expiry period accordingly by considering max_lease_ttl and default_lease_ttl in config.hcl.

Configure these parameters in config.hcl during Vault startup.

max_lease_ttl = "1000h"
default_lease_ttl = "1000h"

Now using this child token, from your laptop/machine, you will be able to fetch the information from Vault server, using APIs.

$ curl -sH "X-Vault-Token: CHILD-TOKEN" -X GET VAULT-URL/v1/secret/SECRET | jq
{
  "request_id": "e53887a2-fe5a-2f27-2121-c716a697f0e8",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 25920000,
  "data": {
    **************
    **************
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

With this child token, users can only read the data but neither write or delete data as configured in auth-policy.

If you still want to secure child token you can save it in Jenkins credentials and will be fetched into bash script when build triggered.

1

I've ended up at exactly the same spot.

We have a Vault server holding a good deal of secrets for different environments and various endpoints for each environment, so we set up kv engines for every environment and added a descriptive path for every secret. For instance, we have called our engines constant-text/environment/engine-name and all Database users are somewhere in db/schema-name/username, db/schema-name/password, db/schema-name/tns-string (we use Oracle).

I will not cover SSL/TLS setup, as you said you have TLS installed.

Sources: in Vault CLI there is a section about environment variables.

This is what I did:

!#/bin/bash
export VAULT_ADDR=https://address.of.vault.server/
export VAULT_TOKEN=value-of-my-root-token-or-an-authorized-token
vault kv get const/environment/engine-name db/schema/tns-string | command

where 'command' will read the secret from stdin. It will be required to parse the vault output format (example given in subsection "Reading Data" in 1) but a manual test shows more than 1000 words. As already mentionned by Tensibai above, there are ways to parse JSON, which I haven't implemented yet. I'm afraid it can't get much simpler. It can get more complex and more secure if you pass the address and the token to the script and then use

export VAULT_ADDR=$1
export VAULT_TOKEN=$2

to pass them on to the environment, but don't forget adding

set VAULT_ADDR=
set VAULT_TOKEN=

at the end to clear the variables again.

0

If you want pure-play bash, then you're already doing the right thing, curl, and although you don't mention it, it's a good idea to use jq when using REST APIs for parsing results, just remember to request json.

If you can install extra tools, then you can install the vault cli itself. Personally, I never install anything except docker, so I'd run that in a container:

$ alias vault="docker run -it --rm --cap-add IPC_LOCK vault"
$ vault --version
Vault v1.5.3 (9fcd81405feb320390b9d71e15a691c3bc1daeef)

As I'm sure you know, the vault cli, just like curl, can return json from most of its commands, which you can pipe to jq (again) for parsing (which is a lot easier than using go-templates IMO).

However, having said that, I have a suspicion that maybe you're asking the wrong question. Getting a secret on your laptop isn't really what vault is for and maybe you're just writing the code on your laptop ready to deploy to a server somewhere in order to provide a secret to that server. If that's the case then doing this in bash may be the wrong approach due to the issue of initial trust.

If you need to get a secret from vault then you have to authenticate first and get a token that has the correct authority (policies) to access the secret you're interested in. Authentication requires some sort of credentials (jwt/username-password/etc.). How do you get those creds to your bash script? You can't commit them to git.

Perhaps you're provisioning an ec2 instance with user-data that contains the creds the script needs (or any other cloud or on-prem equiv. of this process). If that's the case, then you can work out how to provision the machine with the secret it needs instead of the creds to get the secret. This is of course much easier with containers and kubernetes, but you can do the same thing with VMs of various types. In AWS you can do it with Lambda. In Azure with Azure Functions. On-prem you can do it using VMWare. With this approach, you'd be pushing the secrets management function into the platform (shift-left if you like that vocab.), rather than making it a responsibility of your application layer.

Maybe this is going too far and you're just playing around and learning about vault, but you did ask for help with direction. But if it's more than that, you should look at things like SPIFFE and check out the patterns hashicorp has implemented with its K8S sidecar.

-1

Vault has a REST API available for reading.

https://www.vaultproject.io/api/index.html

In your script, just curl the API.

3
  • 1
    I am aware of that API. I was looking for something better than curl and maybe something more secure. I have given up on that idea and am using curl instead. So much for nice and simple/secure.
    – user8517
    Jul 13 '18 at 13:04
  • 1
    You have to install an SSL on Vault for that kind of thing to be secure
    – Mike
    Jul 13 '18 at 13:37
  • I have TLS installed, still was hoping for a more secure way of doing it.
    – user8517
    Jul 13 '18 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy