I am trying to setup a consul 1.3.1 server cluster. I am having a really hard time with it. Currently I can setup a single node perfectly fine and it seems to function as needed. I do this using a acl_master_token in my config file. However when I add in another node that is also a server but not bootstrapped I run into a whole host of issues. Some of the mains ones are related to joining the cluster (can't it gives me a 403 error) which from what I have read means you need the acl_agent_token and when I set that in my config file I still get a 403 error.

I am assuming that I somehow need to create a policy and then apply that policy to my token. I am not sure how to do this inside a ruby script which is what I have to use to set this up (legacy stuff is in it and I can't remove the consul setup to be manual because it all is part of a chef cookbook).

Here is my config file, I have nothing else for config other than I start the consul service on both nodes using: consul agent -data-dir /work-disk/consul_data -config-file /work-disk/consul/conf.json -bind


    "bootstrap_expect": 2, 
    "client_addr": "",
    "datacenter": "<%= @datacenter %>",
    "acl_datacenter": "<%= @datacenter %>",
    "acl_down_policy": "deny",
    "acl_master_token": "<%= @acl_master_token %>",
    "acl_agent_token": "<%= @acl_agent_token %>",
    "data_dir": "<%= @consul_data_dir %>",
    "domain": "consul",
    "enable_script_checks": true,
    "dns_config": {
        "enable_truncate": true,
        "only_passing": true
    "enable_syslog": true,
    "encrypt": "<%= @consul_encryption_key %>",
    "leave_on_terminate": true,
    "log_level": "INFO",
    "rejoin_after_leave": true,
    "server": true

Ignore the chef variable parts, I figured I would include my template rather than my actual conf.json because I would have to redact the values anyway.


When I go to do consul monitor I get a 403 permission error even when I use the master token.

Error message: Error starting monitor: Unexpected response code: 403 (Permission denied)

  • Could you please provide the separate configuration for the existing server and new server? I assume the provided one is for the one node consul cluster. – JSapkota Dec 9 '18 at 4:05
  • @JSapkota The one provided is for a two node cluster, what makes you think it is one node? – user8517 Dec 10 '18 at 12:55
  • Missing join or retry_join parameters? consul.io/docs/agent/options.html#_join – JSapkota Dec 10 '18 at 14:42
  • @JSapkota I am using the join command in my setup because of how I can get the values to put in the join just doesn't work well if I add the join into the conf file. I did add it in to test it and everything works out well. Maybe I need to change how I get the node IPs.. – user8517 Dec 10 '18 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy