I am trying to setup a consul 1.3.1 server cluster. I am having a really hard time with it. Currently I can setup a single node perfectly fine and it seems to function as needed. I do this using a acl_master_token in my config file. However when I add in another node that is also a server but not bootstrapped I run into a whole host of issues. Some of the mains ones are related to joining the cluster (can't it gives me a 403 error) which from what I have read means you need the acl_agent_token and when I set that in my config file I still get a 403 error.

I am assuming that I somehow need to create a policy and then apply that policy to my token. I am not sure how to do this inside a ruby script which is what I have to use to set this up (legacy stuff is in it and I can't remove the consul setup to be manual because it all is part of a chef cookbook).

Here is my config file, I have nothing else for config other than I start the consul service on both nodes using: consul agent -data-dir /work-disk/consul_data -config-file /work-disk/consul/conf.json -bind


    "bootstrap_expect": 2, 
    "client_addr": "",
    "datacenter": "<%= @datacenter %>",
    "acl_datacenter": "<%= @datacenter %>",
    "acl_down_policy": "deny",
    "acl_master_token": "<%= @acl_master_token %>",
    "acl_agent_token": "<%= @acl_agent_token %>",
    "data_dir": "<%= @consul_data_dir %>",
    "domain": "consul",
    "enable_script_checks": true,
    "dns_config": {
        "enable_truncate": true,
        "only_passing": true
    "enable_syslog": true,
    "encrypt": "<%= @consul_encryption_key %>",
    "leave_on_terminate": true,
    "log_level": "INFO",
    "rejoin_after_leave": true,
    "server": true

Ignore the chef variable parts, I figured I would include my template rather than my actual conf.json because I would have to redact the values anyway.


When I go to do consul monitor I get a 403 permission error even when I use the master token.

Error message: Error starting monitor: Unexpected response code: 403 (Permission denied)

  • Could you please provide the separate configuration for the existing server and new server? I assume the provided one is for the one node consul cluster. – JSapkota Dec 9 '18 at 4:05
  • @JSapkota The one provided is for a two node cluster, what makes you think it is one node? – Josh Kirby Dec 10 '18 at 12:55
  • Missing join or retry_join parameters? consul.io/docs/agent/options.html#_join – JSapkota Dec 10 '18 at 14:42
  • @JSapkota I am using the join command in my setup because of how I can get the values to put in the join just doesn't work well if I add the join into the conf file. I did add it in to test it and everything works out well. Maybe I need to change how I get the node IPs.. – Josh Kirby Dec 10 '18 at 14:53

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.