0

My system is running Ubuntu 18.04.2 LTS and Docker version 18.09.5, build e8ff056, Gitlab version gitlab-ee=12.0.3-ee.0.

I am trying to spin up a docker container with the following run command:

docker run -i \
  --hostname gitlab2.michael-thompson.net \
  --name gitlab \
  --restart unless-stopped \
  --volume /docker/gitlab/config:/etc/gitlab:Z \
  --volume /docker/gitlab/log:/var/log/gitlab:Z \
  --volume /docker/gitlab/data:/var/opt/gitlab:Z \
  --net=bridge \
  --env TZ=Australia/Perth \
  gitlab/gitlab-ee:latest

The container boots for 20 seconds and then restarts (and continues in a never ending loop). Everything seems to run well until Gitlab attempts to "migrate" the database, and spits out the following error:

Recipe: gitlab::database_migrations
  * bash[migrate gitlab-rails database] action run
    [execute] rake aborted!
              StandardError: An error has occurred, this and all later migrations canceled:

              PG::DuplicateTable: ERROR:  relation "audit_events" already exists
              : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp)
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'

              Caused by:
              ActiveRecord::StatementInvalid: PG::DuplicateTable: ERROR:  relation "audit_events" already exists
              : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp)
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'

              Caused by:
              PG::DuplicateTable: ERROR:  relation "audit_events" already exists
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'
              Tasks: TOP => db:migrate
              (See full trace by running task with --trace)
              == 20141118150935 AddAuditEvent: migrating ====================================
              -- adapter_name()
                 -> 0.0000s
              -- adapter_name()
                 -> 0.0000s
              -- create_table(:audit_events, {:id=>:integer})

    ================================================================================
    Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of "bash"  "/tmp/chef-script20190704-26-1gi3jcd" ----
    STDOUT: rake aborted!
    StandardError: An error has occurred, this and all later migrations canceled:

    PG::DuplicateTable: ERROR:  relation "audit_events" already exists
    : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp)
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'

    Caused by:
    ActiveRecord::StatementInvalid: PG::DuplicateTable: ERROR:  relation "audit_events" already exists
    : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp)
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'

    Caused by:
    PG::DuplicateTable: ERROR:  relation "audit_events" already exists
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    Tasks: TOP => db:migrate
    (See full trace by running task with --trace)
    == 20141118150935 AddAuditEvent: migrating ====================================
    -- adapter_name()
       -> 0.0000s
    -- adapter_name()
       -> 0.0000s
    -- create_table(:audit_events, {:id=>:integer})
    STDERR:
    ---- End output of "bash"  "/tmp/chef-script20190704-26-1gi3jcd" ----
    Ran "bash"  "/tmp/chef-script20190704-26-1gi3jcd" returned 1

    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

     53: bash "migrate gitlab-rails database" do
     54:   code <<-EOH
     55:     set -e
     56:     log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
     57:     umask 077
     58:     /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
     59:     STATUS=${PIPESTATUS[0]}
     60:     echo $STATUS > #{db_migrate_status_file}
     61:     exit $STATUS
     62:   EOH
     63:   environment env_variables unless env_variables.empty?
     64:   notifies :run, "execute[clear the gitlab-rails cache]", :immediately
     65:   dependent_services.each do |svc|
     66:     notifies :restart, svc, :immediately
     67:   end
     68:   not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
     69:   only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
     70: end

    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:53:in `from_file'

    bash("migrate gitlab-rails database") do
      action [:run]
      default_guard_interpreter :default
      command nil
      backup 5
      returns 0
      user nil
      interpreter "bash"
      declared_type :bash
      cookbook_name "gitlab"
      recipe_name "database_migrations"
      code "    set -e\n    log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n    umask 077\n    /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n    STATUS=${PIPESTATUS[0]}\n    echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93\n    exit $STATUS\n"
      domain nil
      not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93 | grep -Fx 0)"
      only_if { #code block }
    end

    System Info:
    ------------
    chef_version=13.6.4
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client

Recipe: gitlab::gitlab-rails
  * execute[clear the gitlab-rails cache] action run

I've spent the best part of a day on this and can't figure out why it is crashing. I'm not using an external database engine / container, so I expect that that should be brand new with my container, and don't understand how it could be that the table already exists.

I have run the suggested docker exec -it gitlab update-permissions however given my suspicions it is database related, I had low hopes to which I seemed to be correct when trying to start the container again.

I have also tried updating my system via apt (with no luck, to no surprise), stopping and starting my container, to find the same error. I am already running 2 docker containers at this very moment on the same box, without a problem. I am also running another instance of Gitlab elsewhere (not inside a container) with the same config (although tweaked for the new system), also without a problem. It was my intention to migrate into the docker Gitlab if I could get it working.

Thanks

Improve this question
6
  • gitlab-ee being a commercial edition, you should contact gitlab support for help.
    – Tensibai
    Commented Jul 4, 2019 at 9:39
  • Interestingly enough, when I first installed Gitlab (previously, not as a docker container), it was ce. I did not knowingly upgrade to ee and am not paying for enterprise support, and according to the license model I read up on yesterday, it is OK to run ee without paying, and it will continue to operate in the same manner as ce. The reason I am installing ee is so that I can import my backup from my current production gitlab. In short, I do not have enterprise support for this as I am not a paying customer.
    – Michael Thompson
    Commented Jul 5, 2019 at 1:04
  • 1
    According to the error message, the DB content (within /var/opt/gitlab) is proabbly corrupted, the state of the DB doesn't match the recorded state of migration tasks, you should restore a backup before retrying
    – Tensibai
    Commented Jul 5, 2019 at 7:32
  • This is on the initial boot of the container. It is automatically trying to run the initial database migration. I do have a backup which I wish to restore into, but I need gitlab to be initialised before I can do that.
    – Michael Thompson
    Commented Jul 8, 2019 at 2:42
  • 1
    You are mounting /var/opt/gitlab as a volume from the host, so there's obviously something broken in those datas, just empty it to restart from scratch
    – Tensibai
    Commented Jul 8, 2019 at 6:11

1 Answer 1

Reset to default
1

According to the error message

PG::DuplicateTable: ERROR: relation "audit_events" already exists

The DB content (within /var/opt/gitlab) is probably corrupted, the state of the DB doesn't match the recorded state of migration tasks.

As you're mounting /var/opt/gitlab as a volume from the host with --volume /docker/gitlab/data:/var/opt/gitlab:Z, there's obviously something broken in those datas. you should just empty it to restart from scratch and restore a backup once bootstrapped properly.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.