Having Problems configuring solid_queue DB

[Kenneth-joseph]

Apologies if this question has already been asked but I can't find the right way to set up the solid_queue db alongside my primary db. Every time I try to start tsolid_queue I keep getting the error below, I'm all new when it comes to using multiple DBs and I'm here to learn:

bundle exec rake solid_queue:start rake aborted! ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "solid_queue_processes" does not exist LINE 10: WHERE a.attrelid = '"solid_queue_processes"'::regclass

This is my database.yml file setup:

`default: &default
adapter: postgresql
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
timeout: 5000

development:
primary:
<<: *default
database: msangia_development
queue:
<<: *default
database: msangia_queue_dev
migrations_paths: db/queue_migrate
test:
<<: *default
database: msangia_wifi_test

production:
primary:
<<: *default
database: msangia_wifi_production
queue:
<<: *default
database: app_production_queue
migrations_paths: db/queue_migrate`

development.rb environment:

`config.active_job.queue_adapter = :solid_queue

config.solid_queue.connects_to = { database: { writing: :queue } }`

The queue_schema.rb is created, but every time I run rails:db prepare, it's cleared, and nothing happens after. I did run the migrations on my terminal, but I'm still getting the same error when I try to start solid_queue. I created the queue_migrate, copied the queue_schema into it, and tried migrating it, but I'm still getting the same error.
I would appreciate any help, thanks in prior.

[rosa]

Hey @Kenneth-joseph, sorry for the trouble! Could you let me know which version of Rails you are using?

[DonSchado]

Yes, I had the same issue. (tested with 7.2 and 8.0.0.rc2)

first make sure the schema is really loaded into the db.
In my case I checked with "DB Browser for SQLite" and saw that it's missing all the tables.

bin/rails db:schema:load:queue
# Load a database schema file (either db/schema.rb or db/structure.sql, depending on `ENV['SCHEMA_FORMAT']` or `config.active_record.schema_format`) into the queue database

Then I had to add the storage config also to the test env (in database.yml).

actually rails db:reset:queue gives a hint, when you don't have it configured for test:

❯ bin/rails db:reset:queue
Dropped database 'storage/development_queue.sqlite3'
Created database 'storage/development_queue.sqlite3'
bin/rails aborted!
TypeError: Invalid type for configuration. Expected Symbol, String, or Hash. Got nil (TypeError)

       raise TypeError, "Invalid type for configuration. Expected Symbol, String, or Hash. Got #{config.inspect}"
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tasks: TOP => db:reset:queue => db:setup:queue => db:schema:load:queue => db:test:purge:queue
(See full trace by running task with --trace)

So this is the complete change I made after rails new:

diff --git a/app/jobs/application_job.rb b/app/jobs/application_job.rb
index d394c3d..a20ed1b 100644
--- a/app/jobs/application_job.rb
+++ b/app/jobs/application_job.rb
@@ -4,4 +4,7 @@ class ApplicationJob < ActiveJob::Base

  # Most jobs are safe to ignore if the underlying records are no longer available
  # discard_on ActiveJob::DeserializationError
+  def perform
+    puts 'ohai!'
+  end
end
diff --git a/config/database.yml b/config/database.yml
index 2640cb5..ed7bd1a 100644
--- a/config/database.yml
+++ b/config/database.yml
@@ -10,15 +10,25 @@ default: &default
  timeout: 5000

development:
-  <<: *default
-  database: storage/development.sqlite3
+  primary:
+    <<: *default
+    database: storage/development.sqlite3
+  queue:
+    <<: *default
+    database: storage/development_queue.sqlite3
+    migrations_paths: db/queue_migrate

# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
-  <<: *default
-  database: storage/test.sqlite3
+  primary:
+    <<: *default
+    database: storage/test.sqlite3
+  queue:
+    <<: *default
+    database: storage/test_queue.sqlite3
+    migrations_paths: db/queue_migrate


# Store production database in the storage/ directory, which by default
diff --git a/config/environments/development.rb b/config/environments/development.rb
index 4cc21c4..11a03de 100644
--- a/config/environments/development.rb
+++ b/config/environments/development.rb
@@ -69,4 +69,6 @@ Rails.application.configure do

  # Apply autocorrection by RuboCop to files generated by `bin/rails generate`.
  # config.generators.apply_rubocop_autocorrect_after_generate!
+ config.active_job.queue_adapter = :solid_queue
+ config.solid_queue.connects_to = { database: { writing: :queue } }
end
diff --git a/config/environments/test.rb b/config/environments/test.rb
index c2095b1..87e05c2 100644
--- a/config/environments/test.rb
+++ b/config/environments/test.rb
@@ -50,4 +50,6 @@ Rails.application.configure do

  # Raise error when a before_action's only/except options reference missing actions.
  config.action_controller.raise_on_missing_callback_actions = true
+ config.active_job.queue_adapter = :solid_queue
+ config.solid_queue.connects_to = { database: { writing: :queue } }
end

❯ bin/rails runner "ApplicationJob.perform_later"

❯ bin/jobs
ohai!

hope that helps :)

[indigotechtutorials]

I'm facing the same issue right now with postgres

[indigotechtutorials]

I was able to fix this error by running rails db:prepare in development so I guess that part is really important for solid queue as I didn't see any migration files being added only updates to the schema

[Kenneth-joseph]

Hello @rosa
I'm using Rails 7.2.1
The only way I had to go to make solid_queue start was to comment out the connects_to line in my dev environment.
Commented this out:
# config.solid_queue.connects_to = { database: { writing: :queue } }

and defaulted to using a single database. My only concern is whether this is the ideal approach for production since I wouldn't want my database to be overloaded or clustered.
Any help with setting up the multiple db or further essential insights pointing to the most reasonable approach to take would be highly appreciated.

[kylekeesling]

@indigotechtutorials if you’re upgrading from earlier versions you’ll need to manually step through a few of them to generate the necessary migrations. You can see notes on it here:

https://github.com/rails/solid_queue/blob/main/UPGRADING.md

[sevinchek]

I am using Rails 8 and I have solid_queue working in development.
After creating the project I ran:
bin/rails db:setup
bin/rails db:migrate

and made these configurations:

# config/database.yml
development:
  primary: &primary_development
    <<: *default
    database: atenas_development
  cache:
    <<: *primary_development
    database: athens_development_cache
    migrations_paths: db/cache_migrate
  queue:
    <<: *primary_development
    database: athens_development_queue
    migrations_paths: db/queue_migrate
  cable:
    <<: *primary_development
    database: athens_development_cable
    migrations_paths: db/cable_migrate

# config/environments/development.rb
config.active_job.queue_adapter = :solid_queue
config.solid_queue.connects_to = { database: { writing: :queue } }