Docker-Compose Init Postgres Failing

Question

I've been attempting to make a docker-compose file work, and so far it's been closer and closer, but not quite there. Maybe asking the questions will be useful to someone else in the future. Anyway, the previous thread was here:

Docker-Compose Postgresql Init Entrypoint

From which I learned,

1. between runs a docker-compose file should have docker compose volumes --down and
2. if the POSTGRES_DATABASE isn't specified, then the POSTGRES_NAME will replicate to the POSTGRES_DATABASE.
3. another suggestion from Discord has been to rewrite some of the volume signature to include the file path to init.sql as in docker-entrypoint-initdb.d/init.sql

So if I docker-compose down --volume and then sudo docker-compose build && docker-compose up with

version: "3.9"
   
services:
  db:
    restart: always
    image: postgres
    volumes:
      # - ./data/db:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.db
    environment:
      - POSTGRES_NAME=dev-postgres
      - POSTGRES_USER=pixel
      - POSTGRES_DATABASE=lightchan
      - POSTGRES_PASSWORD=stardust
  web:
    build: .
    restart: always
    command: sh -c "./waitfor.sh db:5432 -- python3 manage.py runserver"
    volumes:
      - .:/code
    ports:
      - "8001:8001"
    environment:
      - POSTGRES_NAME=dev-postgres
      - POSTGRES_USER=pixel
      - POSTGRES_DATABASE=lightchan
      - POSTGRES_PASSWORD=stardust
    depends_on:
      - db

Then the following is output:

Attaching to lightchan-db-1, lightchan-web-1
lightchan-db-1   | The files belonging to this database system will be owned by user "postgres".
lightchan-db-1   | This user must also own the server process.
lightchan-db-1   | 
lightchan-db-1   | The database cluster will be initialized with locale "en_US.utf8".
lightchan-db-1   | The default database encoding has accordingly been set to "UTF8".
lightchan-db-1   | The default text search configuration will be set to "english".
lightchan-db-1   | 
lightchan-db-1   | Data page checksums are disabled.
lightchan-db-1   | 
lightchan-db-1   | fixing permissions on existing directory /var/lib/postgresql/data ... ok
lightchan-db-1   | creating subdirectories ... ok
lightchan-db-1   | selecting dynamic shared memory implementation ... posix
lightchan-db-1   | selecting default max_connections ... 100
lightchan-db-1   | selecting default shared_buffers ... 128MB
lightchan-db-1   | selecting default time zone ... Etc/UTC
lightchan-db-1   | creating configuration files ... ok
lightchan-db-1   | running bootstrap script ... ok
lightchan-db-1   | performing post-bootstrap initialization ... ok
lightchan-db-1   | syncing data to disk ... ok
lightchan-db-1   | 
lightchan-db-1   | initdb: warning: enabling "trust" authentication for local connections
lightchan-db-1   | You can change this by editing pg_hba.conf or using the option -A, or
lightchan-db-1   | --auth-local and --auth-host, the next time you run initdb.
lightchan-db-1   | 
lightchan-db-1   | Success. You can now start the database server using:
lightchan-db-1   | 
lightchan-db-1   |     pg_ctl -D /var/lib/postgresql/data -l logfile start
lightchan-db-1   | 
lightchan-db-1   | waiting for server to start....2022-03-02 02:08:59.604 UTC [49] LOG:  starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
lightchan-db-1   | 2022-03-02 02:08:59.605 UTC [49] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
lightchan-db-1   | 2022-03-02 02:08:59.611 UTC [50] LOG:  database system was shut down at 2022-03-02 02:08:59 UTC
lightchan-db-1   | 2022-03-02 02:08:59.617 UTC [49] LOG:  database system is ready to accept connections
lightchan-db-1   |  done
lightchan-db-1   | server started
lightchan-db-1   | CREATE DATABASE
lightchan-db-1   | 
lightchan-db-1   | 
lightchan-db-1   | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
lightchan-db-1   | psql:/docker-entrypoint-initdb.d/init.sql: error: could not read from input file: Is a directory
lightchan-db-1 exited with code 1
lightchan-db-1   | 2022-03-02 02:09:00.594 UTC [28] LOG:  database system was not properly shut down; automatic recovery in progress
lightchan-db-1   | 2022-03-02 02:09:00.596 UTC [28] LOG:  redo starts at 0/16FB5C8
lightchan-db-1   | 2022-03-02 02:09:00.596 UTC [28] LOG:  invalid record length at 0/16FB6C0: wanted 24, got 0
lightchan-db-1   | 2022-03-02 02:09:00.596 UTC [28] LOG:  redo done at 0/16FB678 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
lightchan-db-1   | 2022-03-02 02:09:00.604 UTC [1] LOG:  database system is ready to accept connections
lightchan-web-1  | Watching for file changes with StatReloader
lightchan-db-1   | 2022-03-02 02:09:01.480 UTC [36] FATAL:  database "lightchan" does not exist

So the database can't be seen on the web side but PSQL says it created it. If I uncomment out the data volume on PSQL then I get:

Attaching to lightchan-db-1, lightchan-web-1
lightchan-db-1   | 
lightchan-db-1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
lightchan-db-1   | 
lightchan-db-1   | 2022-03-02 02:11:29.346 UTC [1] LOG:  starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
lightchan-db-1   | 2022-03-02 02:11:29.346 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
lightchan-db-1   | 2022-03-02 02:11:29.346 UTC [1] LOG:  listening on IPv6 address "::", port 5432
lightchan-db-1   | 2022-03-02 02:11:29.359 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
lightchan-db-1   | 2022-03-02 02:11:29.420 UTC [28] LOG:  database system was interrupted; last known up at 2022-03-02 02:11:06 UTC
lightchan-web-1  | Watching for file changes with StatReloader
lightchan-db-1   | 2022-03-02 02:11:30.835 UTC [30] FATAL:  the database system is starting up
lightchan-web-1  | Exception in thread django-main-thread:
lightchan-web-1  | Traceback (most recent call last):
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 230, in ensure_connection
lightchan-web-1  |     self.connect()
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1  |     return func(*args, **kwargs)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 211, in connect
lightchan-web-1  |     self.connection = self.get_new_connection(conn_params)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1  |     return func(*args, **kwargs)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 199, in get_new_connection
lightchan-web-1  |     connection = Database.connect(**conn_params)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
lightchan-web-1  |     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
lightchan-web-1  | psycopg2.OperationalError: connection to server at "db" (192.168.96.2), port 5432 failed: FATAL:  the database system is starting up
lightchan-web-1  | 
lightchan-web-1  | 
lightchan-web-1  | The above exception was the direct cause of the following exception:
lightchan-web-1  | 
lightchan-web-1  | Traceback (most recent call last):
lightchan-web-1  |   File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
lightchan-web-1  |     self.run()
lightchan-web-1  |   File "/usr/local/lib/python3.8/threading.py", line 870, in run
lightchan-web-1  |     self._target(*self._args, **self._kwargs)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 64, in wrapper
lightchan-web-1  |     fn(*args, **kwargs)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/runserver.py", line 127, in inner_run
lightchan-web-1  |     self.check_migrations()
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 505, in check_migrations
lightchan-web-1  |     executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 18, in __init__
lightchan-web-1  |     self.loader = MigrationLoader(self.connection)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 53, in __init__
lightchan-web-1  |     self.build_graph()
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 223, in build_graph
lightchan-web-1  |     self.applied_migrations = recorder.applied_migrations()
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 77, in applied_migrations
lightchan-web-1  |     if self.has_table():
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 55, in has_table
lightchan-web-1  |     with self.connection.cursor() as cursor:
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1  |     return func(*args, **kwargs)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 270, in cursor
lightchan-web-1  |     return self._cursor()
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 246, in _cursor
lightchan-web-1  |     self.ensure_connection()
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1  |     return func(*args, **kwargs)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 230, in ensure_connection
lightchan-web-1  |     self.connect()
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
lightchan-web-1  |     raise dj_exc_value.with_traceback(traceback) from exc_value
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 230, in ensure_connection
lightchan-web-1  |     self.connect()
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1  |     return func(*args, **kwargs)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 211, in connect
lightchan-web-1  |     self.connection = self.get_new_connection(conn_params)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1  |     return func(*args, **kwargs)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 199, in get_new_connection
lightchan-web-1  |     connection = Database.connect(**conn_params)
lightchan-web-1  |   File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
lightchan-web-1  |     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
lightchan-web-1  | django.db.utils.OperationalError: connection to server at "db" (192.168.96.2), port 5432 failed: FATAL:  the database system is starting up
lightchan-web-1  | 
lightchan-db-1   | 2022-03-02 02:11:32.337 UTC [28] LOG:  database system was not properly shut down; automatic recovery in progress
lightchan-db-1   | 2022-03-02 02:11:32.340 UTC [28] LOG:  invalid record length at 0/16FF138: wanted 24, got 0
lightchan-db-1   | 2022-03-02 02:11:32.340 UTC [28] LOG:  redo is not required
lightchan-db-1   | 2022-03-02 02:11:32.368 UTC [1] LOG:  database system is ready to accept connections

Again I used docker-compose volumes --down to clean the errors from the previous thread, but apparently this may cause either the waitfor.sh script to run improperly when the data volumes are included or there's some other error for some reason.

Any ideas anyone? I'm out of debugging options that I know of.

Answer 1

Try opening ports in your db services:

db:
    restart: always
    image: postgres
    ports:
        - 5432:5432