'Symfony 3.4 and MySQl 8. General error: 2006 MySQL server has gone away when running a transaction in a rabbitmq queue

I have one system (Symfony 5, symfony/messenger) sending data to another (Symfony 3.4, enqueue/enqueue-bundle 0.9) using a rabbitmq queue. The data is used to create entities (stored in a MySql 8 database) that are executed within a transaction in case of a failure, to return the Database to the initial state.

My code is something like this (on the Symfony 3.4 side where the error is):

function runInsideTx(callable $fn)
    {
        $this->em->getConnection()->beginTransaction();

        try {
            $resp = $fn();

            $this->em->flush();
            $this->em->getConnection()->commit();

            return $resp;
        } catch (Exception $ex) {
            $this->em->getConnection()->rollBack();

            throw $ex;
        }
    }

And inside runInsideTx I have code like

$this->functions->runInsideTx(function () use ($message) {
    $entity = new SomeEntity();
    ...
    $this->em->persist($entity);
    $this->em->flush();
})

Workers run inside a Docker container using supervisor:

[program:interop-manager_tasks-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /app/bin/console enqueue:transport:consume manager_tasks_processor manager_tasks_prod --message-limit=1 -vvv
autostart=true
autorestart=true
numprocs=2
redirect_stderr=true
stopwaitsecs=3600
stdout_logfile = /var/log/manager_tasks_worker_out.log
stderr_logfile = /var/log/manager_tasks_worker_err.log
user=application
environment=HOME="/home/application",USER="application"

Randomly the error General error: 2006 MySQL server has gone away occurs. Initially I had a MySql 5 Database and changed it to a MySql 8, which I am running inside a docker container with a single database.

Update I have this error too: Warning: Error while sending QUERY packet. This happens sometimes. It could be that it sends the task to the queue three times and it fails the first two and the third it processes it without problems

I also added the following parameters when running the MySQl server:

command: --sql_mode='STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION' --default-authentication-plugin=mysql_native_password --max-allowed-packet=1G --innodb_buffer_pool_size=512M --innodb_lock_wait_timeout=500

But the problem persists.

SHOW VARIABLES LIKE '%timeout%'

+-----------------------------------+----------+
| Variable_name                     | Value    |
+-----------------------------------+----------+
| connect_timeout                   | 10       |
| delayed_insert_timeout            | 300      |
| have_statement_timeout            | YES      |
| innodb_flush_log_at_timeout       | 1        |
| innodb_lock_wait_timeout          | 500      |
| innodb_rollback_on_timeout        | OFF      |
| interactive_timeout               | 28800    |
| lock_wait_timeout                 | 31536000 |
| mysqlx_connect_timeout            | 30       |
| mysqlx_idle_worker_thread_timeout | 60       |
| mysqlx_interactive_timeout        | 28800    |
| mysqlx_port_open_timeout          | 0        |
| mysqlx_read_timeout               | 30       |
| mysqlx_wait_timeout               | 28800    |
| mysqlx_write_timeout              | 60       |
| net_read_timeout                  | 30       |
| net_write_timeout                 | 60       |
| replica_net_timeout               | 60       |
| rpl_stop_replica_timeout          | 31536000 |
| rpl_stop_slave_timeout            | 31536000 |
| slave_net_timeout                 | 60       |
| wait_timeout                      | 28800    |
+-----------------------------------+----------+


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source