'How to fix "InternalError: variable not found in subplan target list"

I have a legacy Django project (django-1.1.29) and some tests in pytest (pytest-4.3.0), all running inside Docker. The database is PostgreSQL 10 and is a docker-compose service on which the application depends on. The python version is 2.7.18.

Lately the tests started to fail with a strange error:

InternalError: variable not found in subplan target list.

The error occurs only when I count the number of objects of a certain model, e.g. Problem.objects.count(). The instruction is turned into the following query

(0.000) SELECT COUNT(*) AS "__count" FROM "problems_problem"; args=() by Django.

The whole log is here:

self = <integration_tests.project_name.subjects.test_views.test_theme_problem_view_set.TestThemeProblemViewSet object at 0x7f01faccbe50>
jclient = <project_name.common.test_utils.json_client.JSONClient object at 0x7f01fadb8ed0>

    def test_all_is_ok(self, jclient, subject_model, content_manager):
        url, data = self._main_prepare(jclient, subject_model, content_manager)
        response = jclient.post_json(url, data[0])
        assert response.status_code == 201
>       assert Problem.objects.count() == 1

integration_tests/project_name/subjects/test_views/test_theme_problem_view_set.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py:85: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
/usr/local/lib/python2.7/dist-packages/django/db/models/query.py:364: in count
    return self.query.get_count(using=self.db)
/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:499: in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:480: in get_aggregation
    result = compiler.execute_sql(SINGLE)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <django.db.models.sql.compiler.SQLCompiler object at 0x7f01faee3a50>
result_type = 'single', chunked_fetch = False

    def execute_sql(self, result_type=MULTI, chunked_fetch=False):
        """
        Run the query against the database and returns the result(s). The
        return value is a single data item if result_type is SINGLE, or an
        iterator over the results if the result_type is MULTI.
    
        result_type is either MULTI (use fetchmany() to retrieve all rows),
        SINGLE (only retrieve a single row), or None. In this last case, the
        cursor is returned if any query is executed, since it's used by
        subclasses such as InsertQuery). It's possible, however, that no query
        is needed, as the filters describe an empty set. In that case, None is
        returned, to avoid any unnecessary database interaction.
        """
        if not result_type:
            result_type = NO_RESULTS
        try:
            sql, params = self.as_sql()
            if not sql:
                raise EmptyResultSet
        except EmptyResultSet:
            if result_type == MULTI:
                return iter([])
            else:
                return
        if chunked_fetch:
            cursor = self.connection.chunked_cursor()
        else:
            cursor = self.connection.cursor()
        try:
            cursor.execute(sql, params)
        except Exception as original_exception:
            try:
                # Might fail for server-side cursors (e.g. connection closed)
                cursor.close()
            except Exception:
                # Ignore clean up errors and raise the original error instead.
                # Python 2 doesn't chain exceptions. Remove this error
                # silencing when dropping Python 2 compatibility.
                pass
>           raise original_exception
E           InternalError: variable not found in subplan target list

/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py:899: InternalError

I would really appreciate if anyone could help me fix this issue or at least give some hints where to find the reasons.

Here is what I've attempted to do so far but none of these helped:

  • Use different versions of PostgreSQL (10, 11, 12, 13)
  • Disable the server-side cursor
pythondjangopostgresqldockerpytest


Solution 1:[1]

It appears postgres has shipped something broken into all their minor versions:

https://www.postgresql.org/message-id/2121219.1644607692%40sss.pgh.pa.us

For example, the postgres:12 dockerhub is now the same as the postgres:12.10 dockerhub image rather than 12.9. If you explicitly specify postgres:12.9 (or the previous minor release for any of the other versions), I believe it will start working.

Solution 2:[2]

I had the same issue. Temporary fix but what I did was modify the query performed by the admin.

Try this in your admin.py

class ProblemAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        return Problem.objects.filter(id__gte=0)

Edit: This only fixes issues you may have with your django admin

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1
Solution 2

Related Questions