Error while popping out Flask app context

Question

I am trying to create an async API using threading (Celery is an overkill in my case). To achieve the same, I subclassed the Thread class in following manner. Since the code that will run inside thread requires app as well as request context, I have pushed both the contexts in the stack.

from threading import Thread
from flask import _app_ctx_stack, _request_ctx_stack


class AppContextThread(Thread):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        # App context and request context are popped from the stack once request is completed but we require them for
        # accessing application data. Hence, storing them and then again pushing into the stack.
        self.app_context = _app_ctx_stack.top
        self.request_context = _request_ctx_stack.top

    def run(self):
        self.app_context.push()
        self.request_context.push()
        super().run()
        print(f"App context top: {_app_ctx_stack.top}")
        print(f"Req context top: {_request_ctx_stack.top}")
        print(f"After app_context: {self.app_context}")
        print(f"After request_context: {self.request_context}")
        self.request_context.pop()
        print(f"After request_context pop: {self.request_context}")
        print(f"After request_context pop -> app_context: {self.app_context}")
        self.app_context.pop()
        print(f"After app_context pop: {self.app_context}")

Now when I try to pop app context out of the stack I get following error even though app context is present in the stack (printed logs for the same).

App context top: <flask.ctx.AppContext object at 0x7f7f512100f0>
Req context top: <RequestContext 'http://0.0.0.0:8000/rest/v1/api' [PUT] of app.app>
After app_context: <flask.ctx.AppContext object at 0x7f7f512100f0>
After request_context: <RequestContext 'http://0.0.0.0:8000/rest/v1/api' [PUT] of app.app>
After request_context pop: <RequestContext 'http://0.0.0.0:8000/rest/v1/api' [PUT] of app.app>
After request_context pop -> app_context: <flask.ctx.AppContext object at 0x7f7f512100f0>


Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/Users/app/utils/app_context_thread.py", line 27, in run
    self.app_context.pop()
  File "/Users/venv/lib/python3.6/site-packages/flask/ctx.py", line 235, in pop
    % (rv, self)
AssertionError: Popped wrong app context.  (None instead of <flask.ctx.AppContext object at 0x7f7f512100f0>)

Could anyone please point me out what I am doing wrong here?

Answer 1

Hopefully this will help someone out even though this question is over a year old.

I have ran into this issue while working on something similar in Flask (threading out and needing the App and Request contexts). I also attempted to create a subclass of the python Thread class to automatically add the contexts to the thread. My code for the class is below and similar to yours but does not have the assertion errors you are coming across.

import threading
from flask import has_app_context, has_request_context, _app_ctx_stack, _request_ctx_stack


class FlaskThread(threading.Thread):
    def __init__(self, group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None):
        super().__init__(group=group, target=target, name=name, args=args, kwargs=kwargs, daemon=daemon)

        # first check if there is application or request context
        if not has_app_context():
            raise RuntimeError("Running outside of Flask AppContext")
        if not has_request_context():
            raise RuntimeError("Running outside of Flask RequestContext")
        
        # set the app and request context variables
        self.app_ctx = _app_ctx_stack.top
        self.request_ctx = _request_ctx_stack.top

    def run(self):
        self.app_ctx.push()
        self.request_ctx.push()
        super().run()

The assertion error states that self.request_context.pop() and self.app_context.pop() are popping nothing from the LocalStack. I believe, from my understanding, the reason nothing is being popped is because the contexts have already been popped by the main flask process.

If you take a look at the Flask source code in flask/ctx.py Github link here, I add the following print statements.

class AppContext

    def push(self):
        """Binds the app context to the current context."""
        self._refcnt += 1
        if hasattr(sys, "exc_clear"):
            sys.exc_clear()
        _app_ctx_stack.push(self)
        print(threading.current_thread(), "PUSH AppContext:", self, "ACEND")
        appcontext_pushed.send(self.app)

    def pop(self, exc=_sentinel):
        """Pops the app context."""
        try:
            self._refcnt -= 1
            if self._refcnt <= 0:
                if exc is _sentinel:
                    exc = sys.exc_info()[1]
                self.app.do_teardown_appcontext(exc)
        finally:
            rv = _app_ctx_stack.pop()
            print(threading.current_thread(), "POP AppContext:", rv, "ACEND")
        assert rv is self, "Popped wrong app context.  (%r instead of %r)" % (rv, self)
        appcontext_popped.send(self.app)

class RequestContext

    def push(self):
        """Binds the request context to the current context."""
        # If an exception occurs in debug mode or if context preservation is
        # activated under exception situations exactly one context stays
        # on the stack.  The rationale is that you want to access that
        # information under debug situations.  However if someone forgets to
        # pop that context again we want to make sure that on the next push
        # it's invalidated, otherwise we run at risk that something leaks
        # memory.  This is usually only a problem in test suite since this
        # functionality is not active in production environments.
        top = _request_ctx_stack.top
        if top is not None and top.preserved:
            top.pop(top._preserved_exc)

        # Before we push the request context we have to ensure that there
        # is an application context.
        app_ctx = _app_ctx_stack.top
        if app_ctx is None or app_ctx.app != self.app:
            app_ctx = self.app.app_context()
            app_ctx.push()
            self._implicit_app_ctx_stack.append(app_ctx)
        else:
            self._implicit_app_ctx_stack.append(None)

        if hasattr(sys, "exc_clear"):
            sys.exc_clear()

        _request_ctx_stack.push(self)
        print(threading.current_thread(), "PUSH RequestContext:", self, "RQEND")
        # Open the session at the moment that the request context is available.
        # This allows a custom open_session method to use the request context.
        # Only open a new session if this is the first time the request was
        # pushed, otherwise stream_with_context loses the session.
        if self.session is None:
            session_interface = self.app.session_interface
            self.session = session_interface.open_session(self.app, self.request)

            if self.session is None:
                self.session = session_interface.make_null_session(self.app)

        if self.url_adapter is not None:
            self.match_request()

    def pop(self, exc=_sentinel):
        """Pops the request context and unbinds it by doing that.  This will
        also trigger the execution of functions registered by the
        :meth:`~flask.Flask.teardown_request` decorator.

        .. versionchanged:: 0.9
           Added the `exc` argument.
        """
        app_ctx = self._implicit_app_ctx_stack.pop()

        try:
            clear_request = False
            if not self._implicit_app_ctx_stack:
                self.preserved = False
                self._preserved_exc = None
                if exc is _sentinel:
                    exc = sys.exc_info()[1]
                self.app.do_teardown_request(exc)

                # If this interpreter supports clearing the exception information
                # we do that now.  This will only go into effect on Python 2.x,
                # on 3.x it disappears automatically at the end of the exception
                # stack.
                if hasattr(sys, "exc_clear"):
                    sys.exc_clear()

                request_close = getattr(self.request, "close", None)
                if request_close is not None:
                    request_close()
                clear_request = True
        finally:
            rv = _request_ctx_stack.pop()
            print(threading.current_thread(), "POP RequestContext: ", rv, "RQEND")
            # get rid of circular dependencies at the end of the request
            # so that we don't require the GC to be active.
            if clear_request:
                rv.request.environ["werkzeug.request"] = None

            # Get rid of the app as well if necessary.
            if app_ctx is not None:
                app_ctx.pop(exc)

            assert rv is self, "Popped wrong request context. (%r instead of %r)" % (
                rv,
                self,
            )

I have an endpoint in Flask that will thread out via FlaskThread. It does not join with the FlaskThread and will immediately return a response. And here is the output that I have edited for ease of reading since there were printing issues due to concurrency. Thread-10 is the main Flask process. Thread-11 is the FlaskThread.

1. <Thread(Thread-10, started daemon 21736)> PUSH AppContext: <flask.ctx.AppContext object at 0x000001924995B940> ACEND
2. <Thread(Thread-10, started daemon 21736)> PUSH RequestContext: <RequestContext 'http://localhost:7777/' [GET] of LIMS> RQEND
3. <FlaskThread(Thread-11, started daemon 38684)> PUSH AppContext: <flask.ctx.AppContext object at 0x000001924995B940> ACEND
4. <Thread(Thread-10, started daemon 21736)> POP RequestContext:   <RequestContext 'http://localhost:7777/' [GET] of LIMS> RQEND
5. <FlaskThread(Thread-11, started daemon 38684)> POP AppContext: <flask.ctx.AppContext object at 0x000001924995B940> ACEND
6. <Thread(Thread-10, started daemon 21736)> PUSH RequestContext: <RequestContext 'http://localhost:7777/' [GET] of LIMS> RQEND

On first call to the endpoint, the main Flask application will push both AppContext and RequestContext (see line 1 and 2). In my code, this endpoint will make a FlaskThread, and in my FlaskThread run() function, I push both contexts (see line 3 and 5). While this is running, the main Flask application will pop the contexts when a response has been generated.

In YOUR code, because you call another pop() in your thread, it will throw the AssertionError because that context has already been popped.

I believe this is because in the thread constructor, there is code that saves the context from the top of the stack to the thread, which explains why the thread will keep running with the data from the contexts, but when you pop, it will give an error because the main Flask process has already popped this context.