Passing django.core.mail.EmailMultiAlternatives instance as async_task with Django-Q

Question

An E-mail template is build-up and send weekly to more than 2000 email recipients (bcc). I must run it as background since it takes few minutes to get rendering back and that some E-mail provider block Senders because of mass mail in a few period of time. I cannot use SendinBlue or MailChimp or any others for contractual reasons.

I use 'django.core.mail.EmailMultiAlternatives' instance in order to add images '(self.attach(img)' and documents '(self.attach_file(filepath)'. After creation all context data like <img src="cid:filename.jpg". I use render_to_string(template_name, request=request, context=context).strip() to get back my filled-in HTML E-mail template.

Background running with async_task of Django-Q

When I pass my EmailMultiAlternatives instance, I get the error:

Traceback (most recent call last):
  File ".pyenv/versions/3.9.7/lib/python3.9/multiprocessing/queues.py", line 245, in _feed
    obj = _ForkingPickler.dumps(obj)
  File ".pyenv/versions/3.9.7/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_thread.RLock' object

If I build my instance template in the task (but with code redundancy), it works but then since I want to loop over my 2000 recipients to send the mail by chunk of 99 every n minutes (avoid mail server blocking), again, I need to pass the EmailMultiAlternatives instanceas a parameter and I get back the same error.

Any idea on how to proceed?

Answer 1

Here is the solution: https://github.com/joeyespo/django-q-email. Thanks a lot to Joe Esposito and Ankit!

As mentioned in the configuration section: DJANGO_Q_EMAIL_USE_DICTS - Store Python dictionaries instead of pickled EmailMessage and EmailMultiAlternatives (default: True)