Matplotlib date formatter with Latex uses math mode

Question

Overview

So I made a switch from an older version of Python + Matplotlib to a newer one and have notices the following change in behaviour which seems to be breaking my code. In the newer version, when I ask Matplotlib to use a date formatter and render the text using LaTeX, it uses math mode. This is very annoying because I have a few \n and this now breaks. For example,

ax.xaxis.set_major_formatter(DateFormatter("%d %b %Y"))

produces and even worse is if I use "%d\n%b\n%Y" which now just breaks with the below error. How can I tell matplotlib to render the date / xtick outside of math mode?.

MWE

import matplotlib as mpl
mpl.rcParams['text.usetex'] = True
import matplotlib.pylab as plt
import datetime as dt
import matplotlib.pyplot as plt
from matplotlib.dates import DateFormatter
from numpy import random

myDates = [dt.datetime(2012,1,1) + dt.timedelta(days=i) for i in range(100)]
myValues = [random.randint(10) for _ in range(len(myDates))]
fig, ax = plt.subplots()
ax.plot(myDates,myValues)

myFmt = DateFormatter("%d %b %Y")
# myFmt = DateFormatter("%d\n%b\n%Y")  # Doesn't work.
ax.xaxis.set_major_formatter(myFmt)
fig.autofmt_xdate()
plt.show()

Output

raise RuntimeError(
RuntimeError: latex was not able to process the following string:
b'$\\\\mathdefault{01'
Here is the full report generated by latex:
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=latex)
 restricted \write18 enabled.
entering extended mode
...
! Missing $ inserted.
<inserted text> 
                $
l.20 \end{document}

Current hack

I can hack a semi-tolerable solution together with something like r'\underset{%Y}{\mathrm{%b}}', but this is really only a temporary fix and unideal.