python requests can't find a folder with a certificate when converted to .exe

Question

I have a program that pools ad stats from different marketing systems. Everything works fine untill i convert it to the .exe format and run it.

Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\tkinter\__init__.py", line 1549, in __call__
    return self.func(*args)
  File "C:\Users\user\Desktop\alg\TSK_7. Marketing\report_gui.py", line 24, in <lambda>
    ok = tk.Button(root, text="DO NOT PRESS", bg="red", command=lambda: self.run())
  File "C:\Users\user\Desktop\alg\TSK_7. Marketing\report_gui.py", line 43, in run
    report.merge_all()
  File "C:\Users\user\Desktop\alg\TSK_7. Marketing\process_data.py", line 400, in merge_all
    fb_df     = self.fetch_fb()
  File "C:\Users\user\Desktop\alg\TSK_7. Marketing\process_data.py", line 156, in fetch_fb
    fb_campaigns = from_fb.run_fb(self.start_date, self.end_date)  # in JSON format
  File "C:\Users\user\Desktop\alg\TSK_7. Marketing\from_fb.py", line 110, in run_fb
    return s.get_stats()
  File "C:\Users\user\Desktop\alg\TSK_7. Marketing\from_fb.py", line 84, in get_stats
    params=params,
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\site-packages\facebookads\adobjects\adaccount.py", line 1551, in get_insights
    return request.execute()
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\site-packages\facebookads\api.py", line 653, in execute
    cursor.load_next_page()
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\site-packages\facebookads\api.py", line 797, in load_next_page
    params=self.params,
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\site-packages\facebookads\api.py", line 305, in call
    timeout=self._session.timeout
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\adapters.py", line 407, in send
    self.cert_verify(conn, request.url, verify, cert)
  File "C:\Users\user\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\adapters.py", line 226, in cert_verify
    "invalid path: {0}".format(cert_loc))
OSError: Could not find a suitable TLS CA certificate bundle, invalid path: C:\Users\user\AppData\Local\Temp\_MEI253762\facebookads\fb_ca_chain_bundle.crt

I tried to fix that by using this code but MEI folder keeps changing its digits everytime i run this code so it's no use.

dst = r'C:\Users\user\AppData\Local\Temp\_MEI120642\facebookads'
file = 'fb_ca_chain_bundle.crt'

try:
    os.makedirs(dst);  ## it creates the destination folder
except:
    pass

shutil.move(file, dst)

So i went to this file

C:\Users\user\AppData\Local\Programs\Python\Python35\Lib\site-packages\requests\adapters.py

and tried to comment if statements that raise this error out but got an SSL error. I couldn't find a piece of code responsible for generating those MEI digits.

def cert_verify(self, conn, url, verify, cert):
    """Verify a SSL certificate. This method should not be called from user
    code, and is only exposed for use when subclassing the
    :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.

    :param conn: The urllib3 connection object associated with the cert.
    :param url: The requested URL.
    :param verify: Either a boolean, in which case it controls whether we verify
        the server's TLS certificate, or a string, in which case it must be a path
        to a CA bundle to use
    :param cert: The SSL certificate to verify.
    """
    if url.lower().startswith('https') and verify:

        cert_loc = None

        # Allow self-specified cert location.
        if verify is not True:
            cert_loc = verify

        if not cert_loc:
            cert_loc = DEFAULT_CA_BUNDLE_PATH

        if not cert_loc or not os.path.exists(cert_loc):
            raise IOError("Could not find a suitable TLS CA certificate bundle, "
                          "invalid path: {0}".format(cert_loc))

        conn.cert_reqs = 'CERT_REQUIRED'

        if not os.path.isdir(cert_loc):
            conn.ca_certs = cert_loc
        else:
            conn.ca_cert_dir = cert_loc
    else:
        conn.cert_reqs = 'CERT_NONE'
        conn.ca_certs = None
        conn.ca_cert_dir = None

    if cert:
        if not isinstance(cert, basestring):
            conn.cert_file = cert[0]
            conn.key_file = cert[1]
        else:
            conn.cert_file = cert
            conn.key_file = None
        if conn.cert_file and not os.path.exists(conn.cert_file):
            raise IOError("Could not find the TLS certificate file, "
                          "invalid path: {0}".format(conn.cert_file))
        if conn.key_file and not os.path.exists(conn.key_file):
            raise IOError("Could not find the TLS key file, "
                          "invalid path: {0}".format(conn.key_file))

Answer 1

I ran into this problem as well. It looks like it comes from the certificate bundle cacert.pem not being included in the requests package directory when the program is compiled. The requests module uses the function certifi.core.where to determine the location of cacert.pem. Overriding this function and overriding the variables set by this function seems to fix the problem.

I added this code to the beginning of my program:

import sys, os


def override_where():
    """ overrides certifi.core.where to return actual location of cacert.pem"""
    # change this to match the location of cacert.pem
    return os.path.abspath("cacert.pem")


# is the program compiled?
if hasattr(sys, "frozen"):
    import certifi.core

    os.environ["REQUESTS_CA_BUNDLE"] = override_where()
    certifi.core.where = override_where

    # delay importing until after where() has been replaced
    import requests.utils
    import requests.adapters
    # replace these variables in case these modules were
    # imported before we replaced certifi.core.where
    requests.utils.DEFAULT_CA_BUNDLE_PATH = override_where()
    requests.adapters.DEFAULT_CA_BUNDLE_PATH = override_where()

Answer 2

This might be an issue with requests package.

I solved this by manually copying the cacert.pem file from /lib/site-packages/certifi to /lib/site-packages/requests

If you want to fix this issue with .exe, then copy cacert.pem file from /lib/site-packages/certifi to dist/library.zip/certifi/.

I am considering you have created exe using py2exe, where py2exe will create library.zip under dist/ which contains of all script dependencies. I don't know if other exe converters create library.zip.

Answer 3

I encountered this same problem with the requests package when converting to .exe with PyInstaller. I was developing with conda's virtual environment. I pivoted to using Python's virtual environment (as my colleague had done this and didn't get the same error), installed my packages as usual, created the .exe with PyInstaller and didn't encounter the same error.

I'm not sure if the cacert.pem file is correctly packaged into the .exe with the Python virtual environment but is somehow ignored with the conda virtual environment. Perhaps someone else can clarify why this was the case but I hope this helps!

Answer 4

I solved it by input the following code:

import os

import sys

import certifi

os.environ['REQUESTS_CA_BUNDLE'] = 
os.path.join(os.path.dirname(sys.argv[0]), certifi.where())

By this I detect the current path to "cacert.pem" and put it in environ

Answer 5

If you want to disable certificate verification, you can use the verify=False parameter in requests.get():

requests.get('https://example.com', verify=False)