Embedding Python in DLL: Access violation reading location when Py_DECREF list object

Question

I am trying to embed Python into an an XLL to allow Python functions to be called within Excel. An XLL is a DLL that also includes at a minimum 2 functions that tell Excel how to register or unregister the exported functions from the DLL, so this can be treated exactly as a traditional DLL. The issue I am having is when decreasing the reference count of a Python list object the program crashes with the following error:

Exception thrown at 0x1E14E37D (python37.dll) in EXCEL.EXE: 0xC0000005: Access violation reading location 0x00000064.

I do not have this issue when decreasing the reference count strings, floats, bool, etc. Only list and tuple object are giving me this issue.

Below I have made a simple example with 2 functions exposed to Excel, testFloat and testList. Both are made very simple to try and debug the issue, so the functions take no arguments and both return xltypenil to Excel, which will fill the cell with 0. Each function creates their own Python object (a float or a list) and then decrements the reference count.

#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include <Windows.h>
#include <XLCALL.H>
#include <FRAMEWRK.H>

LPCWCHAR uFuncs[][3]{
    {L"testFloat", L"Q", L"testFloat"},
    {L"testList", L"Q", L"testList"},
};

BOOL APIENTRY DllMain( HMODULE hModule,
                       DWORD  ul_reason_for_call,
                       LPVOID lpReserved
                     )
{
    switch (ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        break;
    }
    return TRUE;
}

int WINAPI xlAutoOpen(void) {
    if (!Py_IsInitialized()) {
        Py_InitializeEx(0);
    }

    // register functions with Excel
    XLOPER12 xDLL;
    Excel12f(xlGetName, &xDLL, 0);
    for (int i{ 0 }; i < sizeof(uFuncs) / sizeof(uFuncs[0]); i++) {
        Excel12f(xlfRegister, 0, 4,
            (LPXLOPER12)& xDLL,
            (LPXLOPER12)TempStr12(uFuncs[i][0]),
            (LPXLOPER12)TempStr12(uFuncs[i][1]),
            (LPXLOPER12)TempStr12(uFuncs[i][2])
        );
    }
    Excel12f(xlFree, &xDLL, 0);

    return 1;
}

int WINAPI xlAutoClose(void) {
    if (Py_IsInitialized()) {
        Py_FinalizeEx();
    }
    return 1;
}

LPXLOPER12 testFloat(void) {
    static XLOPER12 xRet;
    PyObject* obj{ PyFloat_FromDouble(2.5) };
    Py_DECREF(obj);
    xRet.xltype = xltypeNil;
    return &xRet;
}

LPXLOPER12 testList(void) {
    static XLOPER12 xRet;
    PyObject* obj{ Py_BuildValue("[dd]", 3.4, 4.5) };
    Py_DECREF(obj);  // <---- This is where the debugger says the error is
    xRet.xltype = xltypeNil;
    return &xRet;
}

Currently I am getting the following error:

Exception thrown at 0x1E14E37D (python37.dll) in EXCEL.EXE: 0xC0000005: Access violation reading location 0x00000064.

I am expecting this to run with no errors and return 0 to the Excel cell calling the function.

Answer 1

You need to acquire the GIL before you can safely do anything with the python C API. Quoting directly from the python api doc:

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

/* Perform Python actions here. */
result = CallSomeFunction();
/* evaluate result or handle exception */

/* Release the thread. No Python API allowed beyond this point. */
PyGILState_Release(gstate);

I'd suggest using pybind11 rather that tackling the python API directly.

Even further, I'd suggest either using or a least cribbing the code from my Excel/Python library which allows you to interface between Excel and python directly without needing to touch a C API!