Python in C DLL

Question

I just created a DLL in C with Python in it. When I export my function with python functions in it, I can't call it in my Python code But when I export a classic C function without Python code inside, it works perfectly. I don't understand

C DLL

#include <stdio.h>

#define PY_SSIZE_T_CLEAN
#include <Python.h>

__declspec(dllexport) PyObject* getList()
{
    PyObject *PList = PyList_New(0);
    PyList_Append(PList, Py_BuildValue("i", 1));

    return PList;
}

Python Code

import ctypes

lib = ctypes.cdll.LoadLibrary("EasyProtect.dll")

getList = lib.getList
getList.argtypes = None
getList.restype = ctypes.py_object

print(getList())

My Error

print(getList())
OSError: exception: access violation reading 0x0000000000000010

Answer 1

According to [Python.Docs]: class ctypes.PyDLL(name, mode=DEFAULT_MODE, handle=None) (emphasis is mine):

Instances of this class behave like CDLL instances, except that the Python GIL is not released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.

Thus, this is only useful to call Python C api functions directly.

So, you should replace cdll (CDLL) by pydll (PyDLL).
I enhanced your example a bit.

dll00.c:

#include <stdio.h>
#define PY_SSIZE_T_CLEAN
#include <Python.h>

#if defined(_WIN32)
#  define DLL00_EXPORT_API __declspec(dllexport)
#else
#  define DLL00_EXPORT_API
#endif


#if defined(__cplusplus)
extern "C" {
#endif

DLL00_EXPORT_API PyObject* createList(unsigned int size, int *pValues);

#if defined(__cplusplus)
}
#endif


PyObject* createList(unsigned int size, int *pValues)
{
    PyObject *pList = PyList_New(size);
    for (unsigned int i = 0; i < size; ++i) {
        PyList_SetItem(pList, i, Py_BuildValue("i", pValues[i]));
    }
    return pList;
}

code00.py:

#!/usr/bin/env python

import ctypes as ct
import sys


DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")
IntPtr = ct.POINTER(ct.c_int)


def main(*argv):
    
    dll00 = ct.PyDLL(DLL_NAME)
    create_list = dll00.createList
    create_list.argtypes = (ct.c_uint, IntPtr)
    create_list.restype = ct.py_object

    dim = 5
    int_arr = (ct.c_int * dim)(*range(dim, 0, -1))  # Intermediary step: create an array
    res = create_list(dim, ct.cast(int_arr, IntPtr))
    print("\n{:s} returned: {:}".format(create_list.__name__, res))


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.")
    sys.exit(rc)

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q072231434]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2019\VC\Auxiliary\Build\vcvarsall.bat" x64 > nul

[prompt]> dir /b
code00.py
dll00.c

[prompt]>
[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.09\include" dll00.c  /link /NOLOGO /DLL /OUT:dll00.dll /LIBPATH:"c:\Install\pc064\Python\Python\03.09\libs"
dll00.c
   Creating library dll00.lib and object dll00.exp

[prompt]>
[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.lib
dll00.obj

[prompt]>
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts\python.exe" code00.py
Python 3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)] 064bit on win32


createList returned: [5, 4, 3, 2, 1]

Done.

Answer 2

When you use the Python C API, the GIL (global interpreter lock) must be held. Use PyDLL for that. Load your DLL with:

lib = ctypes.PyDLL("EasyProtect.dll")

As an aside, your DLL has a reference leak. Py_BuildValue returns a new object, and PyList_Append increments the reference when adding it to the list. Py_DECREF should be called on the object returned by Py_BuildValue.

In this case, though, create the list with the size you want and use PyList_SET_ITEM which steals the new object's reference to initialize the list and is optimized for initializing new list items:

#include <stdio.h>

#define PY_SSIZE_T_CLEAN
#include <Python.h>

__declspec(dllexport) PyObject* getList()
{
    PyObject *PList = PyList_New(1);                # size 1 list
    PyList_SET_ITEM(PList, 0, PyLong_FromLong(1));  # assign offset 0
    return PList;
}